diff --git a/README.md b/README.md index 51394e6..3b020e5 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,22 @@ AWS CloudFormation. You may find some of these CloudFormation resources obsolete as AWS team fills in the gaps. There is also some more complex ones, or developed to suite specific needs, such as copying s3 objects between buckets + +[Collection of custom resources for Node.js can be found here](https://github.com/base2Services/cloudformation-custom-resources-nodejs) + + +## Usage + +Clone the repository, install the dependencies, and zip up the folder as Lambda Deployment package. +Note that `boto3` is not packaged as dependency, as it is available in Lambda runtime itself by default. + +``` +git clone https://github.com/base2Services/cloudformation-custom-resources-python.git && \ +cd cloudformation-custom-resources-python && \ +docker run --rm -v $PWD:/src -w /src python:3.6 pip install -r requirements.txt -t . && \ +zip -r ~/cfn-ccr-python3.6.zip . +``` + ## Custom resources ### Creating CloudFormation stack in specific region @@ -14,9 +30,9 @@ It is easy to create sub-stacks in CloudFormation as long as they are in same re In some cases, there is need to create stack in region different than region where parent stack is being create, or for example, to create same stack in multiple regions. Such (sub)stack lifecycle can be controlled via custom resource having it's code in -`src/regional-cfn-stack` folder +`regional-cfn-stack` folder -handler: `src/regional-cfn-stack/handler.lambda_handler` +handler: `regional-cfn-stack/handler.lambda_handler` runtime: `python3.6` Required parameters: @@ -40,7 +56,7 @@ with `*.zip` extensions, this object will be unpacked before it's files are unpa Please note that this lambda function design does not include recursive calls if lambda is timing out, thus it does not permit mass file unpacking, but is rather designed for deployment of smaller files, such as client side web applications. -handler: `src/s3-copy/handler.lambda_handler` +handler: `3-copy/handler.lambda_handler` runtime: `python3.6` Required parameters: @@ -49,3 +65,121 @@ Required parameters: - `Destination` - Destination bucket and prefix in `s3://bucket-name/destination-prefix` format - `CannedAcl` - Canned ACL for created objects in destination No optional parameters. + +### Create Regex Waf Rules + +This custom resource allows create/update/delete match regex rules with regex a pattern set. + +handler: `waf_regex/handler.lambda_handler` +runtime: `python3.6` + +Required parameters: + +- `Name` - Custom Resource name. +- `RegexPatterns` - List format, regex pattern to match. +- `Type` - The part of the web request that you want AWS WAF to search for a specified string +- `Data` - Data such as when the value of Type is HEADER , enter the name of the header that you want AWS WAF to search, for example, User-Agent or Referer +- `Transform` - Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass AWS WAF. +Implementation require to be serialised with other waf condition. +### AmazonMQ Broker + +This custom resource creates a AmazonMQ broker instance. + +**NOTE:** This resource cannot be updated. If a change to the instance is required such as Instance Type, a new broker resource must be created. + +handler: `amazon-mq-broker/handler.lambda_handler` +runtime: `python3.6` + +Required parameters: + +- `Name` - Unique name given to the broker +- `SecurityGroups` - Array of security group ids +- `Subnets` - Array of subnets ids +- `MultiAZ` - String boolean [ 'true', 'false' ] +- `InstanceType` - valid values [ 'mq.t2.micro', 'mq.m4.large' ] +- `Username` - Username for the amq user +- `Password` - Password for the amq user. Must be 12-250 characters long + +No optional parameters. + +Returned Values: + +- `Active` - Active AmazonMQ endpoint +- `Stanby` - Standby AmazonMQ endpoint +- `BrokerId` - Id of the AmazonMQ Broker +- `BrokerArn` - Arn of the broker + +IAM Permissions: + +```json +{ + "Statement": + [ + { + "Effect": "Allow", + "Action": + [ + "mq:*", + "ec2:CreateNetworkInterface", + "ec2:CreateNetworkInterfacePermission", + "ec2:DeleteNetworkInterface", + "ec2:DeleteNetworkInterfacePermission", + "ec2:DetachNetworkInterface", + "ec2:DescribeInternetGateways", + "ec2:DescribeNetworkInterfaces", + "ec2:DescribeNetworkInterfacePermissions", + "ec2:DescribeRouteTables", + "ec2:DescribeSecurityGroups", + "ec2:DescribeSubnets", + "ec2:DescribeVpcs", + "logs:CreateLogGroup", + "logs:CreateLogStream", + "logs:PutLogEvents", + "lambda:InvokeFunction" + ], + "Resource": ["*"] + } + ] +} +``` + +### Auto generated secure ssm parameters + +This custom resource generates a random string `[a-z][A-Z][0-9]` a definable length. The string is then return to the cfn stack and can then be passed into other resources requiring a password. The resource can be updated generating a new password and updating the SSM parameter and returning the new password by passing a dummy parameter into the custom resource. + +handler: `ssm-secure-parameter/handler.lambda_handler` +runtime: `python3.6` + +Required parameters: + +- `Path` - SSM parameter path e.g. `/app/env/password` + +Optional parameters: + +- `Length` - Length of the auto generated password. Defaults to 16 characters + +Returned Values: + +- `Password` - The password generated by the resource + +IAM Permissions: + +```json +{ + "Statement": + [ + { + "Effect": "Allow", + "Action": + [ + "ssm:PutParameter", + "ssm:DeleteParameter", + "logs:CreateLogGroup", + "logs:CreateLogStream", + "logs:PutLogEvents" + ], + "Resource": ["*"] + } + ] +} +``` diff --git a/amazon-mq-broker/__init__.py b/amazon-mq-broker/__init__.py new file mode 100644 index 0000000..323c228 --- /dev/null +++ b/amazon-mq-broker/__init__.py @@ -0,0 +1 @@ +# package marker diff --git a/amazon-mq-broker/cr_response.py b/amazon-mq-broker/cr_response.py new file mode 100644 index 0000000..19fd431 --- /dev/null +++ b/amazon-mq-broker/cr_response.py @@ -0,0 +1,58 @@ +import logging +from urllib.request import urlopen, Request, HTTPError, URLError +import json + +logger = logging.getLogger() +logger.setLevel(logging.INFO) + + +class CustomResourceResponse: + def __init__(self, request_payload): + self.payload = request_payload + self.response = { + "StackId": request_payload["StackId"], + "RequestId": request_payload["RequestId"], + "LogicalResourceId": request_payload["LogicalResourceId"], + "Status": 'SUCCESS', + } + + def respond_error(self, message): + self.response['Status'] = 'FAILED' + self.response['Reason'] = message + self.respond() + + def respond(self, data=None): + event = self.payload + response = self.response + #### + #### copied from https://github.com/ryansb/cfn-wrapper-python/blob/master/cfn_resource.py + #### + + if event.get("PhysicalResourceId", False): + response["PhysicalResourceId"] = event["PhysicalResourceId"] + + if data is not None: + response['Data'] = data + + logger.debug("Received %s request with event: %s" % (event['RequestType'], json.dumps(event))) + + serialized = json.dumps(response) + logger.info(f"Responding to {event['RequestType']} request with: {serialized}") + + req_data = serialized.encode('utf-8') + + req = Request( + event['ResponseURL'], + data=req_data, + headers={'Content-Length': len(req_data),'Content-Type': ''} + ) + req.get_method = lambda: 'PUT' + + try: + urlopen(req) + logger.debug("Request to CFN API succeeded, nothing to do here") + except HTTPError as e: + logger.error("Callback to CFN API failed with status %d" % e.code) + logger.error("Response: %s" % e.reason) + except URLError as e: + logger.error("Failed to reach the server - %s" % e.reason) diff --git a/amazon-mq-broker/handler.py b/amazon-mq-broker/handler.py new file mode 100644 index 0000000..bc4c84f --- /dev/null +++ b/amazon-mq-broker/handler.py @@ -0,0 +1,71 @@ +import sys +import os +import json + +sys.path.append(f"{os.environ['LAMBDA_TASK_ROOT']}/lib") +sys.path.append(os.path.dirname(os.path.realpath(__file__))) + +import cr_response +import logic +import lambda_invoker + +def lambda_handler(event, context): + + print(f"Received event:{json.dumps(event)}") + + lambda_response = cr_response.CustomResourceResponse(event) + cr_params = event['ResourceProperties'] + + # Validate input + for key in ['MultiAZ', 'InstanceType', 'Username', 'Password', 'SecurityGroups', 'Subnets']: + if key not in cr_params: + lambda_response.respond_error(f"{key} property missing") + return + + try: + broker = logic.AmazonMQBrokerLogic(cr_params['Name']) + if event['RequestType'] == 'Create': + if ('WaitComplete' in event) and (event['WaitComplete']): + result = broker.wait_broker_status(event['PhysicalResourceId'], context) + + if result is None: + invoke = lambda_invoker.LambdaInvoker() + invoke.invoke(event) + elif result: + lambda_response.respond(data=event['Data']) + elif not result: + lambda_response.respond_error(f"Creation of AmazonMQ {event['PhysicalResourceId']} failed.") + + else: + response = broker.create( + multi_az=cr_params['MultiAZ'], + instance_type=cr_params['InstanceType'], + user=cr_params['Username'], + password=cr_params['Password'], + security_groups=cr_params['SecurityGroups'], + subnets=cr_params['Subnets'] + ) + + event['PhysicalResourceId'] = response['BrokerId'] + event['Data'] = response + event['WaitComplete'] = True + invoke = lambda_invoker.LambdaInvoker() + invoke.invoke(event) + + elif event['RequestType'] == 'Update': + comparision = broker.compare_broker_properites(event['PhysicalResourceId'], event['ResourceProperties']) + if not comparision: + lambda_response.respond_error("AmazonMQ resource cannot be updated. Create a new resource if changes are required.") + else: + response = broker.get_broker_data(event['PhysicalResourceId'], event['ResourceProperties']['MultiAZ']) + lambda_response.respond(data=response) + + elif event['RequestType'] == 'Delete': + broker.delete(event['PhysicalResourceId']) + lambda_response.respond() + + except Exception as e: + message = str(e) + lambda_response.respond_error(message) + + return 'OK' diff --git a/amazon-mq-broker/lambda_invoker.py b/amazon-mq-broker/lambda_invoker.py new file mode 100644 index 0000000..1fdf724 --- /dev/null +++ b/amazon-mq-broker/lambda_invoker.py @@ -0,0 +1,20 @@ +import boto3 +import os +import json + + +class LambdaInvoker: + def __init__(self): + print(f"Initialize lambda invoker") + + def invoke(self, payload): + bytes_payload = bytearray() + bytes_payload.extend(map(ord, json.dumps(payload))) + function_name = os.environ['AWS_LAMBDA_FUNCTION_NAME'] + function_payload = bytes_payload + client = boto3.client('lambda') + client.invoke( + FunctionName=function_name, + InvocationType='Event', + Payload=function_payload + ) diff --git a/amazon-mq-broker/logic.py b/amazon-mq-broker/logic.py new file mode 100644 index 0000000..824930c --- /dev/null +++ b/amazon-mq-broker/logic.py @@ -0,0 +1,103 @@ +import boto3 +import os +import time + +class AmazonMQBrokerLogic: + + def __init__(self, broker_name): + self.broker_name = broker_name + + def create(self, multi_az, instance_type, user, password, security_groups, subnets): + print(f"Creating AMQ instance {self.broker_name}") + deployment_mode = "ACTIVE_STANDBY_MULTI_AZ" if multi_az.lower() == "true" else "SINGLE_INSTANCE" + + client = boto3.client('mq') + response = client.create_broker( + AutoMinorVersionUpgrade=False, + BrokerName=self.broker_name, + DeploymentMode=deployment_mode, + EngineType='ACTIVEMQ', + EngineVersion='5.15.0', + HostInstanceType=instance_type, + PubliclyAccessible=False, + SecurityGroups=security_groups, + SubnetIds=subnets, + Users=[ + { + 'ConsoleAccess': True, + 'Password': password, + 'Username': user + } + ] + ) + print(f"Broker Id: {response['BrokerId']} Broker Arn: {response['BrokerArn']}") + active = self.endpoint(response['BrokerId'],1) + response.update({'Active': active}) + + standby = self.endpoint(response['BrokerId'],2) if multi_az.lower() == "true" else "NONE" + response.update({'Standby': standby}) + + print(f"Creating Amazon MQ instance\n{response}") + return response + + def wait_broker_status(self, id, lambda_context): + client = boto3.client('mq') + + while True: + response = client.describe_broker(BrokerId=id) + state = response['BrokerState'] + + print(f"Broker state: {state}") + if state == 'RUNNING': + print(f"Matched {state} - OK ") + return True + elif state == 'CREATION_FAILED': + print(f"Matched {state} - ERROR ") + return False + elif lambda_context.get_remaining_time_in_millis() < 10000: + print(f"Less than 10 seconds left of Lambda execution time, exiting with empty hands") + return None + else: + print(f"Waiting for 5 seconds, time remaining" + + f"in this lambda execution {lambda_context.get_remaining_time_in_millis()}ms") + time.sleep(5) + + def compare_broker_properites(self, id, properties): + client = boto3.client('mq') + response = client.describe_broker(BrokerId=id) + + deployment_mode = "ACTIVE_STANDBY_MULTI_AZ" if properties['MultiAZ'].lower() == "true" else "SINGLE_INSTANCE" + + if (properties['SecurityGroups'] == response['SecurityGroups']) and \ + (properties['Subnets'] == response['SubnetIds']) and \ + (properties['InstanceType'] == response['HostInstanceType']) and \ + (deployment_mode == response['DeploymentMode']) and \ + (properties['Name'] == response['BrokerName']): + return True + else: + return False + + def get_broker_data(self, id, multi_az): + data = {} + client = boto3.client('mq') + response = client.describe_broker(BrokerId=id) + + data.update({'BrokerId': response['BrokerId']}) + data.update({'BrokerArn': response['BrokerArn']}) + + active = self.endpoint(response['BrokerId'],1) + data.update({'Active': active}) + + standby = self.endpoint(response['BrokerId'],2) if multi_az.lower() == "true" else "NONE" + data.update({'Standby': standby}) + + return data + + def delete(self,id): + client = boto3.client('mq') + client.delete_broker( + BrokerId=id + ) + + def endpoint(self,id,n): + return f"{id}-{n}.mq.{os.environ['AWS_REGION']}.amazonaws.com" diff --git a/ssm-secure-parameter/__init__.py b/ssm-secure-parameter/__init__.py new file mode 100644 index 0000000..323c228 --- /dev/null +++ b/ssm-secure-parameter/__init__.py @@ -0,0 +1 @@ +# package marker diff --git a/ssm-secure-parameter/cr_response.py b/ssm-secure-parameter/cr_response.py new file mode 100644 index 0000000..16ded13 --- /dev/null +++ b/ssm-secure-parameter/cr_response.py @@ -0,0 +1,62 @@ +import logging +from urllib.request import urlopen, Request, HTTPError, URLError +import json + +logger = logging.getLogger() +logger.setLevel(logging.INFO) + + +class CustomResourceResponse: + def __init__(self, request_payload): + self.payload = request_payload + self.response = { + "StackId": request_payload["StackId"], + "RequestId": request_payload["RequestId"], + "LogicalResourceId": request_payload["LogicalResourceId"], + "NoEcho": True, + "Status": 'SUCCESS', + } + + def respond_error(self, message): + self.response['Status'] = 'FAILED' + self.response['Reason'] = message + self.respond() + + def respond(self, data=None, NoEcho=False): + event = self.payload + response = self.response + #### + #### copied from https://github.com/ryansb/cfn-wrapper-python/blob/master/cfn_resource.py + #### + + if event.get("PhysicalResourceId", False): + response["PhysicalResourceId"] = event["PhysicalResourceId"] + + if data is not None: + response['Data'] = data + + logger.debug("Received %s request with event: %s" % (event['RequestType'], json.dumps(event))) + + if NoEcho: + response['NoEcho'] = NoEcho + + serialized = json.dumps(response) + logger.info(f"Responding to {event['RequestType']} request with: {serialized}") + + req_data = serialized.encode('utf-8') + + req = Request( + event['ResponseURL'], + data=req_data, + headers={'Content-Length': len(req_data),'Content-Type': ''} + ) + req.get_method = lambda: 'PUT' + + try: + urlopen(req) + logger.debug("Request to CFN API succeeded, nothing to do here") + except HTTPError as e: + logger.error("Callback to CFN API failed with status %d" % e.code) + logger.error("Response: %s" % e.reason) + except URLError as e: + logger.error("Failed to reach the server - %s" % e.reason) diff --git a/ssm-secure-parameter/handler.py b/ssm-secure-parameter/handler.py new file mode 100644 index 0000000..ba21b42 --- /dev/null +++ b/ssm-secure-parameter/handler.py @@ -0,0 +1,60 @@ +import sys +import os + +sys.path.append(f"{os.environ['LAMBDA_TASK_ROOT']}/lib") +sys.path.append(os.path.dirname(os.path.realpath(__file__))) + +import cr_response +import logic +import json + +def lambda_handler(event, context): + + print(f"Received event:{json.dumps(event)}") + + lambda_response = cr_response.CustomResourceResponse(event) + cr_params = event['ResourceProperties'] + print(f"Resource Properties {cr_params}") + # Validate input + for key in ['Path']: + if key not in cr_params: + lambda_response.respond_error(f"{key} property missing") + return + + try: + parameter = logic.SSMSecureParameterLogic(cr_params['Path']) + length = 16 or cr_params['Length'] + + if event['RequestType'] == 'Create': + password, version = parameter.create( + length=length, + update=False + ) + + event['PhysicalResourceId'] = cr_params['Path'] + lambda_response.respond(data={ + "Password": password, + "Version": version + }) + + elif event['RequestType'] == 'Update': + password, version = parameter.create( + length=length, + update=True + ) + + event['PhysicalResourceId'] = cr_params['Path'] + lambda_response.respond(data={ + "Password": password, + "Version": version + }) + + elif event['RequestType'] == 'Delete': + parameter.delete() + lambda_response.respond() + + except Exception as e: + message = str(e) + lambda_response.respond_error(message) + + return 'OK' diff --git a/ssm-secure-parameter/logic.py b/ssm-secure-parameter/logic.py new file mode 100644 index 0000000..be6dc35 --- /dev/null +++ b/ssm-secure-parameter/logic.py @@ -0,0 +1,29 @@ +import boto3 +import string +import random + +class SSMSecureParameterLogic: + + def __init__(self, path): + self.path = path + + def create(self, length, update): + password = self.generate_password(length) + client = boto3.client('ssm') + response = client.put_parameter( + Name=self.path, + Value=password, + Overwrite=update, + Type='SecureString' + ) + return password, response['Version'] + + def delete(self): + client = boto3.client('ssm') + client.delete_parameter( + Name=self.path + ) + + def generate_password(self, length): + print(f"Generating a new password {length} chars long") + return ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(length)) diff --git a/waf_regex/__init__.py b/waf_regex/__init__.py new file mode 100644 index 0000000..323c228 --- /dev/null +++ b/waf_regex/__init__.py @@ -0,0 +1 @@ +# package marker diff --git a/waf_regex/cr_response.py b/waf_regex/cr_response.py new file mode 100644 index 0000000..a1aa830 --- /dev/null +++ b/waf_regex/cr_response.py @@ -0,0 +1,58 @@ +import logging +from urllib.request import urlopen, Request, HTTPError, URLError +import json + +logger = logging.getLogger() +logger.setLevel(logging.INFO) + +class CustomResourceResponse: + def __init__(self, request_payload): + self.payload = request_payload + self.response = { + "StackId": request_payload["StackId"], + "RequestId": request_payload["RequestId"], + "LogicalResourceId": request_payload["LogicalResourceId"], + "Status": 'SUCCESS', + } + + + def respond_error(self, message): + self.response['Status'] = 'FAILED' + self.response['Reason'] = message + self.respond({}) + + def respond(self, data): + event = self.payload + response = self.response + #### + #### copied from https://github.com/ryansb/cfn-wrapper-python/blob/master/cfn_resource.py + #### + + if event.get("PhysicalResourceId", False): + response["PhysicalResourceId"] = event["PhysicalResourceId"] + + logger.debug("Received %s request with event: %s" % + (event['RequestType'], json.dumps(event))) + + response["Data"] = data + + serialized = json.dumps(response) + + logger.info(f"Responding to {event['RequestType']} request with: {serialized}") + req_data = serialized.encode('utf-8') + + req = Request( + event['ResponseURL'], + data=req_data, + headers={'Content-Length': len(req_data), 'Content-Type': ''} + ) + req.get_method = lambda: 'PUT' + + try: + urlopen(req) + logger.debug("Request to CFN API succeeded, nothing to do here") + except HTTPError as e: + logger.error("Callback to CFN API failed with status %d" % e.code) + logger.error("Response: %s" % e.reason) + except URLError as e: + logger.error("Failed to reach the server - %s" % e.reason) diff --git a/waf_regex/handler.py b/waf_regex/handler.py new file mode 100644 index 0000000..7ec0d8b --- /dev/null +++ b/waf_regex/handler.py @@ -0,0 +1,49 @@ +import sys +import os +import re +import random + +sys.path.append(f"{os.environ['LAMBDA_TASK_ROOT']}/lib") +sys.path.append(os.path.dirname(os.path.realpath(__file__))) + +import cr_response +from logic import WafRegexLogic +import json + +def lambda_handler(event, context): + + print(f"Received event:{json.dumps(event)}") + + lambda_response = cr_response.CustomResourceResponse(event) + cr_params = event['ResourceProperties'] + waf_logic = WafRegexLogic(cr_params) + try: + # if create request, generate physical id, both for create/update copy files + if event['RequestType'] == 'Create': + print("Create request") + event['PhysicalResourceId'] = waf_logic.new_match_set() + data = { + "MatchID" : event['PhysicalResourceId'] + } + lambda_response.respond(data) + + elif event['RequestType'] == 'Update': + print("Update request") + waf_logic.update_match_set(event['PhysicalResourceId']) + data = { + "MatchID" : event['PhysicalResourceId'] + } + lambda_response.respond(data) + + elif event['RequestType'] == 'Delete': + print(event['PhysicalResourceId']) + waf_logic.remove_match_set(event['PhysicalResourceId']) + print("Delete request") + data = { } + lambda_response.respond(data) + + except Exception as e: + message = str(e) + lambda_response.respond_error(message) + + return 'OK' diff --git a/waf_regex/logic.py b/waf_regex/logic.py new file mode 100644 index 0000000..1a97266 --- /dev/null +++ b/waf_regex/logic.py @@ -0,0 +1,160 @@ +import boto3 +import os +import glob +import logging +import pprint + + +logger = logging.getLogger() +logger.setLevel(logging.INFO) + + +class WafRegexLogic: + + def __init__(self, resource_properties): + self.regex_patterns = resource_properties['RegexPatterns'] + self.match_type = resource_properties['Type'] + self.match_data = resource_properties['Data'] + self.transform = resource_properties['Transform'] + self.match_name = resource_properties['Name'] + self.pattern_name = f"{resource_properties['Name']}-pattern" + self.client = boto3.client('waf-regional') + + def new_pattern_set(self): + changeToken = self.client.get_change_token() + response_create_pattern_set = self.client.create_regex_pattern_set( + Name=self.pattern_name, + ChangeToken=changeToken['ChangeToken'] + ) + for pattern in self.regex_patterns: + self.insert_pattern_set(response_create_pattern_set['RegexPatternSet']['RegexPatternSetId'], pattern) + return response_create_pattern_set['RegexPatternSet']['RegexPatternSetId'] + + def remove_pattern_set(self,pattern_set_id): + pattern_set_object = self.get_pattern_set(pattern_set_id) + for pattern_set_string in pattern_set_object['RegexPatternSet']['RegexPatternStrings']: + self.delete_pattern_set(pattern_set_id, pattern_set_string) + changeToken = self.client.get_change_token() + response = self.client.delete_regex_pattern_set( + RegexPatternSetId=pattern_set_id, + ChangeToken=changeToken['ChangeToken'] + ) + + def get_pattern_set(self,pattern_set_id): + pattern_set_object = self.client.get_regex_pattern_set( + RegexPatternSetId=pattern_set_id + ) + return pattern_set_object + + + def update_pattern_set(self,pattern_set_id): + pattern_set_object = self.get_pattern_set(pattern_set_id) + #delete existing and add a new one + for pattern_set_string in pattern_set_object['RegexPatternSet']['RegexPatternStrings']: + self.delete_pattern_set(pattern_set_id, pattern_set_string) + for pattern in self.regex_patterns: + self.insert_pattern_set(pattern_set_id, pattern) + + + def insert_pattern_set(self,pattern_set_id, pattern_set_string): + changeToken = self.client.get_change_token() + update_regex_patternset = self.client.update_regex_pattern_set( + RegexPatternSetId=pattern_set_id, + Updates=[ + { + 'Action': 'INSERT', + 'RegexPatternString': pattern_set_string + }, + ], + ChangeToken=changeToken['ChangeToken'] + ) + + def delete_pattern_set(self,pattern_set_id, pattern_set_string): + changeToken = self.client.get_change_token() + update_regex_patternset = self.client.update_regex_pattern_set( + RegexPatternSetId=pattern_set_id, + Updates=[ + { + 'Action': 'DELETE', + 'RegexPatternString': pattern_set_string + }, + ], + ChangeToken=changeToken['ChangeToken'] + ) + + # + # Match Sets + # + + def new_match_set(self): + changeToken = self.client.get_change_token() + #create match set + response_create_match_set = self.client.create_regex_match_set( + Name=self.match_name, + ChangeToken=changeToken['ChangeToken'] + ) + #create pattern set + pattern_set_id = self.new_pattern_set() + self.insert_match_set(response_create_match_set['RegexMatchSet']['RegexMatchSetId'], pattern_set_id) + return response_create_match_set['RegexMatchSet']['RegexMatchSetId'] + + def insert_match_set(self, match_set_id, pattern_set_id): + changeToken = self.client.get_change_token() + update_regex_matchset = self.client.update_regex_match_set( + RegexMatchSetId=match_set_id, + Updates=[ + { + 'Action': 'INSERT', + 'RegexMatchTuple': { + 'FieldToMatch': { + 'Type': self.match_type, + 'Data': self.match_data + }, + 'TextTransformation': self.transform, + 'RegexPatternSetId': pattern_set_id + } + }, + ], + ChangeToken=changeToken['ChangeToken'] + ) + + def update_match_set(self,match_set_id): + match_set_object = self.get_match_set(match_set_id) + for match_tuple in match_set_object['RegexMatchSet']['RegexMatchTuples']: + self.update_pattern_set(match_tuple['RegexPatternSetId']) + self.delete_match_set(match_set_id,match_tuple) + self.insert_match_set(match_set_id,match_tuple['RegexPatternSetId']) + + + def get_match_set(self,match_set_id): + match_set_object = self.client.get_regex_match_set( + RegexMatchSetId=match_set_id + ) + return match_set_object + + def remove_match_set(self,match_set_id): + match_set_object = self.get_match_set(match_set_id) + + for match_tuple in match_set_object['RegexMatchSet']['RegexMatchTuples']: + if bool(match_tuple): + self.delete_match_set(match_set_id,match_tuple) + self.remove_pattern_set(match_tuple['RegexPatternSetId']) + + changeToken = self.client.get_change_token() + response = self.client.delete_regex_match_set( + RegexMatchSetId=match_set_id, + ChangeToken=changeToken['ChangeToken'] + ) + + def delete_match_set(self,match_set_id,match_tuple): + changeToken = self.client.get_change_token() + update_regex_matchset = self.client.update_regex_match_set( + RegexMatchSetId=match_set_id, + Updates=[ + { + 'Action': 'DELETE', + 'RegexMatchTuple': match_tuple + }, + ], + ChangeToken=changeToken['ChangeToken'] + )