feat: upload the scripts

Author: vickzkater

.gitignore

@@ -0,0 +1,74 @@
+# Node artifact files
+node_modules/
+
+# Compiled Java class files
+*.class
+
+# Compiled Python bytecode
+*.py[cod]
+
+# Log files
+*.log
+
+# Package files
+*.jar
+
+# Maven
+target/
+
+# JetBrains IDE
+.idea/
+
+# Unit test reports
+TEST*.xml
+
+# Generated by MacOS
+.DS_Store
+
+# Generated by Windows
+Thumbs.db
+
+# Applications
+*.app
+*.exe
+*.war
+
+# Large media files
+*.mp4
+*.tiff
+*.avi
+*.flv
+*.mov
+*.wmv
+
+# Laravel
+/node_modules
+/public/build
+/public/hot
+/public/storage
+/storage/*.key
+/vendor
+.env
+.env.backup
+.env.production
+.phpunit.result.cache
+Homestead.json
+Homestead.yaml
+auth.json
+npm-debug.log
+yarn-error.log
+/.fleet
+/.idea
+/.vscode
+
+# This Project
+/storage/debugbar
+/etc
+/public/uploads/avatar
+/public/uploads/banner
+/public/uploads/config
+/public/uploads/form
+/public/uploads/office
+/public/uploads/page
+/public/uploads/social_media
+/public/uploads/tmp

README.md

@@ -0,0 +1,159 @@
+# Example python script hosted on AWS Lambda #
+
+## What is this? ##
+
+Python script running on AWS Lambda with functions:
+
+- Connect to PostgreSQL database
+- Generate CSV files
+- Upload the CSV files to AWS S3
+
+Developed in Dec 2024.
+
+## Requirements ##
+
+- AWS account
+- AWS Access Key ID (can be obtained at IAM > Users)
+- AWS Secret Access Key (can be obtained at IAM > Users)
+- Region (e.g. ap-southeast-1)
+- Role / ARN (e.g. arn:aws:iam::123456789012:role/lambda-role)
+- DB_HOST
+- DB_PORT
+- DB_NAME
+- DB_USER
+- DB_PASSWORD
+- S3_BUCKET
+- S3_KEY_PREFIX (e.g. reports)
+
+## Installation ##
+
+This project utilizes `awscli` to configure AWS, manage function code and function configuration. So, before clone this project, make sure you have `awscli` installed on your machine.
+
+```bash
+# Install the AWS CLI
+pip install awscli
+
+# Verify the awscli installation
+aws --version
+```
+
+Then configure the AWS account.
+
+```bash
+aws configure
+```
+
+Input your details and input `json` as default output format.
+
+### 1) Clone this project ###
+
+You may use this script by clone this project using git command or download as .zip file then extract it.
+
+### 2) Understand the project structure ###
+
+```text
+This Project
+│
+├── package/                    # folder to be uploaded to AWS Lambda
+│   ├── boto3
+│   ├── lambda_function.py      # python script that we created to run on Lambda
+│   ├── psycopg2
+│   ├── requirements.txt        # contains dependencies - install it using the command 
+│   │                           # "pip install -r requirements.txt -t ."
+│   ├── ... (other dependencies)
+│
+├── event.json                  # used for testing running scripts
+└── README.md
+```
+
+### 3) Prepare the ZIP file to be uploaded to AWS Lambda ###
+
+After prepare the dependencies and the main script, compress it to .zip file
+
+```bash
+# make sure you are in the "package" directory
+cd package
+
+# zip all the files
+zip -r ../lambda_function.zip .
+```
+
+### 4) Create a new function in Lambda ###
+
+For creating a new function in Lambda, we must have `AWS Role/ARN` (for creating Role/ARN, please see "Create AWS New Role/ARN" section below)
+
+```bash
+# make sure you are outside the "package" directory, you are in the root directory of this project
+cd ..
+
+# create a new Lambda function
+aws lambda create-function \
+    --function-name GenerateReports \
+    --runtime python3.11 \
+    --role arn:aws:iam::123456789012:role/lambda-role \
+    --handler lambda_function.lambda_handler \
+    --zip-file fileb://lambda_function.zip
+```
+
+Note:
+
+- `GenerateReports` is function name
+- `python3.11` is runtime Python version
+- `role` is your AWS role/ARN
+- `handler` is your function name
+- `zip-file` is your ZIP file
+
+### 5) Set ENV in Lambda ###
+
+```bash
+aws lambda update-function-configuration \
+    --function-name GenerateReports \
+    --environment "Variables={DB_HOST=localhost,DB_PORT=5432,DB_NAME=xxx_db,DB_USER=user,DB_PASSWORD=xxx,S3_BUCKET=bucket-name,S3_KEY_PREFIX=reports}"
+```
+
+### 6) Update configuration in Lambda ###
+
+```bash
+aws lambda update-function-configuration \
+    --function-name GenerateReports \
+    --timeout 900
+```
+
+### 7) Running test ###
+
+```bash
+aws lambda invoke \
+    --function-name GenerateReports \
+    --payload file://event.json \
+    response.json
+```
+
+It will generate `response.json`, then you can check the response details.
+
+### 8) Update code (if needed) ###
+
+```bash
+aws lambda update-function-code \
+    --function-name GenerateReports \
+    --zip-file fileb://lambda_function.zip
+```
+
+## Dependecies ##
+
+- psycopg2-binary
+- boto3
+
+## *IMPORTANT NOTE ##
+
+We need `psycopg2-binary` for connect to PostgreSQL, to install it - please use from [https://github.com/jkehler/awslambda-psycopg2](https://github.com/jkehler/awslambda-psycopg2). Clone it first, then copy the files based on your Python version (in this project we will use Python 3.11) then paste it into the “package” directory.
+
+## Create AWS New Role/ARN ##
+
+1. Login to AWS Management Console
+2. Go to IAM service
+3. Choose “Roles” on the sidebar menu, then click “Create role”
+4. Choose “AWS Service” as trusted entity
+5. Choose “Lambda” from service list, then click “Next”
+6. For Permissions, add ”AmazonS3FullAccess” so this role can access to S3, then click “Next”
+7. Input the role name, like “lambda-s3-role”
+8. Click “Create role”

event.json

@@ -0,0 +1 @@
+{}

package/bin/jp.py

@@ -0,0 +1,54 @@
+#!/Library/Frameworks/Python.framework/Versions/3.11/bin/python3
+
+import sys
+import json
+import argparse
+from pprint import pformat
+
+import jmespath
+from jmespath import exceptions
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('expression')
+    parser.add_argument('-f', '--filename',
+                        help=('The filename containing the input data.  '
+                              'If a filename is not given then data is '
+                              'read from stdin.'))
+    parser.add_argument('--ast', action='store_true',
+                        help=('Pretty print the AST, do not search the data.'))
+    args = parser.parse_args()
+    expression = args.expression
+    if args.ast:
+        # Only print the AST
+        expression = jmespath.compile(args.expression)
+        sys.stdout.write(pformat(expression.parsed))
+        sys.stdout.write('\n')
+        return 0
+    if args.filename:
+        with open(args.filename, 'r') as f:
+            data = json.load(f)
+    else:
+        data = sys.stdin.read()
+        data = json.loads(data)
+    try:
+        sys.stdout.write(json.dumps(
+            jmespath.search(expression, data), indent=4, ensure_ascii=False))
+        sys.stdout.write('\n')
+    except exceptions.ArityError as e:
+        sys.stderr.write("invalid-arity: %s\n" % e)
+        return 1
+    except exceptions.JMESPathTypeError as e:
+        sys.stderr.write("invalid-type: %s\n" % e)
+        return 1
+    except exceptions.UnknownFunctionError as e:
+        sys.stderr.write("unknown-function: %s\n" % e)
+        return 1
+    except exceptions.ParseError as e:
+        sys.stderr.write("syntax-error: %s\n" % e)
+        return 1
+
+
+if __name__ == '__main__':
+    sys.exit(main())

package/boto3-1.35.83.dist-info/INSTALLER

@@ -0,0 +1 @@
+pip