Skip to content

Commit 2035fb3

Browse files
tasiogrtasiogr
committed
Initial commit
1 parent 02fc424 commit 2035fb3

File tree

11 files changed

+1251
-2
lines changed

11 files changed

+1251
-2
lines changed

‎README.md‎

Lines changed: 102 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,107 @@
1-
## Amazon Sagemaker Object Detection From Scratch
1+
## Amazon Sagemaker Object Detection From Scratch <!-- omit in toc -->
22

33
Build a Custom Object Detection Model from Scratch with Amazon SageMaker and Deploy it at the Edge with AWS DeepLens. This workshop explains how you can leverage DeepLens to capture data at the edge and build a training data set with Amazon SageMaker Ground Truth. Then, train an object detection model with Amazon SageMaker and deploy it to AWS DeepLens.
44

5+
This repository contains the code used during the namesake webinar broadcasted at the [AI & ML Web Day in July 11th 2019](https://pages.awscloud.com/EMEA-field-OE-ai-ml-web-day-2019-reg-event.html).
6+
7+
8+
## Table of Contents <!-- omit in toc -->
9+
- [Directory structure](#directory-structure)
10+
- [Prerequisites](#prerequisites)
11+
- [How to deploy the Lambda functions](#how-to-deploy-the-lambda-functions)
12+
- [Mac/Linux users](#maclinux-users)
13+
- [Windows users](#windows-users)
14+
- [How to run the trigger application](#how-to-run-the-trigger-application)
15+
- [Download the Greengrass Group CA certificate](#download-the-greengrass-group-ca-certificate)
16+
- [Run the trigger application](#run-the-trigger-application)
17+
- [License](#license)
18+
19+
## Directory structure
20+
```
21+
.
22+
├── LICENSE
23+
├── README.md
24+
├── deeplens # Contains the resources for functions to run on the AWS DeepLens device
25+
│ ├── deployment # Deployment templates and scripts
26+
│ │ ├── capture-template.yaml # SAM template to deploy the picture capturing function
27+
│ │ ├── deploy-capture-function.sh # Bash script to automate the deployment of the picture capturing function to AWS Lambda
28+
│ │ ├── deploy-inference-function.sh # Bash script to automate the deployment of the inference function to AWS Lambda
29+
│ │ └── inference-template.yaml # SAM template to deploy the picture inference function
30+
│ └── src # Source code of the functions
31+
│ ├── capture
32+
| | ├── capture.py # Source code for the picture capturing function
33+
| | └── requirements.txt # Dependencies for the picture capturing function
34+
│ └── inference
35+
| └── inference.py # Source code for the inference function
36+
├── notebook
37+
│ └── finger_counting.ipynb # Jupyter notebook to train and deploy the object detection model
38+
└── trigger # Contains the resources for the application that triggers the picture capture and validation
39+
├── certs # Directory to place the certificates for the AWS IoT Thing that represents the trigger
40+
└── src # Source code for the trigger application
41+
├── get_gg_ca_cert.py # Utility to download the root CA certificate of the Greengrass Group the DeepLens device belongs to
42+
└── trigger_app.py # Source code for the trigger application
43+
```
44+
## Prerequisites
45+
In order to run this solution you need:
46+
- Python 3.x
47+
- The [AWS CLI](https://aws.amazon.com/cli/) installed and configured with permissions to:
48+
- Create CloudFormation stacks.
49+
- Create an S3 bucket.
50+
- Create/update Lambda functions.
51+
- Attach an IAM policy to the DeepLensGreengrassGroupRole role.
52+
- List and get Greengrass groups certificate authorities.
53+
- [AWS SAM CLI installed](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)
54+
- An Amazon S3 bucket to which you have write access.
55+
56+
## How to deploy the Lambda functions
57+
Clone or download this Git repository to a local directory and follow the instructions below.
58+
59+
### Mac/Linux users
60+
1. Give execution permissions to the .sh files in the ./deeplens/deployment directory
61+
```
62+
chmod +x ./deeplens/deployment/*.sh
63+
```
64+
2. Run the following commands from the **deeplens/deployment** directory (substitute the arguments between <> with appropriate ones):
65+
66+
```./deploy-capture-function.sh --bucket <your-s3-bucket-name> --stack-name <your-stack-name>```
67+
```./deploy-inference-function.sh --bucket <your-s3-bucket-name> --stack-name <your-stack-name>```
68+
69+
70+
### Windows users
71+
72+
1. Open a terminal window
73+
1. Press windows key + r
74+
2. Introduce cmd and press enter
75+
2. Go to the **deeplens/deployment** folder and run the following commands:
76+
1. ```sam build -t capture-template.yaml```
77+
2. ```sam package --region us-east-1 --s3-bucket <your-s3-bucket> --output-template-file packaged-template.yaml;``` (substitute ```<your-s3-bucket>``` with the S3 bucket name where SAM can upload the package)
78+
3. ```sam deploy --region us-east-1 --template-file ./packaged-template.yaml --stack-name <your-stack-name> --capabilities CAPABILITY_NAMED_IAM;``` (substitute ```<your-stack-name>``` with an appropriate name for your CloudFormation Stack)
79+
80+
## How to run the trigger application
81+
82+
This application requires the following Python packages:
83+
- OpenCV (```pip install python-opencv```)
84+
- AWS IoT Device SDK for Python (```pip install awsiotpythonsdk```)
85+
- Numpy (```pip install numpy```)
86+
87+
Once you have registered an AWS IoT Thing, download the private key and the certificate and place them in the **trigger/certs** folder.
88+
Rename the private key file to ```private.pem.key``` and the certificate file to ```certificate.pem.crt```.
89+
90+
You also need the certificate of the root CA in the Greengrass group of your DeepLens.
91+
92+
### Download the Greengrass Group CA certificate
93+
94+
1. Go to the Greengrass Console and click on Groups. Click on the DeepLens group and then on Settings.
95+
2. Copy the Group ID.
96+
3. Open the *get_gg_ca_cert.py* file in your favorite text editor and substitute ```<YOUR_GREENGRASS_GROUP_ID>``` with the Group Id you copied.
97+
4. Go to the **trigger** directory and run ```python src/get_gg_ca_cert.py```
98+
99+
### Run the trigger application
100+
101+
1. Go to the **trigger/src** directory and run ```python trigger_app.py```
102+
2. The application will open a new window that may be hidden, so switch applications (alt+tab) and you'll see a Python application.
103+
3. Once in that window, click *'y'* to save a picture, *'q'* to quit the application, or any other key to ignore the current picture and request a new one.
104+
5105
## License
6106

7-
This library is licensed under the Apache 2.0 License.
107+
This library is licensed under the Apache 2.0 License.

‎deeplens/deployment/capture-template.yaml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
AWSTemplateFormatVersion: 2010-09-09
2+
3+
Transform: AWS::Serverless-2016-10-31
4+
Description: Deploys the Lambda function to capture pictures into S3
5+
6+
Resources:
7+
# This bucket hosts the pictures captured by the AWS DeepLens
8+
SageMakerFingerPics:
9+
Type: AWS::S3::Bucket
10+
11+
DeepLensCapture:
12+
Type: AWS::Serverless::Function
13+
Properties:
14+
Description: This function takes pictures from the AWS DeepLens and stores them in the referenced bucket in JPG format
15+
Handler: capture.function_handler
16+
Runtime: python2.7
17+
CodeUri: ../src/capture
18+
MemorySize: 256
19+
Timeout: 5
20+
Role: !Join ['', ['arn:aws:iam::', !Ref 'AWS::AccountId', ':role/service-role/AWSDeepLensLambdaRole']]
21+
AutoPublishAlias: prod
22+
Environment:
23+
Variables:
24+
BUCKET_NAME: !Ref SageMakerFingerPics
25+
26+
GreenGrassGroupRolePolicy:
27+
Type: AWS::IAM::ManagedPolicy
28+
Properties:
29+
Description: This policy allows the DeepLens Greengrass group to write on the bucket created and publish on a topic for results
30+
ManagedPolicyName: WriteToFingersBucket
31+
PolicyDocument:
32+
{
33+
"Version": "2012-10-17",
34+
"Statement": [
35+
{
36+
"Effect": "Allow",
37+
"Action": "s3:PutObject",
38+
"Resource": !Join ["/", [!GetAtt SageMakerFingerPics.Arn, "*"]]
39+
}
40+
]
41+
}
42+
Roles:
43+
- AWSDeepLensGreengrassGroupRole

‎deeplens/deployment/deploy-capture-function.sh‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
#!/bin/bash
2+
3+
# This script installs the dependencies for the Lambda for capturing and saving pictures function that will run on Greengrass,
4+
# then, it packages the dependencies together with the Lambda function code and deploys it to AWS with AWS CloudFormation.
5+
# It must be executed from the 'deployment' directory and receives two arguments: --bucket and --stack-name
6+
# Usage: ./deploy-capture-function.sh --bucket <bucketname> --stack-name <stackname>
7+
8+
HELP="You must include --bucket (S3 bucket name) and --stack-name (CloudFormation stack name)"
9+
10+
for i in "$@"
11+
do
12+
case $i in
13+
--bucket|-b|--bucket-name)
14+
BUCKETNAME="${2}"
15+
;;
16+
--stack-name|-s|--stack)
17+
STACKNAME="${1}"
18+
;;
19+
-h|--help)
20+
echo ${HELP}
21+
shift # past argument
22+
;;
23+
*) # unknown option
24+
POSITIONAL+=("$1") # save it in an array for later
25+
shift # past argument
26+
;;
27+
esac
28+
shift
29+
done
30+
31+
if [ -z "$BUCKETNAME" -o -z "$STACKNAME" ]; then
32+
echo ${HELP}
33+
else
34+
# Package the Lambda function
35+
if sam build -t capture-template.yaml; then
36+
# Transform the SAM template to an AWS CloudFormation template
37+
if sam package --region us-east-1 --s3-bucket "${BUCKETNAME}" --output-template-file packaged-template.yaml; then
38+
# Deploy the CloudFormation template
39+
sam deploy --region us-east-1 --template-file ./packaged-template.yaml --stack-name "${STACKNAME}" --capabilities CAPABILITY_NAMED_IAM;
40+
41+
# Remove the intermediate template
42+
rm ./packaged-template.yaml;
43+
fi
44+
fi
45+
fi

‎deeplens/deployment/deploy-inference-function.sh‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
#!/bin/bash
2+
3+
# This script installs the inference function for use within DeepLens.
4+
# Usage: ./deploy-inference-function.sh --bucket <bucketname> --stack-name <stackname>
5+
HELP="You must include --bucket (S3 bucket name) and --stack-name (CloudFormation stack name)"
6+
7+
for i in "$@"
8+
do
9+
case $i in
10+
--bucket|-b|--bucket-name)
11+
BUCKETNAME="${2}"
12+
;;
13+
--stack-name|-s|--stack)
14+
STACKNAME="${1}"
15+
;;
16+
-h|--help)
17+
echo ${HELP}
18+
shift # past argument
19+
;;
20+
*) # unknown option
21+
POSITIONAL+=("$1") # save it in an array for later
22+
shift # past argument
23+
;;
24+
esac
25+
shift
26+
done
27+
28+
if [ -z "$BUCKETNAME" -o -z "$STACKNAME" ]; then
29+
echo ${HELP}
30+
else
31+
# Package the Lambda function
32+
if sam build -t capture-template.yaml; then
33+
# Transform the SAM template to an AWS CloudFormation template
34+
if sam package --region us-east-1 --template-file ./inference-template.yaml --s3-bucket "${BUCKETNAME}" --output-template-file packaged-template.yaml; then
35+
# Deploy the CloudFormation template
36+
sam deploy --region us-east-1 --template-file ./packaged-template.yaml --stack-name "${STACKNAME}" --capabilities CAPABILITY_NAMED_IAM;
37+
38+
# Remove the intermediate template
39+
rm ./packaged-template.yaml;
40+
fi
41+
fi
42+
fi

‎deeplens/deployment/inference-template.yaml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
AWSTemplateFormatVersion: 2010-09-09
2+
3+
Transform: AWS::Serverless-2016-10-31
4+
Description: Deploys the Lambda function to run inferences at the edge
5+
6+
Resources:
7+
DeepLensInfer:
8+
Type: AWS::Serverless::Function
9+
Properties:
10+
Description: This function takes pictures from the AWS DeepLens and detects fingers by running an ML model
11+
Handler: inference.function_handler
12+
Runtime: python2.7
13+
CodeUri: ../src/inference
14+
MemorySize: 256
15+
Timeout: 5
16+
Role: !Join ['', ['arn:aws:iam::', !Ref 'AWS::AccountId', ':role/service-role/AWSDeepLensLambdaRole']]
17+
AutoPublishAlias: prod

‎deeplens/src/capture/capture.py‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
#*****************************************************
2+
# *
3+
# Copyright 2019 Amazon.com, Inc. or its affiliates. *
4+
# All Rights Reserved. *
5+
# *
6+
#*****************************************************
7+
8+
# capture.py
9+
# This Lambda function, when receiving a "capture" message, takes a picture using the DeepLens camera,
10+
# does some preprocessing, stores it in the /tmp directory with a unique id, and publishes it as a base64 encoded blob alongside its id.
11+
# If it receives a "save" message with a valid unique id, it uploads the file from the /tmp directory to an S3 bucket configured as an environment variable.
12+
13+
# This Lambda function requires the AWS Greengrass SDK to run on Greengrass devices. This can be found on the AWS IoT Console.
14+
# Additionally, this Lambda function needs the boto3 package to access S3.
15+
16+
from threading import Thread, Event
17+
import os
18+
import json
19+
import cv2
20+
import greengrasssdk
21+
import boto3
22+
import base64
23+
import time
24+
import random
25+
import uuid
26+
import traceback
27+
28+
# Create a greengrass core sdk client
29+
iotclient = greengrasssdk.client('iot-data')
30+
31+
# Get an object for the S3 bucket
32+
bucket_name = os.environ['BUCKET_NAME']
33+
s3 = boto3.resource('s3')
34+
bucket = s3.Bucket(bucket_name)
35+
36+
# List of valid resolutions
37+
RESOLUTION = {'1080p' : (1920, 1080), '720p' : (1280, 720), '480p' : (858, 480), 'training': (300, 300)}
38+
39+
def function_handler(event, context):
40+
# The event can have two actions: capture or save
41+
if event.get('action') == 'capture':
42+
capture()
43+
elif event.get('action') == 'save':
44+
pic_id = event.get('id')
45+
if pic_id:
46+
key = save_jpeg_to_s3(pic_id)
47+
iotclient.publish(
48+
topic='snap/keys',
49+
qos = 0,
50+
payload = json.dumps({ 'key': key }).encode())
51+
52+
def capture():
53+
thumbnail_topic = 'trigger/thumbnail'
54+
response = {}
55+
56+
cam = cv2.VideoCapture('/opt/awscam/out/ch2_out.mjpeg')
57+
58+
try:
59+
ret, frame = cam.read()
60+
except Exception as e:
61+
response['message'] = repr(e)
62+
63+
if ret:
64+
try:
65+
pic_id = str(uuid.uuid4())
66+
frame = crop_frame_square(frame)
67+
jpeg = convert_to_jpg(frame, RESOLUTION['training'])
68+
save_jpeg_to_temp(jpeg, pic_id)
69+
response = {
70+
'id': pic_id,
71+
'thumbnail': base64.b64encode(bytearray(get_thumbnail(frame)))
72+
}
73+
except Exception:
74+
response['message'] = traceback.format_exc()
75+
else:
76+
response['message'] = 'No picture :('
77+
78+
cam.release()
79+
80+
return iotclient.publish(
81+
topic = thumbnail_topic,
82+
qos = 0,
83+
payload = json.dumps(response).encode()
84+
)
85+
86+
# This function crops the frame to a square, as the training algorithm uses squared pictures
87+
# It does the crop from the center of the picture, taking the smaller side, which is the height
88+
def crop_frame_square(frame):
89+
width = frame.shape[1]
90+
height = frame.shape[0]
91+
92+
centre_x = int(width / 2)
93+
94+
xmin = centre_x - int(height / 2)
95+
xmax = centre_x + int(height / 2)
96+
97+
cropped = frame[0:height, xmin:xmax]
98+
99+
return cropped
100+
101+
def get_thumbnail(frame):
102+
return(cv2.imencode('.jpg', cv2.resize(frame, RESOLUTION['training']))[1])
103+
104+
def convert_to_jpg(frame, resolution):
105+
""" Converts the captured frame to the desired resolution
106+
"""
107+
ret, jpeg = cv2.imencode('.jpg', cv2.resize(frame, resolution))
108+
if not ret:
109+
raise Exception('Failed to set frame data')
110+
return jpeg
111+
112+
def save_jpeg_to_temp(jpeg, pic_id):
113+
file_name = '/tmp/{}'.format(pic_id)
114+
with open(file_name, 'wb') as f:
115+
f.write(jpeg)
116+
117+
def save_jpeg_to_s3(pic_id):
118+
file_name = '/tmp/{}'.format(pic_id)
119+
key = str(time.strftime("raw/%Y_%m_%d_%H_%M_%S.jpg")).format(random.randint(0, 999))
120+
121+
with open(file_name, 'rb') as data:
122+
bucket.upload_fileobj(
123+
Fileobj=data,
124+
Key = key,
125+
ExtraArgs = {
126+
'ContentType': 'image/jpeg',
127+
'ACL': 'bucket-owner-full-control'
128+
})
129+
130+
return key

0 commit comments

Comments
 (0)