S3への画像アップロードをトリガーに、それを加工するようなLambdaを1コマンドでつくれるやつです！
- 通知を設定する方法
- Lambdaで外部ライブラリを使う方法
- SAMでLambdaをデプロイする方法
あたりの情報がひとつにまとまった記事が見つけられなかったので書きます。
ファイル準備
ディレクトリ作成
mkdir sam-lambda-s3
ファイルとか作成
touch template.yaml mkdir function python touch function/function.py function/requirements.txt
template.yml(CloudFormationのやつ)
概ね公式の内容通りですが、一部調整してます。
CloudFormation を使用して、既存の S3 バケットで Lambda 用の Amazon S3 通知設定を作成する方法を教えてください。 https://aws.amazon.com/jp/premiumsupport/knowledge-center/cloudformation-s3-notification-lambda/
先に内容の概要を紹介
S3NotificationLambdaFunction
- S3からの通知を受け取るLambda
- 処理自体はfunction/function.py
LibraryLayer
- S3NotificationLambdaFunctionで使うライブラリを置いておくLambdaLayer
- python/ 以下がデプロイされるように設定している
LambdaInvokePermission
- S3NotificationLambdaFunctionのPermission
LambdaIAMRole
- S3NotificationLambdaFunctionのIAMロール
CustomResourceLambdaFunction
- S3通知を設定するLambda
LambdaTrigger
- S3通知の設定
template.yaml
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'
Description: S3から通知を受けるLambdaを作成する
Parameters:
NotificationBucket:
Type: String
Description: 通知を設定するS3バケット名
S3NotificationLambdaFunctionName:
Type: String
Description: S3からの通知を受ける関数名
Prefix:
Type: String
Description: 通知対象のファイルのPrefix(フォルダ等)
Resources:
S3NotificationLambdaFunction:
Type: 'AWS::Serverless::Function'
Properties:
FunctionName: !Ref S3NotificationLambdaFunctionName
CodeUri: function/
Handler: function.lambda_handler
Role: !GetAtt LambdaIAMRole.Arn
Runtime: python3.9
Timeout: 5
Layers:
- !Ref LibraryLayer
LibraryLayer:
Type: "AWS::Serverless::LayerVersion"
Properties:
LayerName: PythonLibraryLayer
ContentUri: python/
CompatibleRuntimes:
- python3.9
RetentionPolicy: Delete
LambdaInvokePermission:
Type: 'AWS::Lambda::Permission'
Properties:
FunctionName: !GetAtt S3NotificationLambdaFunction.Arn
Action: 'lambda:InvokeFunction'
Principal: s3.amazonaws.com
SourceAccount: !Ref 'AWS::AccountId'
SourceArn: !Sub 'arn:aws:s3:::${NotificationBucket}'
LambdaIAMRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/AmazonS3FullAccess'
- 'arn:aws:iam::aws:policy/AmazonRekognitionFullAccess'
Policies:
- PolicyName: root
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 's3:GetBucketNotification'
- 's3:PutBucketNotification'
Resource: !Sub 'arn:aws:s3:::${NotificationBucket}'
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource: 'arn:aws:logs:*:*:*'
CustomResourceLambdaFunction:
Type: 'AWS::Lambda::Function'
Properties:
Handler: index.lambda_handler
Role: !GetAtt LambdaIAMRole.Arn
Code:
ZipFile: |
from __future__ import print_function
import json
import boto3
import cfnresponse
SUCCESS = "SUCCESS"
FAILED = "FAILED"
print('Loading function')
s3 = boto3.resource('s3')
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
responseData={}
try:
if event['RequestType'] == 'Delete':
print("Request Type:",event['RequestType'])
Bucket=event['ResourceProperties']['Bucket']
delete_notification(Bucket)
print("Sending response to custom resource after Delete")
elif event['RequestType'] == 'Create' or event['RequestType'] == 'Update':
print("Request Type:",event['RequestType'])
LambdaArn=event['ResourceProperties']['LambdaArn']
Bucket=event['ResourceProperties']['Bucket']
Prefix=event['ResourceProperties']['Prefix']
add_notification(LambdaArn, Bucket, Prefix)
responseData={'Bucket':Bucket}
print("Sending response to custom resource")
responseStatus = 'SUCCESS'
except Exception as e:
print('Failed to process:', e)
responseStatus = 'FAILED'
responseData = {'Failure': 'Something bad happened.'}
cfnresponse.send(event, context, responseStatus, responseData)
def add_notification(LambdaArn, Bucket, Prefix):
bucket_notification = s3.BucketNotification(Bucket)
response = bucket_notification.put(
NotificationConfiguration={
'LambdaFunctionConfigurations': [
{
'LambdaFunctionArn': LambdaArn,
'Events': [
's3:ObjectCreated:*'
],
'Filter': {'Key': {'FilterRules': [
{'Name': 'Prefix', 'Value': Prefix}
]}}
}
]
}
)
print("Put request completed....")
def delete_notification(Bucket):
bucket_notification = s3.BucketNotification(Bucket)
response = bucket_notification.put(
NotificationConfiguration={}
)
print("Delete request completed....")
Runtime: python3.9
Timeout: 50
LambdaTrigger:
Type: 'Custom::LambdaTrigger'
DependsOn: LambdaInvokePermission
Properties:
ServiceToken: !GetAtt CustomResourceLambdaFunction.Arn
LambdaArn: !GetAtt S3NotificationLambdaFunction.Arn
Bucket: !Ref NotificationBucket
Prefix: !Ref Prefix
画像処理にPillowを使います
requirements.txt
Pillow
インストール
pip3 install -r function/requirements.txt -t python
function.pyを編集
import json
import boto3
import os
from PIL import Image
WIDTH_INDEX = 0
HEIGHT_INDEX = 1
def lambda_handler(event, context):
s3 = boto3.client('s3')
created_images = event["Records"]
for image in created_images:
created_image_path = image["s3"]["object"]["key"]
bucket_name = image["s3"]["bucket"]["name"]
temp_file_path = u'/tmp/' + os.path.basename(created_image_path)
print(f'target_image is {created_image_path}')
s3.download_file(Bucket=bucket_name, Key=created_image_path, Filename=temp_file_path)
target_image = Image.open(temp_file_path)
image_width = target_image.size[WIDTH_INDEX]
image_height = target_image.size[HEIGHT_INDEX]
is_width_shorter = image_width < image_height
upper_coordinate = 0
left_coordinate = 0
lower_coordinate = 0
right_coordinate = 0
if is_width_shorter:
left_coordinate = 0
right_coordinate = image_width
upper_coordinate = image_height / 2 - image_width / 2
lower_coordinate = image_height / 2 + image_width / 2
else:
upper_coordinate = 0
lower_coordinate = image_height
left_coordinate = image_width / 2 + image_height / 2
right_coordinate = image_width / 2 - image_height / 2
cropped_image = target_image.crop((left_coordinate, upper_coordinate, right_coordinate, lower_coordinate))
cropped_file_temp_path = u'/tmp/cropped-' + os.path.basename(created_image_path)
cropped_image.save(cropped_file_temp_path)
s3.upload_file(Filename=cropped_file_temp_path, Bucket=bucket_name, Key=f'cropped/{created_image_path}')
return {
'statusCode': 200,
'body': json.dumps('Success!')
}
awsコンソールからs3のバケットを作成する（すでに自由に使えるバケットがあるならスキップでOK）
ビルドとデプロイ
sam build sam deploy --guided
以下にダイアログの例を置いておきます。 NotificationBucketは作成したバケット名を指定する必要がありますが、それ以外はよしなに進めればOKです。
$ sam deploy --guided
Configuring SAM deploy
======================
Looking for config file [samconfig.toml] : Found
Reading default arguments : Success
Setting default arguments for 'sam deploy'
=========================================
Stack Name [sam-lambda-s3]: デフォルトでOK
AWS Region [ap-northeast-1]: デフォルトでOK
Parameter NotificationBucket []: <作成したバケット名を入力>
Parameter S3NotificationLambdaFunctionName []: <Lambdaの関数名を自由に入力>
Parameter Prefix []: original/ (このフォルダ以下に画像が追加されると通知が行われ、Lambdaが実行されます。特に好みがなければoriginalと入れておきましょう。)
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [y/N]:
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]:
#Preserves the state of previously provisioned resources when an operation fails
Disable rollback [y/N]:
Save arguments to configuration file [Y/n]:
SAM configuration file [samconfig.toml]:
SAM configuration environment [default]:
デプロイが完了したら、S3に画像上げて→Lambdaが動き→加工された画像が/croppedに入ってるのを確認してみましょう。
以上です。
