One of my highlights from this years re:Invent announcements where the Lambda Layers.
With Lambda Layers AWS offers a way to separate own code or librarys in an extra Layer which can be used by Lambda functions. These Layers can be stacked up to 5 layers and are hierarchically. That means the content from a higher layer overwrites conflicting items from a lower layer. All Layers plus the Lambda code can´t be exceed the 250MB Limit.
But why are Layers such a big thing? That´s pretty easy. The most of use (or better should use) the serverless framework or the serverless application model (SAM) to deploy their functions with the help of CloudFormation. To do that every Tool zips the code and all library´s that the lambda needs and uploads that zip to S3. Bigger zip´s results in much longer deployment times. If you deploy often that can be a massive time wasting thing or even makes it impossible to deploy e.g. while sitting in a train with very low bandwidth and on the other side it is totally boring waiting for 60 megs or more to be uploaded to S3. For an example here are the artifact sizes for one of my Alexa Skills.
4.9 MB before extracting all librarys to a layer versus 5.1 KB after. That results in a massiv saving for the deployment time.
Creating the Layer
Bevor introducing the layer my project had a folder structure like this. All my code and the librarys are located under a src directory, because SAM zips the whole directory as described earlier.
For my first Layer i moved the package.json on step up and so the node_modules folder was also moving one step up resulting in the following folder structure.
At the moment SAM supports Layers but not like Lambda Functions. SAM will not automatically zip the content and uploads it to S3. To do that i write a little bash script that does that for me and deploys the layer as a CloudFormation Stack.
#!/usr/bin/env bash
# Create random artifact name
LAYER_ARTIFACT="layer-${RANDOM}${RANDOM}.zip"
# Creating the required folder structure for the layer
rm -rf nodejs >/dev/null 2>&1
mkdir nodejs >/dev/null 2>&1
mkdir nodejs >/dev/null 2>&1
cp -R ./node_modules nodejs
zip -rq ${LAYER_ARTIFACT} nodejs/
rm -rf nodejs >/dev/null 2>&1
AWS_REGION=eu-west-1
echo "Getting AWS Account ID"
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --output text --query 'Account')
echo "Using AWS Account with ID: ${AWS_ACCOUNT_ID}"
DEPLOYMENT_BUCKET="${AWS_ACCOUNT_ID}-ask-layer"
# Creating the Bucket and enable encryption
echo "Using ${DEPLOYMENT_BUCKET} for deployment artifacts"
aws --region ${AWS_REGION} s3 mb s3://${DEPLOYMENT_BUCKET} >/dev/null 2>&1
aws s3api put-bucket-encryption --bucket ${DEPLOYMENT_BUCKET} --server-side-encryption-configuration '{"Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}]}' >/dev/null 2>&1
# Upload Layer
aws --region ${AWS_REGION} s3 cp $(pwd)/${LAYER_ARTIFACT} s3://${DEPLOYMENT_BUCKET}
# Cleanup
rm ${LAYER_ARTIFACT}
TEMPLATE_FILE=packaged-layer-template.yaml
# Deploy the layer
aws --region ${AWS_REGION} cloudformation package --template-file layer-template.yaml --s3-bucket ${DEPLOYMENT_BUCKET} --output-template-file ${TEMPLATE_FILE}
aws --region ${AWS_REGION} cloudformation deploy --template-file ${TEMPLATE_FILE} --stack-name ask-layer --parameter-overrides ContentBucket=${DEPLOYMENT_BUCKET} ContentKey=${LAYER_ARTIFACT}
And here is the SAM Template for that layer.
AWSTemplateFormatVersion: 2010-09-09
Transform:
- AWS::Serverless-2016-10-31
Parameters:
ContentBucket:
Type: String
ContentKey:
Type: String
Outputs:
LayerARN:
Description: The ARN for that Layer
Value: !Ref AskLayer
Export:
Name: !Sub "${AWS::StackName}-LayerARN"
Resources:
AskLayer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: AskLayer
Description: Base Layer for my Alexa Skills
ContentUri:
Bucket: !Ref ContentBucket
Key: !Ref ContentKey
CompatibleRuntimes:
- nodejs6.10
- nodejs8.10
RetentionPolicy: Retain
The important thing in this template is the Outputs block where the layer ARN is exported so that it can be imported to other stacks. In the SAM Template for my lambda function i only have to import this Output.
AlexaSkillFunction:
Type: AWS::Serverless::Function
Properties:
Handler: index.handler
Runtime: nodejs8.10
CodeUri: ./src
Layers:
- Fn::ImportValue:
!Sub "${AskLayerStackName}-LayerARN"
And as the result the Lambda function is using this layer.
And now i don´t have to upload the librarys on every code change in my lambda.
To extract 3rd party librarys to an extra Lambda function is going to my best practices list for lambda functions an will save me a lot of time in my future work. And i will recommend this to every Lambda developer. And the last words: Thank you AWS for that feature!!!