AWS Re:Invent 2020 held a number of great announcements for AWS Lambda! One of these included the support for Docker Containers — previously the only supported packaging was Zip that was stored in S3 behind the scenes.
At the same time, Lambda also saw the memory configuration expand — from the 3GB limit initially, all the way up to 10GBs! This is especially useful as Lambda will also support Docker images that reach 10GB in size — though this size increase is only for Docker containers, ZIP packages remain at a maximum of 250MB.
AWS provides two ways to support docker lambda
- amazon/nodejs12.x-base
- amazon/nodejs10.x-base
- amazon/python3.8-base
- amazon/python3.7-base
- amazon/python3.6-base
- amazon/python2.7-base
- amazon/ruby2.7-base
- amazon/ruby2.5-base
- amazon/go1.x-base
- amazon/java11-base
- amazon/java8.al2-base
- amazon/java8-base
- amazon/dotnetcore3.1-base
- amazon/dotnetcore2.1-base
Apparently, Lambda is not actually running the container, rather it will build a function from the container image. I have noticed when running the Java 11 Base Lambda image a pretty significant cold start penalty — up to 20 seconds.
To test out the docker containers, I’ll utilize AWS Serverless Application Model to generate, build, and deploy the lambda!
Build & Deploy Docker Lambda
Initialization
To simplify some of these commands, I’ll create some environment variables to hold the default region and AWS Account ID:
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity | jq -r '.Account')
export AWS_REGION=$(aws configure get default.region)Since we will be building out an image for the lambda, we want to store it in an Elastic Container Registry. We can create one with the following command:
aws ecr create-repository --repository-name gizmo-lambda-container | jq '.repository.repositoryUri'Which should output the container registry url, similar to: 196295636944.dkr.ecr.us-east-1.amazonaws.com/gizmo-lambda-container
We can then authenticate with the AWS ECR repo:
aws ecr get-login-password | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGIO
N}.amazonaws.comNow we will use the AWS Serverless Application Model to build and deploy our application. In an empty directory we will create our project:
sam initYou will be prompted to select several options:
Which template source would you like to use?
1 - AWS Quick Start Templates
2 - Custom Template LocationWe want to use AWS Quick Start Templates.
What package type would you like to use?
1 - Zip (artifact is a zip uploaded to S3)
2 - Image (artifact is an image uploaded to an ECR image repository)And here we want to use Image! For this example, I went with the amazon/java11-base base image. Since it is Java we are prompted about the dependency manager, which I recommend gradle.
Build
The skeleton project that is generated has our Dockerfile and application code already setup. The AWS SAM command will build the Docker image for us!
sam buildAnd we can see the Docker build steps in the output:
Building codeuri: . runtime: None metadata: {'DockerTag': 'java11-gradle-v1', 'DockerContext': './HelloWorldFunction', 'Dockerfile': 'Dockerfile'} functions: ['HelloWorldFunction']
Building image for HelloWorldFunction function
Setting DockerBuildArgs: {} for HelloWorldFunction function
Step 1/16 : FROM public.ecr.aws/lambda/java:11 as build-image
---> 9d9f8f6bbeea
Step 2/16 : ARG SCRATCH_DIR=/var/task/build
---> Running in 3f077f57a99f
---> 1790443364e1
Step 3/16 : COPY src/ src/
---> 6357ad3066ed
Step 4/16 : COPY gradle/ gradle/
---> 86b36578535b
Step 5/16 : COPY build.gradle gradlew ./
---> 7a0e366bbda5
Step 6/16 : RUN mkdir build
---> Running in 57ceac09d965
---> 6d547fcbb584
Step 7/16 : COPY gradle/lambda-build-init.gradle ./build
---> ee1fb2ece820
Step 8/16 : RUN ./gradlew --project-cache-dir $SCRATCH_DIR/gradle-cache -Dsoftware.amazon.aws.lambdabuilders.scratch-dir=$SCRATCH_DIR --init-script $SCRATCH_DIR/lambda-build-init.gradle build
---> Running in 06116e2de319
Downloading https://services.gradle.org/distributions/gradle-5.1.1-bin.zip
.................................................................................
Welcome to Gradle 5.1.1!
Here are the highlights of this release:
- Control which dependencies can be retrieved from which repositories
- Production-ready configuration avoidance APIs
For more details see https://docs.gradle.org/5.1.1/release-notes.html
Starting a Gradle Daemon (subsequent builds will be faster)
> Task :compileJava
> Task :processResources NO-SOURCE
> Task :classes
> Task :jar
> Task :assemble
> Task :compileTestJava
> Task :processTestResources NO-SOURCE
> Task :testClasses
> Task :test
> Task :check
> Task :build
BUILD SUCCESSFUL in 17s
4 actionable tasks: 4 executed
---> f394216fbb70
Step 9/16 : RUN rm -r $SCRATCH_DIR/gradle-cache
---> Running in 0b8e94e2670c
---> 7f9033989b1a
Step 10/16 : RUN rm -r $SCRATCH_DIR/lambda-build-init.gradle
---> Running in 4965d01c3455
---> fd5e12377577
Step 11/16 : RUN cp -r $SCRATCH_DIR/*/build/distributions/lambda-build/* .
---> Running in 9c29934509fe
---> 434e1178004e
Step 12/16 : FROM public.ecr.aws/lambda/java:11
---> 9d9f8f6bbeea
Step 13/16 : COPY --from=build-image /var/task/META-INF ./
---> af6f91bce6e5
Step 14/16 : COPY --from=build-image /var/task/helloworld ./helloworld
---> 9a7cea4f8cbd
Step 15/16 : COPY --from=build-image /var/task/lib/ ./lib
---> 09a9974d2dec
Step 16/16 : CMD ["helloworld.App::handleRequest"]
---> Running in 11e9683c59dc
---> c9fbfd205cd4
Successfully built c9fbfd205cd4
Successfully tagged helloworldfunction:java11-gradle-v1
Build Succeeded
Built Artifacts : .aws-sam/build
Built Template : .aws-sam/build/template.yaml
Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guidedA very nice aspect of the AWS SAM is the ability to run the container locally!
sam local invokeWith the output matching what we would normally see in the Cloudwatch logs:
Invoking Container created from helloworldfunction:java11-gradle-v1
Image was not found.
Building image..........
Skip pulling image and use local one: helloworldfunction:rapid-1.13.2.
START RequestId: 22cd82ae-e042-4863-9447-4831faf9490a Version: $LATEST
END RequestId: 22cd82ae-e042-4863-9447-4831faf9490a
REPORT RequestId: 22cd82ae-e042-4863-9447-4831faf9490a Init Duration: 1.02 ms Duration: 1262.58 ms Billed Duration: 1300 ms Memory Size: 128 MB Max Memory Used: 128 MB
{"statusCode":200,"headers":{"X-Custom-Header":"application/json","Content-Type":"application/json"},"body":"{ \"message\": \"hello world\", \"location\": \"71.174.101.210\" }"}%Deployment
Finally, we can have AWS SAM create our Lambda and connect it to a REST API for us!
sam deploy --guidedAnd we get the image being pushed to the ECR repository along with the Cloudformation stack deploying the REST API and Lambda.
Configuring SAM deploy
======================
Looking for config file [samconfig.toml] : Not found
Setting default arguments for 'sam deploy'
=========================================
Stack Name [sam-app]: gizmo-example
AWS Region [us-east-1]:
Image Repository []: 196295636944.dkr.ecr.us-east-1.amazonaws.com/gizmo-lambda-container
Images that will be pushed:
helloworldfunction:java11-gradle-v1 to 196295636944.dkr.ecr.us-east-1.amazonaws.com/gizmo-lambda-container:helloworldfunction-c9fbfd205cd4-java11-gradle-v1
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [y/N]: y
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]: Y
HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
Save arguments to configuration file [Y/n]: Y
SAM configuration file [samconfig.toml]:
SAM configuration environment [default]:
Looking for resources needed for deployment: Not found.
Creating the required resources...
Successfully created!
Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-1owsb6dv8zl8j
A different default S3 bucket can be set in samconfig.toml
Saved arguments to config file
Running 'sam deploy' for future deployments will use the parameters saved above.
The above parameters can be changed by modifying samconfig.toml
Learn more about samconfig.toml syntax at
https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html
The push refers to repository [196295636944.dkr.ecr.us-east-1.amazonaws.com/gizmo-lambda-container]
0bd0a3c72bc7: Pushed
11bc4c037913: Pushed
e3d7108f2c0b: Pushed
d1d758bc3380: Pushed
577ed33a1f65: Pushed
d6fa53d6caa6: Pushed
016e6d3f9722: Pushed
898f760e15c4: Pushed
af6d16f2417e: Pushed
helloworldfunction-c9fbfd205cd4-java11-gradle-v1: digest: sha256:507124a3f49f85a91a97508c50acc1b66be3731d1cfeb187a245a44cefb5979e size: 2205
Deploying with following values
===============================
Stack name : gizmo-example
Region : us-east-1
Confirm changeset : True
Deployment image repository : 196295636944.dkr.ecr.us-east-1.amazonaws.com/gizmo-lambda-container
Deployment s3 bucket : aws-sam-cli-managed-default-samclisourcebucket-1owsb6dv8zl8j
Capabilities : ["CAPABILITY_IAM"]
Parameter overrides : {}
Signing Profiles : {}
Initiating deployment
=====================
HelloWorldFunction may not have authorization defined.
Uploading to gizmo-example/40514ed797d174c979e4a5a29672a62a.template 1199 / 1199.0 (100.00%)
Waiting for changeset to be created..
CloudFormation stack changeset
-------------------------------------------------------------------------------------------------------------------------------------
Operation LogicalResourceId ResourceType Replacement
-------------------------------------------------------------------------------------------------------------------------------------
+ Add HelloWorldFunctionHelloWorldPer AWS::Lambda::Permission N/A
missionProd
+ Add HelloWorldFunctionRole AWS::IAM::Role N/A
+ Add HelloWorldFunction AWS::Lambda::Function N/A
+ Add ServerlessRestApiDeployment47fc AWS::ApiGateway::Deployment N/A
2d5f9d
+ Add ServerlessRestApiProdStage AWS::ApiGateway::Stage N/A
+ Add ServerlessRestApi AWS::ApiGateway::RestApi N/A
-------------------------------------------------------------------------------------------------------------------------------------
Changeset created successfully. arn:aws:cloudformation:us-east-1:196295636944:changeSet/samcli-deploy1607463634/9467a1b3-b809-4a57-aecb-8638ec49f4ca
Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]: y
2020-12-08 16:41:12 - Waiting for stack create/update to complete
CloudFormation events from changeset
-------------------------------------------------------------------------------------------------------------------------------------
ResourceStatus ResourceType LogicalResourceId ResourceStatusReason
-------------------------------------------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS AWS::IAM::Role HelloWorldFunctionRole -
CREATE_IN_PROGRESS AWS::IAM::Role HelloWorldFunctionRole Resource creation Initiated
CREATE_COMPLETE AWS::IAM::Role HelloWorldFunctionRole -
CREATE_IN_PROGRESS AWS::Lambda::Function HelloWorldFunction -
CREATE_IN_PROGRESS AWS::Lambda::Function HelloWorldFunction Resource creation Initiated
CREATE_COMPLETE AWS::Lambda::Function HelloWorldFunction -
CREATE_IN_PROGRESS AWS::ApiGateway::RestApi ServerlessRestApi Resource creation Initiated
CREATE_IN_PROGRESS AWS::ApiGateway::RestApi ServerlessRestApi -
CREATE_COMPLETE AWS::ApiGateway::RestApi ServerlessRestApi -
CREATE_IN_PROGRESS AWS::Lambda::Permission HelloWorldFunctionHelloWorldPer -
missionProd
CREATE_IN_PROGRESS AWS::ApiGateway::Deployment ServerlessRestApiDeployment47fc -
2d5f9d
CREATE_IN_PROGRESS AWS::ApiGateway::Deployment ServerlessRestApiDeployment47fc Resource creation Initiated
2d5f9d
CREATE_IN_PROGRESS AWS::Lambda::Permission HelloWorldFunctionHelloWorldPer Resource creation Initiated
missionProd
CREATE_COMPLETE AWS::ApiGateway::Deployment ServerlessRestApiDeployment47fc -
2d5f9d
CREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdStage -
CREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdStage Resource creation Initiated
CREATE_COMPLETE AWS::ApiGateway::Stage ServerlessRestApiProdStage -
CREATE_COMPLETE AWS::Lambda::Permission HelloWorldFunctionHelloWorldPer -
missionProd
CREATE_COMPLETE AWS::CloudFormation::Stack gizmo-example -
-------------------------------------------------------------------------------------------------------------------------------------
CloudFormation outputs from deployed stack
----------------------------------------------------------------------------------------------------------------------------------------
Outputs
----------------------------------------------------------------------------------------------------------------------------------------
Key HelloWorldFunctionIamRole
Description Implicit IAM Role created for Hello World function
Value arn:aws:iam::196295636944:role/gizmo-example-HelloWorldFunctionRole-13AV2P47KN805
Key HelloWorldApi
Description API Gateway endpoint URL for Prod stage for Hello World function
Value https://uwuy5qpvmg.execute-api.us-east-1.amazonaws.com/Prod/hello/
Key HelloWorldFunction
Description Hello World Lambda Function ARN
Value arn:aws:lambda:us-east-1:196295636944:function:gizmo-example-HelloWorldFunction-LX0IVO85A1CM
----------------------------------------------------------------------------------------------------------------------------------------
Successfully created/updated stack - gizmo-example in us-east-1We ended up with a REST API endpoint: https://uwuy5qpvmg.execute-api.us-east-1.amazonaws.com/Prod/hello/:
> curl https://uwuy5qpvmg.execute-api.us-east-1.amazonaws.com/Prod/hello/
{ "message": "hello world", "location": "3.239.84.207" }%If we curl this, we should get the output — note back to the beginning of the blog post, that I often see a long delay in the first request as the lambda is experiencing the cold start penalty.
Originally published at gizmo.codes on December 8, 2020.
