Docker Image With Aws Cli Installed

2/13/2022by admin

At re:Invent 2020, AWS announced support for authoring, shipping and deploying the popular serverless Lambda services via Docker images. Further, they allow images up to 10GB in size. As multiple authorities noted, this is a game changer, particularly for the scientific Python community as this would allow us to author machine learning and even deep learning inferencing functions using AWS Lambdas. This blog takes a quick look at authoring a “hello-world” style lambda using Docker.

To understand how Docker works or how to build Docker images, checkout my Docker 101 cheatsheet. Having said that, you do not need to know much about Docker or containerization for simple functions.

Docker image with aws cli installed online

FROM python:2.7 RUN pip install awscli -upgrade -user Once the docker image is built from this dockerfile, I run it. But when I get into the container and try to run the AWS CLI, it can't find it, because it is not in the PATH environment variable. Install Aws Cli Into Docker Image We recommend using an ELB in AWS in front of your Rancher servers. In order for ELB to work correctly with Rancher’s websockets, you will need to enable proxy protocol mode and ensure HTTP support is disabled.

The Lambda runtime images¶

AWS provides the runtime images for different languages, including Python. The images are shared at DockerHub: amazon/aws-lambda-provided and ECR Public: public.ecr.aws/lambda/provided. As expected, the base image is a flavor of linux called Amazon Linux.

The RIC and the RIE

Image

The Python Docker images come with two important components pre-loaded. The RIC - Runtime Interface Client provides the interface between Lambda (infrastructure) & the function code. Think about the invoker from API gateway that runs your handler function. The RIE - Runtime Interface Emulator is a tool that allows you to test the code locally.

High-level workflow¶

The steps involved in publishing a Lambda function using Docker images will look like below

  1. Download base images in your language of choice
  2. Package your code + dependencies + resource files into the image using Docker CLI
  3. Push the image into AWSECR - AWS Elastic Container Registry. Note: You are not charged storage for these images.
  4. Create a function, choose ‘Container Image’ and the rest should be similar

Gotchas¶

  • Image max size is 10 GB
  • Application code should run on a read-only file system. Only the /tmp dir is writable with 512 MB storage
  • The container is instantiated with a user with least set of permissions. Ensure app code conforms with this.
  • Container settings

Steps¶

  • Install Docker for desktop and AWSCLI
  • Create your regular Lambda function into a folder called app. Store your app.py, requirements.txt files into it. See sample below:

app.py:

requirements.txt:

  • Create a docker file as below:
  • Build Docker image using syntax docker build -t <username/imagename:tag> <build context>. For example: docker build -t atmamani/aws-lambda-docker:v1 .. This prints an output like below:
  • Run the container locally using the command:
  • You can test by posting curl -XPOST 'http://localhost:9000/2015-03-31/functions/function/invocations' -d '{}'
  • If you notice any errors, you can edit the files and rebuild the image and update the tag. THen you can retest the application.
  • Authenticate the AWSCLI by following this help.
  • Pass the auth from AWSCLI to Docker CLI, so Docker can tag and later push the images. Docker CLI normally pushes to the default DockerHub registry. However, for Lambda to work, you need to push to Amazon ECR registry. In ECR, I used the web UI to make a repository called ml2web.
  • Run below, where you replace --region us-west-2 with your AWS region. You also replace --profile <aws_profile_name> with your AWSCLI profile name. If you are using just 1 account for all (don’t do this), then you can skip this argument. The Docker username needs to be AWS always. The URL like string is the name of your ECR registry.

This threw an error saying permission denied, however it still works for me.

  • Then tag your image using the command docker tag atmamani/aws-lambda-docker:v2 <your_ecr_name>.dkr.ecr.us-west-2.amazonaws.com/ml2web:v2

  • Then push to ECR using the command: docker push <your_ECR_name>.dkr.ecr.us-west-2.amazonaws.com/ml2web:v2

Cli

Docker Image With Aws Cli Installed Download

  • Finally, use the Lambda web UI to create a new function. The only difference is, you choose ‘container image’ in the radio and provide the URI to the ECR registry. You can then browse for your image and tag and choose it.

Aws Cli Install Windows

Sources¶

Comments are closed.