Lambda Layer via Modules

Get Your AWS Credentials

If you're using the Skillmix Labs feature, open the lab settings (the beaker icon) on the right side of the code editor. Then, click the Start Lab button to start hte lab environment.

Wait for the credentials to load. Then run this in the terminal:

$ aws configure --profile smx-lab
AWS Access Key ID [None]: AKIA3E3W34P42CSHXDH5
AWS Secret Access Key [None]: vTmqpOqefgJfse8i6QwzgpjgswPjHZ6h/oiQq4zf
Default region name [None]: us-west-2
Default output format [None]: json

Be sure to name your credentials profile 'smx-lab'.

Note: If you're using your own AWS account you'll need to ensure that you've created and configured a named AWS CLI profile named smx-lab.

Create the Terraform File

We'll be doing all of our work in one Terraform file. Create a new directory on your computer somewhere, and then create a file named main.tf in it.

Create the Terraform & Provider Block

Next, we will create a Terraform configuration that will allow us to use the AWS provider. This configuration will require us to specify the version of the AWS provider that we want to use, as well as the version of Terraform that we are using. We will also specify the AWS profile and region that we want to use. This code will ensure that the correct versions of Terraform and the AWS provider are used, and that the AWS provider is configured correctly.

Append this code to the main.tf file:

terraform {
  required_version = ">= 1.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.9"
    }
    random = {
      source  = "hashicorp/random"
      version = ">= 2.0"
    }
  }
}

provider "aws" {
  profile = "smx-temp"
  region = "us-west-2"
}

Create the Data Resource Blocks

Next, we will create a data source that will allow us to access the AWS caller identity of the current user. This data source will be called "aws_caller_identity" and will be set to the "current" provider, which is the AWS Cross Account provider. This data source will allow us to access the identity of the current user, which can be used for authentication and authorization purposes.

Append this code to the main.tf file:

data "aws_caller_identity" "current" {}

data "aws_region" "current" {}

Create the Python File

We will need a file Python file to use in this project. The function imports mysql-connector-python package that is provisioned using Lambda Layer and prints a list of files in /opt directory. /opt is a location where Lambda Layers attach content.

In your project directory, create a folder named src. In that folder create a file named app.py. Put this code into that file:

from mysql import connector
from mysql.connector import Error
from os import listdir
from os import path, walk

def lambda_handler(event, context):
  '''lists the files in the /opt directory

  Lambda extracts the layer contents into the /opt directory when
  setting up the execution environment for the function. Reference:
  https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html

  Returns
  -------
  list
    a list of files in the /opt directory
  '''
  target_dir = "/opt"
  target_dir_files = []
  for (dirpath, _, filenames) in walk(target_dir):
    target_dir_files += [path.join(dirpath, file) for file in filenames]
  return target_dir_files

Create the Dependencies Directory

We need to store our dependencies list in a requirements.txt file in our project. This file will then be used to build our lambda layer.

In the root of the project, create a folder and file like this: ./dependencies/mysql-connector-python. In that folder, create a file named requirements.txt and add this text:

mysql-connector-python==8.0.28

Create the Lambda Function Resource

Next, we will create a Lambda function using the Terraform AWS module. This code will create a Lambda function with the name generated from the random_pet resource, a description of "My awesome lambda function", a handler of "app.lambda_handler", a runtime of "python3.8", and a source path of the module's src/app.py file. Additionally, the code will attach a Lambda layer to the function and add two tags to the function.

Notice the layers block. This points to our layers resource that we'll create next.

Append this code to the main.tf file:

module "lambda_function" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "~> 4.0"

  function_name = "${random_pet.this.id}-lambda"
  description   = "My awesome lambda function"
  handler       = "app.lambda_handler"
  runtime       = "python3.8"
  publish       = true

  source_path = "${path.module}/src/app.py"

  layers = [
    module.lambda_layer.lambda_layer_arn,
  ]

  tags = {
    Pattern = "terraform-lambda-layer"
    Module  = "lambda_function"
  }
}

Create the Lambda Layers Resource

Next, we will create a Lambda layer using the Terraform module "lambda_layer". This module will create a layer with the name specified in the "layer_name" variable, and will include the "mysql-connector-python" package from the "source_path" variable. The "compatible_runtimes" variable will specify which runtimes the layer is compatible with, and the "runtime" variable will specify which runtime to use for the "pip install" command. Finally, the "tags" variable will add tags to the layer for easier identification.

Notice the source_path block. It points to the directory and file we created previously.

Append this code to the main.tf file:

module "lambda_layer" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "~> 4.0"

  create_layer = true

  layer_name          = "${random_pet.this.id}-layer"
  description         = "My amazing lambda layer (pip install)"
  compatible_runtimes = ["python3.8"]

  runtime = "python3.8" # Runtime is required for "pip install" to work

  source_path = [
    {
      path             = "${path.module}/dependencies/mysql-connector-python"
      pip_requirements = true     # Will run "pip install" with default "requirements.txt" from the path
      prefix_in_zip    = "python" # required to get the path correct
    }
  ]

  tags = {
    Pattern = "terraform-lambda-layer"
    Module  = "lambda_layer"
  }
}

Create Pet Name Resource

Next, we will create a random pet resource using Terraform. This code will generate a random pet name with two words. The random_pet resource will create a random pet name that is composed of two words, each of which is randomly selected from a list of words. This resource can be used to generate unique names for resources in your Terraform configuration.

Append this code to the main.tf file:

resource "random_pet" "this" {
  length = 2
}

Create Outputs

Let's output some helpful information. Append this to the end of the main.tf file.

# Lambda Function
output "lambda_function_arn" {
  description = "The ARN of the Lambda Function"
  value       = module.lambda_function.lambda_function_arn
}

output "lambda_function_invoke_arn" {
  description = "The Invoke ARN of the Lambda Function"
  value       = module.lambda_function.lambda_function_invoke_arn
}

output "lambda_function_name" {
  description = "The name of the Lambda Function"
  value       = module.lambda_function.lambda_function_name
}

output "lambda_function_qualified_arn" {
  description = "The ARN identifying your Lambda Function Version"
  value       = module.lambda_function.lambda_function_qualified_arn
}

output "lambda_function_version" {
  description = "Latest published version of Lambda Function"
  value       = module.lambda_function.lambda_function_version
}

# IAM Role of Lambda Function
output "lambda_role_arn" {
  description = "The ARN of the IAM role created for the Lambda Function"
  value       = module.lambda_function.lambda_role_arn
}

# Lambda Layer
output "lambda_layer_arn" {
  description = "The ARN of the Lambda Layer with version"
  value       = module.lambda_layer.lambda_layer_arn
}

output "lambda_layer_layer_arn" {
  description = "The ARN of the Lambda Layer without version"
  value       = module.lambda_layer.lambda_layer_layer_arn
}

output "lambda_layer_version" {
  description = "The Lambda Layer version"
  value       = module.lambda_layer.lambda_layer_version
}

Deploying the Project

Now that we have all of our code written, we can deploy the project. Open a terminal, navigate to the project, and run these commands.

# initialize the project 
$ terraform init 

# plan the project 
$ terraform plan 

# apply the project 
$ terraform apply

Test the Project

After deployment, you can invoke the Lambda function. You should see a list of files in /opt directory.

You do need to have the AWS CLI tool installed to perform this step.

To do this, you can run these commands in the terminal:

$ aws lambda invoke --function-name <lambda-function-name> output.json

You should see a success message.

Destroy the Project

Time to destroy the project!

$ terraform destroy