API Gateway to DDB

Interactive Mode

About this Architecture

Here is some information about this architecture.

This solution shows how to build an API Gateway that integrates directly with a DynamoDB table. The gateway takes requests from the internet, and matches them to its path configuration. If there's a path match, it will forward the request to DynamoDB to save the record.

This API is protected with an API key, so only requests made with the key will be accepted.

How to Build This Solution

Here are the steps you can follow to build this solution on your own.

Here are the steps needed to build this architecture.

If you're using the Skillmix lab feature, here are the instructions to follow:

Start the Lab

On the right sidebar, click on the Lab button to open the lab environment window. When the sidebar opens, click on the Start Lab button. Wait for all of the credentials to appear.

Configure the AWS CLI Profile

Once the lab credentials are ready, you can configure the AWS CLI profile. Here is how you can do it on various operating systems.

  • Linux/Mac: open this file with a text editor ~/.aws/credentials

  • Windows: open this file with a text editor %USERPROFILE%.aws credentials

Once you have the file open, add this profile to it. Use the keys from the lab session you started.

# ...other profiles

[smx-lab]
aws_access_key_id=<lab access key>
aws_secret_access_key=<lab secret key>

Create the Terraform File

We'll be doing all of our work in one Terraform file. Create a new directory on your computer somewhere, and then create a file named main.tf in it.

Create the Terraform & Provider Blocks

Next, we will create a Terraform configuration that will allow us to use the AWS provider with a specific version and a specific profile. This code will set the required version of Terraform to be greater than or equal to 0.14.9, and will set the required provider to be the AWS provider with a version of 3.27 or higher. It will also set the profile to be "smx-temp" and the region to be "us-west-2".

Append this code to the main.tf file:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }

  required_version = ">= 0.14.9"
}

provider "aws" {
  profile = "smx-lab"
  region  = "us-west-2"
}

Create the Data Blocks

Next, we will create two data sources that will allow us to access information about the current AWS account and region. The first line of code, data "aws_caller_identity" "current" {}, will create a data source that will provide information about the current AWS account, such as the account ID and the ARN. The second line of code, data "aws_region" "current" {}, will create a data source that will provide information about the current AWS region, such as the region name and the region code.

Append this code to the main.tf file:

data "aws_caller_identity" "current" {}

data "aws_region" "current" {}

Create the IAM Role for the Gateway

Next, we will create an IAM role for our API Gateway using the Terraform code below. This code will create an IAM role with an assume role policy that allows the API Gateway service to assume the role. This will allow the API Gateway to access other AWS services on our behalf.

Append this code to the main.tf file:

resource "aws_iam_role" "APIGWRole" {
  # uncomment the 'permissions_boundary' argument if running this lab on skillmix.io 
  # permissions_boundary = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:policy/LabUserNewResourceBoundaryPolicy"
  assume_role_policy = <<POLICY1
{
  "Version" : "2012-10-17",
  "Statement" : [
    {
      "Effect" : "Allow",
      "Principal" : {
        "Service" : "apigateway.amazonaws.com"
      },
      "Action" : "sts:AssumeRole"
    }
  ]
}
POLICY1
}

Create the IAM Role Policy for the Gateway

Next, we will create an AWS IAM policy. This policy will allow us to put items and query a DynamoDB table. The code will define the version of the policy, the effect, the action, and the resource that the policy will apply to. This will ensure that the policy is applied correctly and securely.

Append this code to the main.tf file:

resource "aws_iam_policy" "APIGWPolicy" {
  policy = <<POLICY2
{
  "Version" : "2012-10-17",
  "Statement" : [
    {
      "Effect" : "Allow",
      "Action" : [
        "dynamodb:PutItem",
        "dynamodb:Query"
      ],
      "Resource" : [ "${aws_dynamodb_table.MyDynamoDBTable.arn}",
      "${aws_dynamodb_table.MyDynamoDBTable.arn}/index/*" ]
    }
  ]
}
POLICY2
}

Attach the Policy to the Role (Gateway)

Next, we will create an IAM role policy attachment that will attach the policy we just created to the IAM role. This will allow the IAM role to access the resources that are specified in the policy.

Append this code to the main.tf file:

resource "aws_iam_role_policy_attachment" "APIGWPolicyAttachment" {
  role       = aws_iam_role.APIGWRole.name
  policy_arn = aws_iam_policy.APIGWPolicy.arn
}

Create the DynamoDB Table

Next, we will create an AWS DynamoDB table. This code will create a table called "Pets" with a hash key of "id" and two attributes, "id" and "PetType". It will also create a global secondary index called "PetType-index" with a hash key of "PetType" and two non-key attributes, "PetName" and "PetPrice". The table will be set to "PROVISIONED" billing mode with read and write capacities of 5.

Append this code to the main.tf file:

resource "aws_dynamodb_table" "MyDynamoDBTable" {
  name           = "Pets"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  hash_key       = "id"

  attribute {
    name = "id"
    type = "S"
  }

  attribute {
    name = "PetType"
    type = "S"
  }

  global_secondary_index {
    name               = "PetType-index"
    hash_key           = "PetType"
    write_capacity     = 5
    read_capacity      = 5
    projection_type    = "INCLUDE"
    non_key_attributes = ["PetName", "PetPrice"]
  }
}

Create API Gateway Resource

Next, we will create an AWS API Gateway REST API resource using Terraform. This code will create an API Gateway with two paths, "/pets" and "/pets/{PetType}", and will enable API key authentication. The code also defines the request and response templates for each path, as well as the integration type and credentials.

Append this code to the main.tf file:

resource "aws_api_gateway_rest_api" "MyApiGatewayRestApi" {
  name = "APIGW DynamoDB Serverless Pattern Demo"
  body = jsonencode({
    "swagger" : "2.0",
    "info" : {
      "version" : "2022-03-21T11:36:12Z",
      "title" : "APIGW DynamoDB Serverless Pattern Demo"
    },
    "basePath" : "/v1",
    "schemes" : ["https"],
    "paths" : {
      "/pets" : {
        "post" : {
          "consumes" : ["application/json"],
          "produces" : ["application/json"],
          "responses" : {
            "200" : {
              "description" : "200 response"
            }
          },
          "security" : [{
            "api_key" : []
          }],
          "x-amazon-apigateway-integration" : {
            "type" : "aws",
            "credentials" : "${aws_iam_role.APIGWRole.arn}",
            "httpMethod" : "POST",
            "uri" : "arn:aws:apigateway:${data.aws_region.current.name}:dynamodb:action/PutItem",
            "responses" : {
              "default" : {
                "statusCode" : "200",
                "responseTemplates" : {
                  "application/json" : "{}"
                }
              }
            },
            "requestTemplates" : {
              "application/json" : "{\"TableName\":\"Pets\",\"Item\":{\"id\":{\"S\":\"$context.requestId\"},\"PetType\":{\"S\":\"$input.path('$.PetType')\"},\"PetName\":{\"S\":\"$input.path('$.PetName')\"},\"PetPrice\":{\"N\":\"$input.path('$.PetPrice')\"}}}"
            },
            "passthroughBehavior" : "when_no_templates"
          }
        }
      },
      "/pets/{PetType}" : {
        "get" : {
          "consumes" : ["application/json"],
          "produces" : ["application/json"],
          "parameters" : [{
            "name" : "PetType",
            "in" : "path",
            "required" : true,
            "PetType" : "string"
          }],
          "responses" : {
            "200" : {
              "description" : "200 response"
            }
          },
          "security" : [{
            "api_key" : []
          }],
          "x-amazon-apigateway-integration" : {
            "type" : "aws",
            "credentials" : "${aws_iam_role.APIGWRole.arn}",
            "httpMethod" : "POST",
            "uri" : "arn:aws:apigateway:${data.aws_region.current.name}:dynamodb:action/Query",
            "responses" : {
              "default" : {
                "statusCode" : "200",
                "responseTemplates" : {
                  "application/json" : "#set($inputRoot = $input.path('$'))\n{\n\t\"pets\": [\n\t\t#foreach($field in $inputRoot.Items) {\n\t\t\t\"id\": \"$field.id.S\",\n\t\t\t\"PetType\": \"$field.PetType.S\",\n\t\t\t\"PetName\": \"$field.PetName.S\",\n\t\t\t\"PetPrice\": \"$field.PetPrice.N\"\n\t\t}#if($foreach.hasNext),#end\n\t\t#end\n\t]\n}"
                }
              }
            },
            "requestParameters" : {
              "integration.request.path.PetType" : "method.request.path.PetType"
            },
            "requestTemplates" : {
              "application/json" : "{\"TableName\":\"Pets\",\"IndexName\":\"PetType-index\",\"KeyConditionExpression\":\"PetType=:v1\",\"ExpressionAttributeValues\":{\":v1\":{\"S\":\"$util.urlDecode($input.params('PetType'))\"}}}"
            },
            "passthroughBehavior" : "when_no_templates"
          }
        }
      }
    },
    "securityDefinitions" : {
      "api_key" : {
        "type" : "apiKey",
        "name" : "x-api-key",
        "in" : "header"
      }
    }
  })
}

Create the API Gateway Deployment

Next, we will create an AWS API Gateway Deployment resource. This resource will deploy the API Gateway Rest API that we created in the previous step. The code above will create a deployment resource with the ID of the Rest API that we created. This will allow us to deploy our API Gateway Rest API to the AWS environment.

Append this code to the main.tf file:

resource "aws_api_gateway_deployment" "MyApiGatewayDeployment" {
  rest_api_id = aws_api_gateway_rest_api.MyApiGatewayRestApi.id
}

Create the API Gateway Stage Resource

Next, we will create an AWS API Gateway Stage resource. This code will create a stage named "v1" for the API Gateway deployment, and set up an access log to be sent to a CloudWatch Log Group. The access log will contain information such as the request ID, IP address, request time, HTTP method, route key, status, protocol, and response length.

Append this code to the main.tf file:

resource "aws_api_gateway_stage" "MyApiGatewayStage" {
  deployment_id = aws_api_gateway_deployment.MyApiGatewayDeployment.id
  rest_api_id   = aws_api_gateway_rest_api.MyApiGatewayRestApi.id
  stage_name    = "v1"
  depends_on    = [aws_api_gateway_account.ApiGatewayAccountSetting]

  access_log_settings {
    destination_arn = aws_cloudwatch_log_group.MyLogGroup.arn
    format          = "{ \"requestId\":\"$context.requestId\", \"ip\": \"$context.identity.sourceIp\", \"requestTime\":\"$context.requestTime\", \"httpMethod\":\"$context.httpMethod\",\"routeKey\":\"$context.routeKey\", \"status\":\"$context.status\",\"protocol\":\"$context.protocol\", \"responseLength\":\"$context.responseLength\" }"
  }
}

Create the API Gateway Account Resource

Next, we will create an AWS API Gateway Account resource using Terraform. This resource will allow us to set up an API Gateway Account. The code above will create the API Gateway Account resource and assign it the CloudWatch Role ARN from the AWS IAM Role we created earlier. This will allow us to monitor and log the API Gateway Account's activity.

Append this code to the main.tf file:

resource "aws_api_gateway_account" "ApiGatewayAccountSetting" {
  cloudwatch_role_arn = aws_iam_role.APIGatewayCloudWatchRole.arn
}

Create the API Gateway CloudWatch Role

Next, we will create an IAM role for Amazon API Gateway to write logs to Amazon CloudWatch. This is done by using the "aws_iam_role" resource in Terraform. The code provided will create an IAM role with the assume_role_policy that allows Amazon API Gateway to assume the role and write logs to CloudWatch.

Append this code to the main.tf file:

resource "aws_iam_role" "APIGatewayCloudWatchRole" {
  # uncomment the 'permissions_boundary' argument if running this lab on skillmix.io 
  # permissions_boundary = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:policy/LabUserNewResourceBoundaryPolicy"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "apigateway.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF
}

Create the API Gateway CloudWatch Role Policy

Next, we will create an AWS IAM role policy that will allow us to access the CloudWatch Logs service. This policy will grant access to the following actions: CreateLogGroup, CreateLogStream, DescribeLogGroups, DescribeLogStreams, PutLogEvents, GetLogEvents, and FilterLogEvents. The policy will also grant access to all resources.

Append this code to the main.tf file:

resource "aws_iam_role_policy" "APIGatewayCloudWatchPolicy" {
  role = aws_iam_role.APIGatewayCloudWatchRole.id

  policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams",
                "logs:PutLogEvents",
                "logs:GetLogEvents",
                "logs:FilterLogEvents"
            ],
            "Resource": "*"
        }
    ]
}
EOF
}

Create a Gateway Setting to Push Logs to CloudWatch

Next, we will create an API Gateway Method Settings resource. This resource will enable CloudWatch logging and metrics for the API Gateway stage we created previously. We will set the rest_api_id to the ID of the API Gateway Rest API resource, the stage_name to the stage_name of the API Gateway Stage resource, and the method_path to "*/*". We will also set the metrics_enabled setting to true and the logging_level setting to "INFO".

Append this code to the main.tf file:

resource "aws_api_gateway_method_settings" "MyApiGatewaySetting" {
  rest_api_id = aws_api_gateway_rest_api.MyApiGatewayRestApi.id
  stage_name  = aws_api_gateway_stage.MyApiGatewayStage.stage_name
  method_path = "*/*"

  settings {
    # Enable CloudWatch logging and metrics
    metrics_enabled = true
    logging_level   = "INFO"
  }
}

Create an API Gateway Usage Plan

Next, we will create an AWS API Gateway Usage Plan using Terraform. This code will create a usage plan with a limit of 1000 requests per month, a burst limit of 20 requests, and a rate limit of 100 requests. Additionally, this code will associate the usage plan with an API Gateway Rest API and Stage that have already been created.

Append this code to the main.tf file:

resource "aws_api_gateway_usage_plan" "MyApiGatewayUsagePlan" {
  name = "apigw-dynamodb-terraform-usage-plan"

  quota_settings {
    limit  = 1000
    period = "MONTH"
  }

  throttle_settings {
    burst_limit = 20
    rate_limit  = 100
  }

  api_stages {
    api_id = aws_api_gateway_rest_api.MyApiGatewayRestApi.id
    stage  = aws_api_gateway_stage.MyApiGatewayStage.stage_name
  }
}

Create an API Gateway Key

Next, we will create an API key for our API Gateway using the aws_api_gateway_api_key resource. This code will create an API key with the name "apigw-dynamodb-terraform-api-key" that can be used to authenticate requests to our API Gateway.

Append this code to the main.tf file:

resource "aws_api_gateway_api_key" "MyAPIKey" {
  name = "apigw-dynamodb-terraform-api-key"
}

Create an API Gateway Usage Plan

Next, we will create an API Gateway Usage Plan Key, which is used to control access to an API Gateway Usage Plan. This code creates a resource called "MyAPIGWUsagePlanKey" that uses the ID of an API Key aws_api_gateway_api_key.MyAPIKey.id and an API Gateway Usage Plan aws_api_gateway_usage_plan.MyApiGatewayUsagePlan.id to control access to the API Gateway Usage Plan.

Append this code to the main.tf file:

resource "aws_api_gateway_usage_plan_key" "MyAPIGWUsagePlanKey" {
  key_id        = aws_api_gateway_api_key.MyAPIKey.id
  key_type      = "API_KEY"
  usage_plan_id = aws_api_gateway_usage_plan.MyApiGatewayUsagePlan.id
}

Create the API Gateway CloudWatch Group

Next, we will create an AWS CloudWatch Log Group using Terraform. This code will create a log group with the name prefix "/aws/APIGW/terraform". This log group will be used to store log data from the API Gateway service. This log group will be created in the same region as the rest of the resources in your Terraform configuration.

Append this code to the main.tf file:

resource "aws_cloudwatch_log_group" "MyLogGroup" {
  name_prefix = "/aws/APIGW/terraform"
}

Create the CloudWatch Log Resource Policy

Next, we will create an AWS CloudWatch Log Resource Policy using Terraform. This policy will allow the Amazon API Gateway and Amazon CloudWatch Logs services to create log streams and put log events into the log group we created earlier. The policy document contains the necessary permissions and conditions to ensure that only the specified services can access the log group.

Append this code to the main.tf file:

resource "aws_cloudwatch_log_resource_policy" "MyCloudWatchLogPolicy" {
  policy_name     = "Terraform-CloudWatchLogPolicy-${data.aws_caller_identity.current.account_id}"
  policy_document = <<POLICY3
{
  "Version": "2012-10-17",
  "Id": "CWLogsPolicy",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [ 
          "apigateway.amazonaws.com",
          "delivery.logs.amazonaws.com"
          ]
      },
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents"
        ],
      "Resource": "${aws_cloudwatch_log_group.MyLogGroup.arn}",
      "Condition": {
        "ArnEquals": {
          "aws:SourceArn": "${aws_api_gateway_rest_api.MyApiGatewayRestApi.arn}"
        }
      }
    }
  ]
}
POLICY3  
}

Create the Terraform Outputs

Lastly, let's create some outputs. We will need these outputs to test the solution.

Append this code to the main.tf file:

output "APIGW-URL" {
  value       = "${aws_api_gateway_stage.MyApiGatewayStage.invoke_url}/pets"
  description = "The API Gateway Invocation URL"
}

# Display the APIGW Key to use for testing
output "APIGW-Key" {
  value       = aws_api_gateway_usage_plan_key.MyAPIGWUsagePlanKey.value
  description = "The APIGW Key to use for testing"
}

Deploying the Project

Now that we have all of our code written, we can deploy the project. Open a terminal, navigate to the project, and run these commands.

# initialize the project 
$ terraform init 

# plan the project 
$ terraform plan 

# apply the project 
$ terraform apply

Testing the Project

To test our project we will create a record and read it.

First, let's create a record in the table.

  • Add your API key in the <KEY> placeholder

  • Add your API url in the <URL> placeholder

# create a record curl -H 'x-api-key: <KEY>' -H 'Content-Type: application/json' --request POST '<URL>' --data-raw '{ "PetType": "dog", "PetName": "tito", "PetPrice": 250 }'

Next, let's read the record that you just created. Again, replace the key and URL with your values.

  • Add your API key in the <KEY> placeholder

  • Add your API url in the <URL>/dog placeholder

# read the record 
$ curl -H 'x-api-key: <KEY>' --request GET '<URL>'

Source

This project was sourced from the AWS Repo: https://github.com/aws-samples/serverless-patterns