Public ALB to Private API Gateway

Interactive Mode

About this Architecture

Here is some information about this architecture.

This innovative serverless pattern enables seamless access for users across multiple AWS accounts to their respective Private REST APIs via the internet. This pattern serves to promote the separation of network resources, such as load balancers, virtual private clouds, and endpoints, and application resources, such as Lambda and API Gateway, into distinct and manageable accounts.

How to Build This Solution

Here are the steps you can follow to build this solution on your own.

Create the Terraform File

We'll be doing all of our work in one Terraform file. Create a new directory on your computer somewhere, and then create a file named main.tf in it.

Create the Terraform & Provider Block

Next, we will create a Terraform configuration that will allow us to use the AWS provider. This configuration will require us to specify the version of the AWS provider that we want to use, as well as the version of Terraform that we are using. We will also specify the AWS profile and region that we want to use. This code will ensure that the correct versions of Terraform and the AWS provider are used, and that the AWS provider is configured correctly.

Append this code to the main.tf file:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.11.0"
    }
  }
}

# account 1 provider
provider "aws" {
  profile = "smx-temp"
  region  = "us-west-2"
}

# account 2 provider
provider "aws" {
 profile = "crossaccount"
 region  = "us-west-2"
 alias   = "crossaccount"
}

Create the Data Resource Blocks

Next, we will create a data source that will allow us to access the AWS caller identity of the current user. This data source will be called "aws_caller_identity" and will be set to the "current" provider, which is the AWS Cross Account provider. This data source will allow us to access the identity of the current user, which can be used for authentication and authorization purposes.

Append this code to the main.tf file:

data "aws_caller_identity" "current" {
  provider = aws.crossaccount
}

Create Variables

Next, we will create three variables that will be used to define the region, custom domain name prefix, and domain name for our Terraform code. The first variable, "region", will define the region in which the code will be executed. The second variable, "custom_domain_name_prefix", will define the custom domain name prefix that will be used in the code. Finally, the third variable, "domain_name", will define the domain name that will be used in the code.

Append this code to the main.tf file:

variable "region" {}

variable "custom_domain_name_prefix" {}

variable "domain_name" {}

Create the Public Load Balancer

Next, we will create an Application Load Balancer (ALB) using the aws_lb resource. This ALB will be named "public-alb-test" and will be accessible from the public internet. It will be associated with the security group that allows HTTPS traffic, and will be placed in two public subnets.

Append this code to the main.tf file:

resource "aws_lb" "public_alb" {
  name               = "public-alb-test"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.allow_https.id]
  subnets            = [aws_subnet.public_subnet.id,aws_subnet.public_subnet2.id]
}

Create the Load Balancer Target Group

Next, we will create an AWS load balancer target group using the Terraform code above. This target group will be named "tf-example-lb-tg" and will use port 443 and the HTTPS protocol. It will use an IP target type and will be associated with the VPC ID of the VPC created earlier. Additionally, the health check will use port 443 and will look for a response of either 200 or 403.

Append this code to the main.tf file:

resource "aws_lb_target_group" "test" {
  name     = "tf-example-lb-tg"
  port     = 443
  protocol = "HTTPS"
  target_type = "ip"
  vpc_id   = aws_vpc.vpc.id

  health_check {
    matcher     = "200,403"
    port        = 443
  }
}

Crate the Target Group Attachement

Next, we will create a resource called "aws_lb_target_group_attachment" with the name "nic01". This resource will attach a target group to a network interface, allowing traffic to be routed to the network interface. The target group is specified by the "target_group_arn" argument, which is set to the ARN of the target group created earlier. The "target_id" argument is set to the private IP address of the network interface, and the "port" argument is set to 443, which is the port used for HTTPS traffic.

Append this code to the main.tf file:

resource "aws_lb_target_group_attachment" "nic01" {
  target_group_arn = aws_lb_target_group.test.arn
  target_id        = data.aws_network_interface.nic1.private_ip
  port             = 443
}

Create the Load Balancer Listener

Next, we will create an AWS load balancer listener using the Terraform code above. This listener will be configured to use HTTPS on port 443, with the ELBSecurityPolicy-2016-08 SSL policy and the certificate ARN specified in the aws_acm_certificate resource. The listener will forward requests to the target group specified in the aws_lb_target_group resource.

Append this code to the main.tf file:

resource "aws_lb_listener" "front_end" {
  load_balancer_arn = aws_lb.public_alb.arn
  port              = "443"
  protocol          = "HTTPS"
  ssl_policy        = "ELBSecurityPolicy-2016-08"
  certificate_arn   = aws_acm_certificate.cert.arn

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.test.arn
  }
}

Create the Certificate Resource

Next, we will create an AWS ACM Certificate resource using Terraform. This resource will allow us to securely connect to our custom domain name. The code will create a certificate with the domain name specified in the variables custom_domain_name_prefix and domain_name, and will use the DNS validation method to validate the certificate.

Append this code to the main.tf file:

resource "aws_acm_certificate" "cert" {
  domain_name       = "${var.custom_domain_name_prefix}.${var.domain_name}"
  validation_method = "DNS"
}

Create the Route53 Zone

Next, we will create a data source that will allow us to access the Route 53 zone associated with the domain name specified in the variable. The data source will be called "r53" and will use the aws_route53_zone data source type. We will specify the domain name in the name parameter and set the private_zone parameter to false, indicating that this is a public zone.

Append this code to the main.tf file:

data "aws_route53_zone" "r53" {
  name         = "${var.domain_name}"
  private_zone = false
}

Create a Route53 Record

Next, we will create a resource in Terraform that will create a Route 53 record for each domain validation option associated with an AWS ACM certificate. This resource will allow us to overwrite existing records, set the name, records, TTL, and type of the record, and specify the Route 53 zone ID.

Append this code to the main.tf file:

resource "aws_route53_record" "cert_record" {
  for_each = {
    for dvo in aws_acm_certificate.cert.domain_validation_options : dvo.domain_name => {
      name   = dvo.resource_record_name
      record = dvo.resource_record_value
      type   = dvo.resource_record_type
    }
  }

  allow_overwrite = true
  name            = each.value.name
  records         = [each.value.record]
  ttl             = 60
  type            = each.value.type
  zone_id         = data.aws_route53_zone.r53.zone_id
}

Create the Certificate Validation Resource

Next, we will create an AWS ACM Certificate Validation resource. This resource will validate the certificate that we created in the previous step. It will use the certificate's ARN (Amazon Resource Name) and the FQDNs (Fully Qualified Domain Names) of the Route 53 records that we created. This will ensure that the certificate is valid and can be used for our application.

Append this code to the main.tf file:

resource "aws_acm_certificate_validation" "validation" {
  certificate_arn         = aws_acm_certificate.cert.arn
  validation_record_fqdns = [for record in aws_route53_record.cert_record : record.fqdn]
}

Create the Load Balancer CNAME Record

Next, we will create a resource in AWS Route 53. This code will create a CNAME record, which is used to point a domain name to the load balancer. The record will have a TTL (time to live) of 300 seconds, and the records will be set to the DNS name of the public ALB (Application Load Balancer) that we created earlier.

Append this code to the main.tf file:

resource "aws_route53_record" "record" {
  zone_id = data.aws_route53_zone.r53.zone_id
  name    = "${var.custom_domain_name_prefix}"
  type    = "CNAME"
  ttl     = "300"
  records = [aws_lb.public_alb.dns_name]
}

Create the Lambda Permissions Resource

Next, we will create an AWS Lambda permission that allows API Gateway to invoke the Lambda function. This code creates an AWS Lambda permission resource with a statement ID, action, function name, principal, source ARN, and provider. The statement ID is "AllowExecutionFromAPIGateway", the action is "lambda:InvokeFunction", the function name is the name of the Lambda function, the principal is "apigateway.amazonaws.com", the source ARN is the execution ARN of the API Gateway deployment and the HTTP method of the API Gateway method, and the provider is "aws.crossaccount".

Append this code to the main.tf file:

resource "aws_lambda_permission" "apigw_lambda" {
  statement_id  = "AllowExecutionFromAPIGateway"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda.function_name
  principal     = "apigateway.amazonaws.com"

  source_arn = "${aws_api_gateway_deployment.deploy.execution_arn}*/${aws_api_gateway_method.get.http_method}/"
  provider = aws.crossaccount
}

Create the Lambda Function File

We will need a zip file of a function to use as our Lambda function code. This is a two step part. First, create a file named lambda_function.py in the root of your project. In that file add the code below. Then, zip that file and name the zipped file code.zip.

import json

def lambda_handler(event, context):
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Create the Lambda Function Resource

Next, we will create an AWS Lambda function using Terraform. This code will create a Lambda function called "mylambda" with the filename "code.zip" and a runtime of "python3.9". The role for this Lambda function will be set to the ARN of an IAM role that we will create in a later step. The handler for this Lambda function will be set to "lambda_function.lambda_handler" and the provider will be set to "aws.crossaccount". Finally, the source code hash will be set to the base64 SHA256 hash of the "code.zip" file.

Append this code to the main.tf file:

resource "aws_lambda_function" "lambda" {
  filename      = "code.zip"
  function_name = "mylambda"
  role          = aws_iam_role.role.arn
  handler       = "lambda_function.lambda_handler"
  runtime       = "python3.9"

  source_code_hash = filebase64sha256("code.zip")
  provider = aws.crossaccount
}

Create the Lambda IAM Role

Next, we will create an AWS IAM role using Terraform. This role will be named "myrole" and will allow the service "lambda.amazonaws.com" to assume the role. This will be done using the assume_role_policy and the provider will be set to "aws.crossaccount".

Append this code to the main.tf file:

resource "aws_iam_role" "role" {
  name = "myrole"

  assume_role_policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
POLICY
provider = aws.crossaccount
}

Create the VPC Resource

Next, we will create an AWS VPC (Virtual Private Cloud) using the Terraform resource "aws_vpc". This code will create a VPC with a CIDR block of 20.0.0.0/16 and enable DNS hostnames. This will allow us to create a secure, isolated network for our resources.

Append this code to the main.tf file:

resource "aws_vpc" "vpc" {
  cidr_block = "20.0.0.0/16"
  enable_dns_hostnames = "true"
}

Create the VPC Subnets

Next, we will create three subnets in two different availability zones. The first two subnets will be in the same availability zone, and the third subnet will be in a different availability zone. The first two subnets will have the same CIDR block and tags, while the third subnet will have a different CIDR block and tags. The subnets will be created using the Terraform resource "aws_subnet" and the variables "vpc_id", "cidr_block", "availability_zone", and "tags".

Append this code to the main.tf file:

resource "aws_subnet" "private_subnet" {
  vpc_id = aws_vpc.vpc.id
  cidr_block = "20.0.0.0/24"
  availability_zone = "${var.region}a"
  tags = {
    Name = "Subnet for ${var.region}a"
  }
}

resource "aws_subnet" "public_subnet" {
  cidr_block = "20.0.1.0/24"
  vpc_id = aws_vpc.vpc.id
  availability_zone = "${var.region}a"
  tags = {
    Name = "Subnet for ${var.region}a"
  }
}

resource "aws_subnet" "public_subnet2" {
  cidr_block = "20.0.2.0/24"
  vpc_id = aws_vpc.vpc.id
  availability_zone = "${var.region}c"
  tags = {
    Name = "Subnet for ${var.region}c"
  }
}

Create the Internet Gateway resource

Next, we will create an Internet Gateway resource in Terraform. This resource will be associated with the VPC we created earlier, and will be tagged with the name "igw". This will allow us to easily identify the Internet Gateway in the AWS console.

Append this code to the main.tf file:

resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.vpc.id

  tags = {
    Name = "igw"
  }
}

Create a Route Table & Private Route

Next, we will create a private route table for our VPC. This code creates an AWS Route Table resource with the name "private_rt" and assigns it to the VPC we created earlier. The tags field allows us to assign a name to the resource, which in this case is "private_rt". This will help us identify the resource in the future.

Append this code to the main.tf file:

resource "aws_route_table" "private_rt" {
  vpc_id = aws_vpc.vpc.id

  tags = {
    Name = "private_rt"
  }
}

Create a Route Table & Public Route

Next, we will create an AWS route table called "public_rt" that is associated with the VPC we created earlier. This route table will have a route that points to an internet gateway, allowing traffic to flow out of the VPC. Finally, we will add a tag to the route table so that it can be easily identified.

Append this code to the main.tf file:

resource "aws_route_table" "public_rt" {
  vpc_id = aws_vpc.vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }

  tags = {
    Name = "public_rt"
  }
}

Create Route Table Associations

Next, we will create three route table associations to link our subnets to the appropriate route tables. The first association will link the private subnet to the private route table, the second will link the public subnet to the public route table, and the third will link the second public subnet to the public route table.

Append this code to the main.tf file:

resource "aws_route_table_association" "a" {
  subnet_id      = aws_subnet.private_subnet.id
  route_table_id = aws_route_table.private_rt.id
}

resource "aws_route_table_association" "b" {
  subnet_id      = aws_subnet.public_subnet.id
  route_table_id = aws_route_table.public_rt.id
}

resource "aws_route_table_association" "c" {
  subnet_id      = aws_subnet.public_subnet2.id
  route_table_id = aws_route_table.public_rt.id
}

Create the Security Group

Next, we will create an AWS security group called "allow_https" that allows HTTPS inbound traffic from any IP address. This security group will also allow all outbound traffic from any IP address. Finally, we will add a tag to the security group with the name "allow_https".

Append this code to the main.tf file:

resource "aws_security_group" "allow_https" {
  name        = "allow_https"
  description = "Allow HTTPS inbound traffic"
  vpc_id      = aws_vpc.vpc.id

  ingress {
    description      = "HTTPS from VPC"
    from_port        = 443
    to_port          = 443
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = {
    Name = "allow_https"
  }
}

Create the AWS VPC Endpoint

Next, we will create an AWS VPC Endpoint to allow access to the Amazon API Gateway service. This code will create an endpoint in the VPC with the ID specified in the aws_vpc resource, and will use the security group and subnet IDs specified in the aws_security_group and aws_subnet resources. The endpoint will be of type "Interface" and will have private DNS enabled.

Append this code to the main.tf file:

resource "aws_vpc_endpoint" "execute_api" {
  vpc_id            = aws_vpc.vpc.id
  service_name      = "com.amazonaws.${var.region}.execute-api"
  vpc_endpoint_type = "Interface"

  security_group_ids = [
    aws_security_group.allow_https.id,
  ]

  subnet_ids        = [aws_subnet.private_subnet.id]
  private_dns_enabled = true
}

Create the Network Interface Data Resource

Next, we will create a data source that will allow us to access the first network interface associated with the VPC endpoint we created earlier. The data source will be called "nic1" and will use the "aws_network_interface" data source. We will use the "id" argument to specify the first network interface ID from the list of network interface IDs associated with the VPC endpoint.

Append this code to the main.tf file:

data "aws_network_interface" "nic1" {
  id = tolist(aws_vpc_endpoint.execute_api.network_interface_ids)[0]
}

Create the API Gateway Resource

Next, we will create an AWS API Gateway REST API resource called "private_api" that is configured to be private and accessible only through a VPC endpoint. This is done by setting the endpoint configuration to "PRIVATE" and providing the ID of the VPC endpoint as the value for the vpc_endpoint_ids parameter. The provider parameter is set to aws.crossaccount to ensure that the API is accessible from other AWS accounts.

Append this code to the main.tf file:

resource "aws_api_gateway_rest_api" "private_api" {
    name = "private_api"

    endpoint_configuration {
        types = ["PRIVATE"]
        vpc_endpoint_ids = [aws_vpc_endpoint.execute_api.id]
    }
    provider = aws.crossaccount
}

Create the API Gateway Method

Next, we will create an AWS API Gateway Method resource with the name "get" that will allow us to make a GET request to the API. This resource will have no authorization, and will be associated with the root resource ID and the ID of the private API that we created earlier. The provider for this resource will be the AWS Cross Account provider.

Append this code to the main.tf file:

resource "aws_api_gateway_method" "get" {
    authorization = "NONE"
    http_method   = "GET"
    resource_id   = aws_api_gateway_rest_api.private_api.root_resource_id
    rest_api_id   = aws_api_gateway_rest_api.private_api.id
    provider = aws.crossaccount
}

Create the API Gateway Integration

Next, we will create an AWS API Gateway integration. This integration will allow us to proxy requests from the API Gateway to an AWS Lambda function. The integration will use the REST API ID, resource ID, and HTTP method from the API Gateway, and the invoke ARN from the Lambda function. The integration type will be set to AWS_PROXY and the integration HTTP method will be set to POST. Finally, the provider will be set to aws.crossaccount.

Append this code to the main.tf file:

resource "aws_api_gateway_integration" "lambda_proxy" {
    rest_api_id             = aws_api_gateway_rest_api.private_api.id
    resource_id             = aws_api_gateway_rest_api.private_api.root_resource_id
    http_method             = aws_api_gateway_method.get.http_method
    integration_http_method = "POST"
    type                    = "AWS_PROXY"
    uri                     = aws_lambda_function.lambda.invoke_arn
    provider = aws.crossaccount
}

Create the API Gateway Deployment

Next, we will create an AWS API Gateway Deployment resource using the Terraform code above. This resource will trigger a redeployment when changes are made to the REST API, such as the root resource ID, the GET method, or the Lambda proxy integration. Additionally, the lifecycle block will ensure that the deployment is created before it is destroyed. Finally, the provider block will specify that the deployment should be created in the AWS Cross Account.

Append this code to the main.tf file:

resource "aws_api_gateway_deployment" "deploy" {
    rest_api_id = aws_api_gateway_rest_api.private_api.id

  triggers = {
    # NOTE: The configuration below will satisfy ordering considerations,
    #       but not pick up all future REST API changes. More advanced patterns
    #       are possible, such as using the filesha1() function against the
    #       Terraform configuration file(s) or removing the .id references to
    #       calculate a hash against whole resources. Be aware that using whole
    #       resources will show a difference after the initial implementation.
    #       It will stabilize to only change when resources change afterwards.
    redeployment = sha1(jsonencode([
        aws_api_gateway_rest_api.private_api.root_resource_id,
        aws_api_gateway_method.get.id,
        aws_api_gateway_integration.lambda_proxy.id,
    ]))
  }

  lifecycle {
        create_before_destroy = true
  }
  provider = aws.crossaccount
}

Create the API Gateway Stage

Next, we will create an AWS API Gateway stage called "dev". This stage will be associated with the deployment ID of the AWS API Gateway deployment, the REST API ID of the private API, and the stage name of "dev". Additionally, this stage will be created using the AWS Cross Account provider.

Append this code to the main.tf file:

resource "aws_api_gateway_stage" "dev" {
    deployment_id = aws_api_gateway_deployment.deploy.id
    rest_api_id   = aws_api_gateway_rest_api.private_api.id
    stage_name    = "dev"
    provider = aws.crossaccount
}

Create the Gateway API Policy

Next, we will create an AWS API Gateway Rest API policy. This policy will deny access to the API from any source other than the specified VPC endpoint, while allowing access from the specified VPC endpoint. This is done by setting the "Effect" to "Deny" and the "Principal" to "*", and then setting the "Condition" to "StringNotEquals" with the "aws:sourceVpce" set to the ID of the VPC endpoint.

Append this code to the main.tf file:

resource "aws_api_gateway_rest_api_policy" "policy" {
    rest_api_id   = aws_api_gateway_rest_api.private_api.id

  policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Deny",
            "Principal": "*",
            "Action": "execute-api:Invoke",
            "Resource": "${aws_api_gateway_rest_api.private_api.execution_arn}*",
            "Condition": {
                "StringNotEquals": {
                    "aws:sourceVpce": "${aws_vpc_endpoint.execute_api.id}"
                }
            }
        },
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "execute-api:Invoke",
            "Resource": "${aws_api_gateway_rest_api.private_api.execution_arn}*"
        }
    ]
}
EOF
    provider = aws.crossaccount
}

Create the Outputs

Add this to the end of the main.tf file to output the final URL

output "final_url" {
    value = "https://${var.custom_domain_name_prefix}.${var.domain_name}/${aws_api_gateway_stage.dev.stage_name} -H 'Host:${aws_api_gateway_rest_api.private_api.id}.execute-api.${var.region}.amazonaws.com'"
}

Deploying the Project

Now that we have all of our code written, we can deploy the project. Open a terminal, navigate to the project, and run these commands.

# initialize the project 
$ terraform init 

# plan the project 
$ terraform plan 

# apply the project & fill in the prompts
$ terraform apply

   var.custom_domain_name_prefix
    - Enter a value: *{enter only your domain name prefix}*

   var.domain_name
    -  Enter a value: *{enter your domain name}*

   var.region
    - Enter a value: *{enter the region for deployment}*

Test the Project

Get the URL from the Terraform output. Then, execute this command.

$ curl https://<custom_domain>/<stage-nae> -H 'Host:<execute-api-invoke-url>'

You should see the output: Hello from Lambda!

Destroy the Project

Run the destroy command to end it all

$ terraform destroy