Here is some information about this architecture.
Here are the steps you can follow to build this solution on your own.
In this lesson, you will learn about Terraform’s remote modules.
As you have learned in the Modules lesson of this course, Terraform modules provide us with the flexibility of splitting up our configuration file into small, manageable chunks. It also gives us the ability to reuse a Terraform configuration without rewriting or copying and pasting. Apart from storing and using Terraform modules on our local computer, Terraform also allows us to store and retrieve modules from remote repositories.
Remote modules are Terraform configuration modules that are stored and retrieved from a remote storage location. Using Terraform's remote module, you can write a configuration once and use it many times by fetching it directly from the online storage system.
Remote file repositories such as GitHub, Bitbucket, Amazon S3, and Terraform Registry are often used to store and fetch remote modules.
As we highlighted earlier, Terraform allows you to store and use modules from different sources. Let's learn how to use Terraform modules that are stored on some of the most common remote storage systems.
S3 buckets are one of the most widely used remote module storage systems supported by Terraform. To access a module stored in an S3 bucket, you must have the appropriate AWS permissions and credentials in your Terraform project—either in your configuration, as a shared credential, or through environment variables. With the right permissions, you can access the modules in an S3 bucket in multiple ways:
Using the S3 bucket protocol:
module "consul" {
source = "s3::https://s3-eu-west-1.amazonaws.com/consulbucket/consul.zip"
}
Using the bucket URL:
module "consul" {
source = "consulbucket.s3-eu-west-1.amazonaws.com/consul.zip"
}
To access a remote Terraform module from GitHub, you just need to input the URL of the module repository as the value of the source argument:
module "consul" {
source = "github.com/hashicorp/example"
}
If the module is stored in a subdirectory of a GitHub repository, you can reference it in the URL as used in the following examples.
Fetches through HTTPS:
module "consul" {
source = "github.com/hashicorp/example//subdir"
}
Fetches through SSH:
module "consul" {
source = "git@github.com:hashicorp/example.git//subdir"
}
To fetch a Terraform module from a private GitHub repository, you must provide Terraform with the necessary credentials to authenticate you as a user with read access to the repository.
If the git profile on your local computer has read access to the private repository, you can specify the module source as an SSH URL, and terraform will use your default SSH credentials to authenticate the access:
module "consul" {
source = "git@github.com:hashicorp/example.git//subdir"
}
If you're on a different computer or a remote machine, an easy way to go about it is to create a GitHub machine user with read access to the Terraform module repository in question. Then you will embed its credentials into the module's source argument:
module "consul" {
source = "git::https://MACHINE-USER:MACHINE-PASS@github.com/org/privatemodules//modules/foo"
}
An HTTP or HTTPS URL can also be used to redirect Terraform to get a module from any of the other sources mentioned above. Terraform will make a GET request to the given URL and resolve the module URL by going through the following steps:
Look through the module URL to find an X-Terraform_Get header, which should contain the actual URL of the module source.
Look through the HTTP(S) page for a <meta> with the name terraform-get. For example:
<meta name="terraform-get" content="github.com/hashicorp/example" />
After performing the checks and getting the actual source URL, Terraform will fetch the module as appropriate.
Terraform Registry is a remote storage system that holds Terraform configuration packages, modules, providers, libraries, and other Terraform resources. Think of Terraform Registry as a package storage system like NPM. As NPM is used to store, install and use Node.js packages, Terraform registry is used to store and retrieve Terraform packages.
Terraform Registry allows you to use configuration code created by others in your project. You can easily use a module that is stored in the Terraform registry by specifying the source string in the module block:
module "consul" {
source = "hashicorp/consul/aws"
version = "0.1.0"
}
The code above uses the AWS consul module that is stored on the remote Terraform registry.
You will learn more about the registry in the lab section of this lesson.
After creating a module locally, you can publish it to Terraform's public or private registry so that you can fetch it from there whenever you want to create a similar architecture.
To publish a module on the public registry, Terraform requires you to fulfill the following requirements:
Module Storage: To publish a module on Terraform Registry, you must upload it to GitHub and the repository must be public.
Naming Convention: The module repository must be named in a three-part format: terraform-<PROVIDER>-<NAME>. The <PROVIDER> part should be the name of the main provider where the code creates the infrastructure. The <NAME> part should be the name of the kind of infrastructure the module configures. It can be a multiple-word name separated by hyphens, such as s3-bucket.
Repository Description: The module's GitHub repository must include a simple, one-sentence description of the module. This is what Terraform uses to populate the description of the module in the registry.
Module Structure: The module must adhere to the standard module structure defined by Terraform. Refer to the Modules lesson of this course or the Terraform standard module structure doc to get an overview of the structure.
Release Tag: Terraform Registry uses tags to version modules. Therefore, it requires that you include a release tag in the format x.y.z. For example, 1.4.1, or optionally, v1.4.1—appending v to the tag number.
Note that the group of requirements above is specific to a public registry. If you want to register your module in a private registry, the requirements are slightly different.
Once you complete these requirements, you can proceed to turn your locally created module into a remote module on Terraform Registry.
In this lab session, you will create a Terraform module locally, then upload it to Terraform Registry.
Since Terraform Registry requires that we upload our module to GitHub, we will start by creating the repository.
Go to GitHub and sign in to your account.
Create a new repository.
Ensure to name the repository according to the recommended naming convention, i.e terraform-aws-s3-bucket.
Add a description: A Terraform module that provisions AWS S3 buckets.
Make the repository public.
Initialize a README file with the repository.
Choose a license (MIT License).
Click on “Create Repository”.
You will clone the repository on GitHub after successfully creating it on GitHub.
Copy the URL of your repository.
Open your terminal or command line.
Run git clone <url>
Note: Replace <url> with the URL of your GitHub repository.
Once Git successfully clones the repository to your computer, open the folder in your code editor.
In line with the Terraform standard file structure, we will now create three files:
main.tf
variables.tf
outputs.tf
Run the following command on the CLI to create the files:
$ touch main.tf variables.tf outputs.tf
After creating the file structure, it's time to populate the files with our desired configuration. Our aim is to create a module that provisions AWS S3 buckets.
Open the main.tf file and include the following configuration:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
provider "aws" {
region = s3_region
}
resource "aws_s3_bucket_acl" "my_bucket_acl" {
bucket = aws_s3_bucket.my_bucket.id
acl = s3_acl
}
resource "aws_s3_bucket" "my_bucket" {
bucket = s3_bucket_name
}
Open variables.tf and paste the following code:
#define the variables
variable "s3_region"{
type = string
description = "S3 bucket region"
}
variable "s3_bucket_name"{
type = string
description = "S3 bucket name"
}
variable "s3_acl"{
type = string
description = "S3 bucket ACL"
}
Open outputs.tf and include the following:
output "s3_bucket_id" {
value = aws_s3_bucket.my_bucket.id
}
output "s3_bucket_arn" {
value = aws_s3_bucket.my_bucket.arn
}
Ensure that you save all the files.
Now that we are done creating our S3 module, it's time to commit and push it to GitHub.
Run the following commands, one after the other:
#Add all files to the git staging area
$ git add .
#commit the files to the local repository
$ git commit -m "Initial commit"
#Push files to remote GitHub repository
$ git push
To tick all the boxes required to publish our module on Terraform Registry, we need to include a release tag.
Run the following commands:
#add release tag to the local repository
$ git tag v1.0.0
#push release tag to remote GitHub repository
$ git push --tags
After running all the above commands successfully, check your GitHub repository to ensure that the files are there.
Let’s get started with publishing our remote module on Terraform Registry.
Visit registry.terraform.io, sign in with GitHub and authorize HashiCorp.
Once you're logged in, click on “Module” in the "Publish" drop-down menu.
Click the checkbox titled "Select Repository on GitHub". Since we followed all the prerequisites, our module repository will show up in the dropdown.
Click on the module repository name.
Check the checkbox, then click "Publish Module".
Once the module is published successfully, you will automatically be redirected to your new remote module page.
Congratulations, you now know how to create a module and publish it on Terraform Registry.
Now that you have published your S3 bucket module on Terraform Registry, you can use it in other projects by fetching it from the remote repository.
In the “Provisioning Instruction” pane on the right side of the module page, you will find the code that you can use to fetch the remote module in your projects.