Here is some information about this architecture.
Here are the steps you can follow to build this solution on your own.
In this lesson, you will learn about automating Terraform.
Making the provisioning, configuring, and managing of infrastructure automated and repeatable is the primary purpose of an IaC tool like Terraform. While using Terraform in its most basic form satisfies the requirement for infrastructure automation by allowing you to manage infrastructure resources with a single CLI command, there is still room for more automation.
Running Terraform workflow commands repeatedly can become tiresome when managing the provisioning of numerous infrastructures on a large scale. Likewise, when using modern continuous integration and continuous deployment (CI/CD) tools, it is essential that you can integrate Terraform into the pipeline to provision infrastructure on the fly. What if you could run your Terraform configuration on autopilot?
Fortunately, Terraform allows you to externally drive the workflow without interacting with the pipeline to ensure consistency between runs, integrate with CI/D pipelines and reduce human interaction that can introduce errors, especially in teams with multiple members working on the project.
Fundamentally, you can automate Terraform using CI/CD tools, custom automation systems, or Terraform Cloud.
Terraform Cloud
Terraform Cloud provides a variety of features centered on automating Terraform workflows. It provides features that extend the core Terraform CLI functionality by:
integrating with version control systems
automating plan and apply lifecycles
running security and validity checks
serving as remote state storage.
You can configure these Terraform Cloud features to achieve as much workflow automation as it provides.
Alternatively, you can run the Terraform configuration using a different automation system while using Terraform Cloud as a remote backend to store your Terraform state file.
CI/CD Tools
Continuous integration and continuous deployment (CI/CD) tools like Jenkins, GitHub Actions, and CircleCI provide an ideal environment for running Terraform in automation. By writing your preferred command sequence in scripts that the CI/CD system executes, you can use any of these tools to drive Terraform workflows automatically.
Custom Automation Systems
Some organizations will prefer to build a homegrown solution for managing their workflows. Terraform automation is possible with such solutions as well, so long as it provides the CLI, file system, and internet access that Terraform needs to function.
The outcome of running Terraform in automation is the same as when you run all workflows through the CLI, but because there is no human interaction, there are some things you should keep in mind before you start.
CLI Workflow
Automation of Terraform eliminates interactions that let you input CLI options and respond to prompts. As a result, you need a way to provide such options while Terraform is running in an automation system. The following commands can be used in your automation scripts in place of the usual workflow commands to suffice for the absence of CLI interaction:
terraform init -input=false to initialize the working directory.
terraform plan -out=tfplan -input=false to create a plan and save it to the tfplan file.
terraform apply -input=false tfplan to apply the plan stored in the tfplan file.
The -input=false option instructs Terraform not to ask for any input and to anticipate finding necessary values in the configuration file. For this reason, it is recommended that you provide values for your variables in a .tfvars file. Alternatively, in your automation script, you can use the -var or -var-file options on the terraform plan to specify the values for any variables.
Terraform Output
Terraform typically concludes the execution of a command by offering the user a potential next action or command. Because an automation system may not display the command it just completed, Terraform's suggestion can appear ambiguous or misleading—especially if it suggests that the user bypasses security checks or the automation tool entirely.
Setting the TF_IN_AUTOMATION environment variable to any acceptable, non-empty value stops it from generating command line suggestions. It tells Terraform that it is running in an automated environment that will handle the next step automatically.
Planning and Applying on Different Machines
When running Terraform in an orchestration tool, there are chances that the plan will be run on a particular machine and applied on another. In such cases, you must take extra measures to ensure Terraform builds, deletes, or modifies the configuration as intended.
After the plan is completed, archive the entire Terraform working directory and save it to a location where it will be accessible to the machine that will run the apply step—a "build artifact" in the orchestration tool is usually a good choice.
Before running the apply step, obtain the archive created in the previous step and extract it at the same absolute path. This recreates everything that existed after the plan in the new machine, avoiding strange behaviors.
The saved plan file may contain absolute paths to child files and modules. Therefore, the new machine where the apply step will be run must have a similar file structure to the machine that runs the plan step. Running the plan and apply steps in isolated machines such as Docker containers ensures consistency in the file system.
The operating system and CPU architecture of the machine that runs the plan must be identical to the machine that applies the plan.
The version of Terraform provider plugins in the planning environment must be identical to that of the application environment to ensure correct interpretation and avoid errors.
Auto-Approval of Plans
The goal of automating Terraform is, typically, complete automation—which includes the automatic application of plans. Although it is strongly advised to review plans manually before implementing them, such rules can be easily disregarded in a pre-production or testing environment. You can omit the plan command from your automation script and automatically approve a configuration as shown below:
$ terraform init -input=false
$ terraform apply -input=false -auto-approve
Using the apply command as written above generates a plan and automatically applies the plan without prompting for approval.
Pre-installed Plugins
When you run terraform init, Terraform automatically downloads and installs all required provider plugins and stores them in the .terraform directory. This is ideal for an interactive workflow because it is more straightforward—and you can easily resolve any plugin version issues with a single command. But in automated settings, it might not be a good idea.
It would be best if you provided Terraform with a pre-installed fixed set of plugins rather than allowing it to download the necessary plugins automatically. Using the -plugin-dir option, you can specify the pre-installed fixed plugin set in the terraform init command as shown below:
terraform init -input=false -plugin-dir=/usr/lib/custom-terraform-plugins
Ensure that the plugins you download from releases.hashicorp.com are the correct versions for the target operating system.
Alongside all the ones highlighted above, Terraform considers and provides support for situations where you may want to:
interactively approve plans
test validity of configuration file using terraform plan
deploy the same configuration in multiple environments.
Endeavor to read the Terraform documentation to get an in-depth understanding of the various instances where you need to adjust Terraform for an automated workflow.
In this lab, you will use a continuous integration tool, GitHub Action, to automate Terraform.
If you're using the Skillmix Labs feature, open the lab settings (the beaker icon) on the right side of the code editor. Then, click the Start Lab button to start hte lab environment.
Wait for the credentials to load. Then run this in the terminal.
Be sure to enter in your own access key and secret key and name your profile 'smx-lab'.
$ aws configure --profile smx-lab
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]: us-west-2
Default output format [None]: Note: If you're using your own AWS account you'll need to ensure that you've created and configured a named AWS CLI profile named smx-lab.
First, create the GitHub repository.
Create a new repository on your GitHub account.
Use "skillmix-demo" as the repository name of the repository.
Skip other options and click “Create Repository.”
We need to configure our AWS credentials in the repository.
In the newly created repository, navigate to “Settings” >> “Secrets” >> "Actions"
Create two new repository secrets with the following names:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
Set the values to the AWS credentials you obtained from the Lab Environment you created earlier.
Open your command line and clone the newly created GitHub repository to your local environment using the command below:
$ git clone https://github.com/<YOUR-USER-NAME>/skillmix-demo
Replace <YOUR-USER-NAME> above with your GitHub username. If you used a different repository name, you should also replace skillmix-demo with that name.
It is time to create our Terraform configuration.
Create a main.tf file in the directory of the cloned GitHub repository:
#change directory
$ cd skillmix-demo
#create the main.tf file
$touch main.tf
Open the main.tf file in your code editor, paste the following code, and save it:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
provider "aws" {
region = "us-west-2"
}
#create EC2 instance
resource "aws_instance" "web_server" {
ami = "ami-0c2ab3b8efb09f272"
instance_type = "t2.micro"
tags = {
Name = "skillmix-lab-instance"
}
}
You should be familiar with the code above. The configuration creates a simple EC2 instance of type t2.micro.
In the Terraform project directory, create a terraform.yml file with the path as follows .github/workflows/terraform.yml:
#create the .github directory
$ mkdir .github
#cd to the .github directory
$ cd .github
#create a workflows subdirectory
$ mkdir workflows
#cd to the workflows subdirectory
$ cd workflows
#create a terraform.yml file
$ touch terraform.yml
Open the terraform.yml file and input the following code:
name: "Skillmix Automated Terraform"
on:
push:
branches:
- main
jobs:
terraform:
name: "Terraform"
runs-on: ubuntu-latest
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: "us-east-1"
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup Terraform
uses: hashicorp/setup-terraform@v1
- name: Terraform Init
id: init
run: terraform init -input=false
- name: Terraform Validate
id: validate
run: terraform validate -no-color
- name: Terraform Plan
id: plan
if: github.event_name == 'push'
run: terraform plan -no-color -input=false
- name: Terraform Apply
if: github.ref == 'refs/head/main' && github.event.name == 'push'
run: terraform apply -auto-approve -input=false
Code Review
name: "Skillmix Automated Workflow" specifies a name for the GitHub Actions workflow.
on:
push:
branches:
- main
The on block declares the conditions that should be met before this workflow run. In this case, we declared that the workflow should run when a commit is pushed to the repository.
The jobs: block declares all the actions that the workflow should perform.
terraform:
terraform:
name: "Terraform"
runs-on: ubuntu-latest
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: "us-east-1"
This declares a Terraform that runs on a machine running the most recent version of Ubuntu. In the env: block, we specify that Terraform retrieves the necessary authentication credentials from the secrets we created in the GitHub repository earlier and runs the configuration in the us-west-2 region.
steps:
This defines following 6 steps of action for Terraform to carry out in this workflow:
Check out
- name: Checkout
uses: actions/checkout@v3
This will check out from the current configuration using the GitHub Action's action/checkout@v3 docker image. It also initiates the CLI to be used in the workflow.
Setup Terraform
- name: Setup Terraform
uses: hashicorp/setup-terraform@v1
This step retrieves the CLI from the previous step and configures a Terraform environment with the most recent stable version of Terraform.
Terraform Init
- name: Terraform Init
id: init
run: terraform init -input=false
This initializes Terraform in the GitHub Action workflow, downloading the necessary provider plugins as specified in the configuration file.
Terraform Validate
- name: Terraform Validate
id: validate
run: terraform validate -no-color
This function validates the configuration.
Terraform Plan
- name: Terraform Plan
id: plan
if: github.event_name == 'push'
run: terraform plan -no-color -input=false
The if attribute declares the condition to be met before this step runs. Terraform will generate a plan within the GitHub Actions Workflow whenever there is a push event in the current branch.
Terraform Apply
- name: Terraform Apply
if: github.ref == 'refs/head/main' && github.event.name == 'push'
run: terraform apply -auto-approve -input=false
This final step involves running the apply command after the conditions have been met. Terraform will automatically apply our configuration once a push event occurs and the event happens in the main branch.
Push Files to GitHub
After creating the configuration and automation files, it's time to push them to GitHub. Open your command line and run the following GitHub workflow commands.
#add all files to the staging area
$ git add .
#commit files to local repository
$ git commit -m "Populate configuration and automation files"
#push code to remote repository
$ git push -u origin main
We have fulfilled the requirement in our terraform.yml file, so the automated workflow must have been triggered.
Navigate to the GitHub repository.
Go to the Actions tab in your GitHub repository; you should see the workflow similar to the snapshot below.
Select the workflow run to see the steps that was executed.
Expand each of the steps to see the CLI logs.
Congratulations, you now understand how to use GitHub actions to automate Terraform.