Managing Infra with Terraform

Published in

Geek Culture

37 min readAug 7, 2021

Terraform is IaC (Infrastructure as Code) tool to provision and manage any cloud, infrastructure, or service. It follows a declarative-style and push-type model. It is masterless by default and does not require you to install any extra agents.

Disclaimer: This is going to be a lengthy writeup, as we are going to cover almost everything that you need to know to get started, deploy with best practices, troubleshoot, and some tips on figuring out the right configuration.

· Terraform References
· Getting Started with Terraform
· Terraform Installation
· Terraform CLI
∘ Terraform Help
∘ The terraform init
∘ The terraform plan
∘ The terraform refresh
∘ The terraform apply
∘ The terraform destroy
∘ The terraform import
∘ The terraform state
∘ Terraform Commands Workflow
· Terraform Configuration
∘ Terraform Settings
∘ TF Modules
∘ The Providers in TF
∘ The Resources in TF
∘ Data Sources in TF
· Terraform Pull Request Automation: Atlantis

Terraform References

Some standard resources:

Best Practices:

Terraform Recommended Practices by HashiCorp

Some Guides:

Getting Started with Terraform

Terraform Installation

Terraform can be installed by downloading Terraform. On Mac, it can also be installed using brew (but not using apt-get on Ubuntu).

Once installed, you can check its version by running terraform version.

Terraform CLI

Terraform Help

You may check Terraform CLI Documentation.

Here is the list of Terraform commands, which you can obtain by running terraform help:

terraform help
# Usage: terraform [global options] <subcommand> [args]
#
# The available commands for execution are listed below.
# The primary workflow commands are given first, followed by
# less common or more advanced commands.
#
# Main commands:
#   init          Prepare your working directory for other commands
#   validate      Check whether the configuration is valid
#   plan          Show changes required by the current configuration
#   apply         Create or update infrastructure
#   destroy       Destroy previously-created infrastructure
#
# All other commands:
#   console       Try Terraform expressions at an interactive command prompt
#   fmt           Reformat your configuration in the standard style
#   force-unlock  Release a stuck lock on the current workspace
#   get           Install or upgrade remote Terraform modules
#   graph         Generate a Graphviz graph of the steps in an operation
#   import        Associate existing infrastructure with a Terraform resource
#   login         Obtain and save credentials for a remote host
#   logout        Remove locally-stored credentials for a remote host
#   output        Show output values from your root module
#   providers     Show the providers required for this configuration
#   refresh       Update the state to match remote systems
#   show          Show the current state or a saved plan
#   state         Advanced state management
#   taint         Mark a resource instance as not fully functional
#   untaint       Remove the 'tainted' state from a resource instance
#   version       Show the current Terraform version
#   workspace     Workspace management
#
# Global options (use these before the subcommand, if any):
#   -chdir=DIR    Switch to a different working directory before executing the
#                 given subcommand.
#   -help         Show this help output, or the help for a specified subcommand.
#   -version      An alias for the "version" subcommand.

Any of these commands can be added to the -help flag to get more information, such as terraform <command> -help. For example,

terraform plan -help
# Usage: terraform plan [options] [DIR]
#
#   Generates a speculative execution plan, showing what actions Terraform
#   would take to apply the current configuration. This command will not
#   actually perform the planned actions.
#
#   You can optionally save the plan to a file, which you can then pass to
#   the "apply" command to perform exactly the actions described in the plan.
#
# Options:
#
#   -compact-warnings   If Terraform produces any warnings that are not
#                       accompanied by errors, show them in a more compact form
#                       that includes only the summary messages.
#
#   -destroy            If set, a plan will be generated to destroy all resources
#                       managed by the given configuration and state.
#
#   -detailed-exitcode  Return detailed exit codes when the command exits. This
#                       will change the meaning of exit codes to:
#                       0 - Succeeded, diff is empty (no changes)
#                       1 - Errored
#                       2 - Succeeded, there is a diff
#
#   -input=true         Ask for input for variables if not directly set.
#
#   -lock=true          Lock the state file when locking is supported.
#
#   -lock-timeout=0s    Duration to retry a state lock.
#
#   -no-color           If specified, output won't contain any color.
#
#   -out=path           Write a plan file to the given path. This can be used as
#                       input to the "apply" command.
#
#   -parallelism=n      Limit the number of concurrent operations. Defaults to 10.
#
#   -refresh=true       Update state prior to checking for differences.
#
#   -state=statefile    Path to a Terraform state file to use to look
#                       up Terraform-managed resources. By default it will
#                       use the state "terraform.tfstate" if it exists.
#
#   -target=resource    Resource to target. Operation will be limited to this
#                       resource and its dependencies. This flag can be used
#                       multiple times.
#
#   -var 'foo=bar'      Set a variable in the Terraform configuration. This
#                       flag can be set multiple times.
#
#   -var-file=foo       Set variables in the Terraform configuration from
#                       a file. If "terraform.tfvars" or any ".auto.tfvars"
#                       files are present, they will be automatically loaded.

The `terraform init`

The terraform init initializes a new or existing Terraform working directory by creating initial files, loading any remote state, downloading required provider plugins (such as aws), modules, etc. It sets up all the local data necessary to run Terraform that is typically not committed to version control. By default this command will not upgrade an already-installed module; use the -upgrade option to additionally upgrade to the newest available version.

This command is always safe to run multiple times. Though subsequent runs may give errors, this command will never delete your configuration or state.

The output of this command specifies which version of the plugin was installed, and suggests specifying that version in configuration to ensure that running terraform init in the future will install a compatible version.

As you change Terraform configurations, Terraform builds an execution plan that only modifies what is necessary to reach your desired state. By using Terraform to change infrastructure, you can version control not only your configurations but also your state so you can see how the infrastructure evolved over time.

When terraform init is run without provider version constraints, it prints a suggested version constraint string for each provider. To constrain the provider version as suggested, add a required_providers block inside a terraform block.

terraform {
  backend "local" {
  }
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "3.0"
    }
    google = {
      source = "hashicorp/google"
      version = "3.50.0"
    }
  }
}

Provider version constraints can also be specified using a version argument within a provider block, but that simultaneously declares a new provider configuration that may cause problems particularly when writing shared modules.

In the configuration above, the google provider’s source is defined as "hashicorp/google" which is shorthand for "registry.terraform.io/hashicorp/google".

The `terraform plan`

The terraform plan command is used to create an execution plan (by looking at the configuration files, loading variables from *.tfvars, etc). Terraform performs a refresh (such as, the state of data resources), unless explicitly disabled, and then determines what actions are necessary to achieve the desired state specified in the configuration files.

Note: The refreshed state will be used to calculate the plan, but will not be persisted to local or remote state storage.

This command is a convenient way to check whether the execution plan for a set of changes matches your expectations without making any changes to real resources or to the state.

The optional -out argument can be used to save the generated plan to a file for later execution with terraform apply, which can be useful when running Terraform in automation.

Interesting Facts:

The terraform plan shows you what changes will be “inplace” or would need to recreate the object(s).
If an object created by Terraform (such as BigQuery Table) is updated manually (such as a label is added/removed, or a column is added/removed), running terraform plan again would show those manually done changes.
To force recreating an object (such as BiqQuery view using SELECT * whose base table has been altered, and now the view can be queried for correct data but just its schema is stale on UI), we can mark this view resource as terraform taint, and then run terrform plan.

Using -var flag, you can set a variable in Terraform configuration, or you can use -var-file flag to set variables in Terraform configuration from a file.

Using -state flag, you can provide the path to read and save state. It defaults to terraform.tfstate.

Using -refresh=false flag, you can disable refresh prior to checking the difference. This is helpful in cases when you have lots of resources under current terraform plan and refresh before everytime you execute the plan is expensive.

Using -parallelism=n you can increase parallelism (number of concurrent operations) from default value of 10. Note that there is a request cap by Terraform as well as providers (such as GCP), so increasing parallelism beyond a certain value might actually not help.

The flag -destroy if set, a plan will be generated to destroy all resources managed by the given configuration and state.

The flag -lock-timeout if set (by default, it is 0s), specifies duration to retry (that is, it will keep on retrying within that duration) a state lock. Otherwise, if this flag is not set, the command fails immediately if not able to get the lock.

Debugging Terraform:

Terraform has detailed logs which can be enabled by setting TF_LOG environment variable to log levels TRACE, DEBUG, INFO, WARN or ERROR.

The `terraform refresh`

The terraform refresh updates the state file of your infrastructure with metadata that matches the physical resources they are tracking.

This will not modify your infrastructure, but it can modify your state file to update metadata. This metadata might cause new changes to occur when you generate a plan or call apply next.

Similar to terraform plan, it accepts -var, -var-file, and -state flags.

The `terraform apply`

The terraform apply command is used to apply the changes required to reach the desired state of the configuration, or the pre-determined set of actions (terraform apply <terraform_plan_output_file>) generated by a terraform plan execution plan.

There is a lot of information that is known only after terraform apply (such as IP address of EC2 instance that is being created). The terraform apply saves that information to terraform.tfstate file by default.

Using -auto-approve you can skip interactive approval of plan before applying.

Similar to terraform plan, it accepts -var, -var-file, and -state flags.

Note: Be aware that terraform apply is not ACID compliant, so if it is possible that a part or changes go through and then terraform apply encounters some issue and so skips the rest of the changes.

The `terraform destroy`

It destroys Terraform-managed infrastructure. You can rather use a combination of terraform plan -destory and terraform apply.

The `terraform import`

What is Terraform Import?

Terraform is able to import existing infrastructure. This allows you to take resources you’ve created by some other means and bring them under Terraform management.

Warning: Terraform expects that each remote object it is managing will be bound to only one resource address, which is normally guaranteed by Terraform itself having created all objects. If you import existing objects into Terraform, be careful to import each remote object to only one Terraform resource address.

Currently State Only:

The current implementation of Terraform import can only import resources into the state. It does not generate configuration. A future version of Terraform will also generate configuration.

Because of this, prior to running terraform import it is necessary to write manually a resource configuration block for the resource, to which the imported object will be mapped.

The terraform import command:

The terraform import command is used to import existing resources into Terraform.

terraform import [options] ADDRESS ID

For example,

terraform import \
  module.create_edw-dev__CHI_analyst_general__gem_db_view.google_bigquery_table.view_obj \
  projects/edw-dev/datasets/CHI_analyst_general/tables/gem_db_view

Here, module.create_edw-dev__CHI_analyst_general__gem_db_view.google_bigquery_table.view_obj is the ADDRESS available in the directory from where the above command is executed, and projects/edw-dev/datasets/CHI_analyst_general/tables/gem_db_view is the resource ID (whose pattern you can confirm by looking at terraform.tfstate after creating a resource of the same type via Terraform).

Import will find the existing resource from ID and import it into your Terraform state at given ADDRESS.

The ADDRESS must be a valid resource address.

The ID is dependent on the resource type being imported. Reference the provider documentation for details on the ID format.

The `terraform state`

Getting Help:

# getting help
terraform show -help
terraform state -help

Getting State Info:

# show current state or a saved plan
terraform show# list resources in the state
terraform state list# get state details of a resource
# pick RESOURCE_ADDRESS from `terraform state list` output
terraform state show RESOURCE_ADDRESS

Altering State:

# pull the state from its location and output it to stdout
terraform state pull# update remote state from a local state file
terraform state push# remove instances from state
terraform state rm [options] ADDRESS...# move state
terraform state mv [options] SOURCE DESTINATION# replace provider in state
terraform state replace-provider [options] FROM_PROVIDER_FQN TO_PROVIDER_FQN

Force unlock remote terraform lock:

cd <projet_dir>
terraform init -lock=false
terraform force-unlock <lock_id>

Terraform Commands Workflow

Here, is the general workflow:

# update format all Terraform configuration files to a canonical format
terraform fmt# initialize terraform by installing required providers, setting backend
terraform init# update the state file of your infrastructure with metadata that matches the physical resources they are tracking
terraform refresh# it loads the current state and brings up interactive console for experimenting with Terraform interpolations
terraform console# generate an execution plan
terraform plan# build/change infrastructure according to configuration
terraform apply

Here, is an example of using the console:

# main.tf
terraform {
  backend "local" {
  }
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "3.50.0"
    }
  }
}provider "google" {
  credentials = file("path_to_sa_json_file")
  project     = "edw-dev"
}data "google_service_account" "bqowner" {
  account_id = "service_account_name"
}data "google_project" "project" {
}data "google_compute_regions" "available" {
}data "google_compute_zones" "available" {
}data "google_container_registry_repository" "container_registry" {
}data "google_project" "project" {
}resource "google_bigquery_dataset" "dummy_dataset" {
  dataset_id                  = "dummy_dataset"
  friendly_name               = "dm"
  description                 = "This is a temporarily test dataset"
  location                    = "US"
  default_table_expiration_ms = 3600000
  labels = {
    env = "dev"
    created_by = "terraform"
  }
  access {
    role          = "OWNER"
    user_by_email = data.google_service_account.bqowner.email
  }
}terraform init
terraform refresh
terraform show
terraform console> data.google_project.project
> data.google_project.project.id
> data.google_compute_regions.available.names
> "Hello ${data.google_project.project.name}"
> google_bigquery_dataset.dummy_dataset

Terraform Configuration

Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build the described infrastructure. As the configuration changes, Terraform is able to determine what changed and create an incremental execution plan which can be applied .

The infrastructure Terraform can manage includes low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc.

The core Terraform workflow has three steps:

Write: Author infrastructure as code
Plan: Preview changes before applying
Apply: Provision reproducible infrastructure

The set of files used to describe the infrastructure in Terraform is simply known as a Terraform configuration which is written in a Terraform language. The main purpose of this language is to declare resources. All other language features exist only to make the definition of resources more flexible and convenient.

Configuration files can also be JSON, but it is recommended to only using JSON when the configuration is generated by a machine.

Code in Terraform language is stored in plain text files with .tf file extension. There is also a JSON-based variant of language that is named with .tf.json file extension.

The following example describes a simple network topology for Amazon Web Services, just to give a sense of the overall structure and syntax of the Terraform language.

# terraform configuration block to configure terraform behavior
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 1.0.4"
    }
  }
}variable "aws_region" {}variable "base_cidr_block" {
  description = "A /16 CIDR range definition, such as 10.1.0.0/16, that the VPC will use"
  default = "10.1.0.0/16"
}variable "availability_zones" {
  description = "A list of availability zones in which to create subnets"
  type = list(string)
}provider "aws" {
  region = var.aws_region
}resource "aws_vpc" "main" {
  # Referencing the base_cidr_block variable allows the network address
  # to be changed without modifying the configuration.
  cidr_block = var.base_cidr_block
}resource "aws_subnet" "az" {
  # Create one subnet for each given availability zone.
  count = length(var.availability_zones)  # For each subnet, use one of the specified availability zones.
  availability_zone = var.availability_zones[count.index]  # By referencing the aws_vpc.main object, Terraform knows that the subnet
  # must be created only after the VPC is created.
  vpc_id = aws_vpc.main.id  # Built-in functions and operators can be used for simple transformations of
  # values, such as computing a subnet address. Here we create a /20 prefix for
  # each subnet, using consecutive addresses for each availability zone,
  # such as 10.1.16.0/20 .
  cidr_block = cidrsubnet(aws_vpc.main.cidr_block, 4, count.index+1)
}

Terraform Settings

The special terraform configuration block type is used to configure some behaviors of Terraform itself, such as requiring a minimum Terraform version to apply your configuration.

# main.tf
terraform {
  required_version = <VERSION_STRING>,
  required_providers {
    # specify providers
  },
  backend "s3" {
    # (backend-specific settings...)
  }
}

The required_version setting can be used to constrain which version of the Terraform CLI can be used with your configuration. When you use a child module, each module can specify its own version requirements. The requirements of all modules in the tree must be satisfied.

The required_providers setting is a map specifying a version constraint for each provider required by your configuration.

A backend configuration (where the state is stored) is given in a nested backend block within a terraform block. A backend in Terraform determines how the state is loaded and how an operation such as plan or apply are executed. This abstraction enables non-local file state storage, remote execution, etc. By default, Terraform uses the "local" backend.

TF Modules

What is a TF Module

A module is a collection of .tf and/or .tf.json files kept together in a directory. It is a container for multiple resources that are used together. A resource describes a single (or more) infrastructure object(s), while a module might describe a set of objects and the necessary relationships between them in order to create a higher-level system.

Every Terraform configuration has at least one module, known as its root module, which consists of the resources defined in the .tf files in the main working directory.

A Terraform module only consists of top-level configuration files in a directory; nested directories are treated as completely separate modules, and are not automatically included in the configuration.

Terraform evaluates all of the configuration files in a module, effectively treating the entire module as a single document. Separating various blocks into different files is purely for the convenience of readers and maintainers, and has no effect on the module’s behavior.

You might like to follow official guides on Standard Module Structure.

Note: It is always best to use available modules (such as terraform-google-modules/bigquery for Google BigQuery) rather than creating custom ones if available modules can serve your purpose, as they are maintained by the module provider and you can check their source code as well (such as terraform-google-modules/terraform-google-bigquery).

File Overrides

Terraform normally loads all of the .tf and .tf.json files within a directory and expects each one to define a distinct set of configuration objects. If two files attempt to define the same object, Terraform returns an error.

In some rare cases, it is convenient to be able to override specific portions of an existing configuration object in a separate file. For example, a human-edited configuration file in the Terraform language native syntax could be partially overridden using a programmatically generated file in JSON syntax.

For these rare situations, Terraform has special handling of any configuration file whose name ends in _override.tf or _override.tf.json. This special handling also applies to a file named literally override.tf or override.tf.json.

Terraform initially skips these override files when loading configuration, and then afterward processes each one in turn (in lexicographical order). For each top-level block defined in an override file, Terraform attempts to find an already-defined object corresponding to that block and then merges the override block contents into the existing object.

The merging behavior is slightly different for each block type, and some special constructs within certain blocks are merged in a special way.

TF Child Modules and their Meta-Arguments

Modules are called from within other modules using module blocks. To call a module means to instantiate the module, providing values for its input variables, and saving the instance to a local name. For example,

# File: publish_bucket/bucket-and-cloudfront.tfvariable "name" {}                          # this is the input parameter of the current moduleresource "aws_s3_bucket" "example" {
  # ...
}resource "aws_iam_user" "deploy_user" {
  # ...
}# File: my_buckets.tfmodule "servers" {                          # instantiating module `./app-cluster` with local name `servers`
  source = "./app-cluster"  servers = 5
}module "consul" {
  source  = "hashicorp/consul/aws"
  version = "0.0.5"  servers = 3
}module "assets_bucket" {
  source = "./publish_bucket"               # module `./publish_bucket` instantiated with local name `assets_bucket`
  name   = "assets"
}module "media_bucket" {
  source = "./publish_bucket"               # module `./publish_bucket` instantiated again
  name   = "media"
}

The label immediately after the module keyword is a local name, which the calling module can use to refer to this instance of the module.

Within the block body (that is, in between { and }) are the (input) arguments for the module. Most of the arguments correspond to input variables defined by the module. Terraform also defines a few meta-arguments that are reserved by Terraform and used for its own purpose.

The module.<MODULE NAME> is a value representing the result of a module block. If the corresponding module block does not have either count nor for_each set then the value will be an object with one attribute for each output value defined in the child module. To access one of the module's output values, use module.<MODULE NAME>.<OUTPUT NAME>. If the corresponding module uses for_each then the value will be a map of objects whose keys correspond with the keys in the for_each expression, and whose values are each object with one attribute for each output value defined in the child module, each representing one module instance. If the corresponding module uses count then the result is similar to for for_each except that the value is a list with the requested number of elements, each one representing one module instance.

All modules require a source argument, which is a meta-argument defined by Terraform CLI, whose value is either the path to a local directory of the module's configuration files or a remote module source that Terraform should download and use. The same source address can be specified in multiple module blocks to create multiple copies of the resources defined within, possibly with different variable values.

When using modules installed from a module registry, it is recommended to explicitly constraining the acceptable version numbers, using version argument, to avoid unexpected or unwanted changes.

In addition to modules from the local filesystem, Terraform can load modules from a public or private registry (such as Terraform Registry). This makes it possible to publish modules for others to use and to use modules that others have published.

Along with source and version, Terraform defines a few more optional meta-arguments that have special meaning across all modules:

Note: You also might like to look at TF Resource meta-arguments (for explanations).

count: Creates multiple instances of a module from a single module block.
for_each: Creates multiple instances of a module from a single module block.
providers: Passes provider configurations to a child module.
depends_on: Creates explicit dependencies between the entire module and the listed targets.

TF Module’s Blocks, Arguments, and Expressions

Here is the general form of a block:

<BLOCK TYPE> "<BLOCK LABEL>" "<BLOCK LABEL>" {
  # Block body
  <IDENTIFIER> = <EXPRESSION> # Argument
}

For example,

resource "aws_vpc" "main" {
  cidr_block = var.base_cidr_block
}

Blocks usually represent the configuration of some kind of object, like a resource. Blocks have a block type, can have zero or more labels, and have a body that contains any number of arguments and nested blocks.

Arguments appear within the block, they are assigned with some values.

Expressions represent a value, either literally or by referencing and combining other values (that is computed value). They appear as values of arguments, or within other expressions.

You can experiment with the behavior of Terraform’s expressions from the Terraform expression console, by running the terraform console command.

The Terraform language uses the following types of values:

Value TypeExamplesstring"hello there"number15, 6.28booltrue, falselist["a", 5.123, true]]map{name = "Mabel", age = 52}nullnull

Terraform automatically converts numbers and bool values to strings when needed. It also converts strings to numbers or bools, as long as the string contains a valid representation of a number or bool value.

Elements of list/tuple and map/object values can be accessed using the square-bracket index notation, like local.list[3]. The expression within the brackets must be a whole number for list and tuple values or a string for map and object values.

Map/object attributes with names that are valid identifiers can also be accessed using the dot-separated attribute notation, like local.object.attrname. In cases where a map might contain arbitrary user-specified keys, we recommend using only the square-bracket index notation (local.map["keyname"]).

TF Module’s Input Variables

Input variables serve as parameters for a Terraform module, allowing aspects of the module to be customized without altering the module’s own source code, and allowing modules to be shared between different configurations.

When you declare variables in the root module of your configuration, you can set their values using CLI options and environment variables. When you declare them in child modules, the calling module should pass values in the module block.

Note: For brevity, input variables are often referred to as just “variables” or “Terraform variables”.

Each input variable accepted by a module must be declared using a variable block:

variable "image_id" {
  type = string
}variable "availability_zone_names" {
  type    = list(string)
  default = ["us-west-1a"]
}variable "docker_ports" {
  type = list(object({
    internal = number
    external = number
    protocol = string
  }))
  default = [
    {
      internal = 8300
      external = 8300
      protocol = "tcp"
    }
  ]
}

The label after the variable keyword is a name for the variable, which must be unique among all variables in the same module. The name is used to assign a value to the variable from outside and to reference the variable's value from within the module.

The variable declaration can optionally include a type argument to specify what value types are accepted for the variable. The type argument in a variable block allows you to restrict the type of value that will be accepted as the value for a variable. If no type constraint is set then a value of any type is accepted.

The variable declaration can also optionally include a default argument. If present, the variable is considered to be optional and the default value will be used if no value is set when calling the module or running Terraform. The default argument requires a literal value and cannot reference other objects in the configuration.

In addition to type constraint, a module author can specify arbitrary custom validation rules for a particular variable using a validation block nested within the corresponding variable block.

Setting a variable as sensitive prevents Terraform from showing its value in the plan or apply output, when that variable is used within a configuration.

Within the module that declared a variable, its value can be accessed from within expressions as var.<NAME>, where <NAME> matches the label given in the declaration block. For example,

resource "aws_instance" "example" {
  instance_type = "t2.micro"
  ami           = var.image_id
}

The variable assigned to a variable can be accessed only from expressions within the module where it was declared.

If you are using Terraform 0.12 or later, you can assign the special value null to an argument to mark it as "unset".

variable "env" {
  type    = "string"
  default = null
}

Sharing variables/providers across modules:

Refer:

Variables on the Command Line:

To specify individual variables on the command line, use the -var option when running the terraform plan and terraform apply commands. For example,

terraform apply -var="image_id=ami-abc123"
terraform apply -var='image_id_list=["ami-abc123","ami-def456"]'
terraform apply -var='image_id_map={"us-east-1":"ami-abc123","us-east-2":"ami-def456"}'

The -var option can be used any number of times in a single command.

Variable Definitions (.tfvars) Files:

To set lots of variables, it is more convenient to specify their values in a variable definition file (with a filename ending in either .tfvars or .tfvars.json) and then specify the file on the command line with -var-file option. For example,

terraform apply -var-file="testing.tfvars"

A variable definition file uses the same basic syntax as Terraform language files but consists only of variables name assignments.

image_id = "ami-abc123"
availability_zone_names = [
  "us-east-1a",
  "us-west-1c",
]

Environment Variables:

As a fallback for the other ways of defining variables, Terraform searches the environment of its own process for environment variables named TF_VAR_ followed by the name of a declared variable. For example,

export TF_VAR_image_id=ami-abc123
terraform plan

TF Module’s Output Variables

Output values are like the multiple return values of a Terraform module (similar to public members of Python’s module).

They have several uses:

A child module can use outputs to expose a subset of its resource attributes to a parent module.
A root module can use outputs to print certain values in the CLI output after running terraform apply.
When using remote state, root modules outputs can be accessed by other configurations via a terraform_remote_state data source.

Declaring an Output Type:

Each output value exported by a module must be declared using an output block:

output "instance_ip_addr" {
  value = aws_instance.server.private_ip
}

The label immediately after the output keyword is the name, which must be a valid identifier.

The value argument takes an expression whose result is to be returned to the user.

The output blocks can optionally include description, sensitivity, and depends_on arguments.

Accessing Child Module’s Outputs:

In a parent module, outputs of child modules are available in expressions as module.<MODULE-NAME>.<OUTPUT-NAME> (for example, module.web_server.instance_ip_addr).

The resources defined in a module are encapsulated, so the calling module cannot access their attributes directly. However, the child module can declare output values to selectively export certain values to be accessed by the calling module.

Optional Arguments:

The output blocks can optionally include description, sensitive, and depends_on arguments.

TF Module’s Local Variables

A local value assigns a name to an expression, so you can use it multiple times within a module without repeating it.

Declaring Local Values:

A set of related local values can be declared together in a single locals block. For example,

locals {
  service_name = "forum"
  owner        = "Community Team"
}

The expressions in local values are not limited to literal constants; they can also reference other values in the module in order to transform or combine them, including variables, resource attributes, or other local values.

Using Local Values:

Once a local value is declared, you can reference it in expressions as local.<NAME> (for example, local.service_name).

A local value can only be accessed in expressions within the module where it was declared.

Module Versions in TF

It is recommended to explicitly constraining the acceptable version numbers for each external module to avoid unexpected changes.

Use the version attribute in the module block to specify versions.

Version constraints are supported only for modules installed from a module registry, such as the Public Terraform Registry for Modules (for example, terraform-google-bigquery)or Terraform Cloud's Private Module Registry.

Note: After adding, removing, or modifying module blocks, you must re-run terraform init to allow Terraform the opportunity to adjust the installed modules.

TF Expressions

Expressions are used to or compute values within a configuration.

A gist:

Use terraform console to test expressions.
TF’s no type is null
TF’s value types are: string (for example, "hello"), number (for example, 15, 3.14), bool (that is, true and false), list (for example, ["us-west-1a", 52, true, false, null]), map (for example, {name = "Mabel", age = 52})
TF string is a series of characters delimited by double-quote character (")
TF list can be indexed using square-bracket notation (for example, local.names[0])
Where possible, Terraform automatically converts values from one type to another in order to produce the expected type. If this isn’t possible, Terraform will produce a type mismatch error and you must update the configuration with a more suitable expression.
Within quoted and heredoc string expressions, the sequences ${ and %{ begin template sequences.
A ${ ... } sequence is an interpolation, which evaluates the expression given between the markers, converts the result to a string if necessary, and then inserts it into the final string (for example, "Hello, ${var.name}!")
A %{ ... } sequence is a directive, which allows for conditional results and iteration over collections, similar to conditional and for expressions. For example,
"Hello, %{ if var.name != "" }${var.name}%{ else }unnamed%{ endif }!"
and
<<EOT %{ for ip in aws_instance.example.*.private_ip ~} server ${ip} %{ endfor ~} EOT
Operators: ==, !=, >, <, >=, <=, &&, ||, !, +, -, *, /
Conditional expression: condition ? true_val : false_val
The for expression: [for s in var.names : upper(s)], {for s in var.names : s => upper(s)}, [for s in var.names : upper(s) if s != ""]

Built-in Functions in TF

The Terraform language includes a number of built-in functions that you can call from within expressions to transform and combine values.

The Terraform language does not support user-defined functions, and so only the functions built into the language are available to use.

The Providers in TF

What is a TF Provider?

A provider usually provides resources to manage a single cloud or on-premises infrastructure platform (for example, AWS, GCP, Docker, Kubernetes, Consul, Datadog). Providers are distributed separately from Terraform itself, but Terraform can automatically install most providers when initializing a working directory.

In order to manage resources, a Terraform module must specify which providers it requires. Additionally, most providers need some configuration in order to access their remote APIs, and the root module must provide that configuration.

The behaviors of resources rely on their associated resource types, and these types are defined by providers.

The provider block is used to configure the named provider (such as "aws"). A provider is responsible for creating and managing resources. A provider is a plugin that Terraform uses to translate the API interactions with the service.

For example,

provider "google" {                    # name of the provider being configured
  project = "acme-app"
  region  = "us-central1"
}

TF Provider Requirement

Terraform relies on plugins called “providers” to interact with remote systems. Terraform configurations must declare which providers they require, so that Terraform can install and use them.

A provider requirement consists of a local name, a source location, and a version constraint:

terraform {
  required_providers {
    mycloud = {
      source  = "mycorp/mycloud"
      version = "~> 1.0"
    }
  }
}provider "mycloud" {
  # ...
}

The required_providers block must be nested inside the top-level terraform block (which can also contain other settings).

Each argument in the required_providers block enables one provider. The key determines the provider's local name (used to reference within Terraform module), and the value is an object with the following elements:

source: the global source address for the provider you intend to use, such as hashicorp/aws.
version: a version constraint specifying which subset of available provider versions the module is compatible with.

Each module should at least declare the minimum provider version it is known to work with, using the >= version constraint syntax.

Outside of the required_providers block, Terraform configurations always refer to providers by their local names (not its global source address).

Built-in Providers:

While most Terraform providers are distributed separately as plugins, there is currently one provider that is built-in to Terraform itself, which provides the terraform_remote_state data source.

Because this provider is built into Terraform, you don’t need to declare it in the required_providers block in order to use its features. However, for consistency, it does have a special provider source address, which is terraform.io/builtin/terraform.

TF Provider Configuration

Provider configuration belongs in the root module of a Terraform configuration. A provider configuration is created using a provider block.

# The default provider configuration
provider "aws" {
  region = "us-east-1"
}# Additional provider configuration for west coast region
provider "aws" {
  alias  = "west"
  region = "us-west-2"
}

The name given in the block header ("aws" in this example) is the local name of the provider to configure. This provider should already be included in a required_providers block.

The body of the provider block (that is, between { and }) contains configuration arguments for the provider itself.

Resource providers (such as AWS, GCP, Github, HTTP, Kubernetes, MySQL, MongoDB, Cloudflare, Datadog, Okta, etc) are fully documented within providers reference.

The profile attribute in "aws" provider block refers to the AWS config file in ~/.aws/credentials.

Omitting provider block:

Unlike many other objects in the Terraform language, a provider block may be omitted if its contents would otherwise be empty. Terraform assumes an empty default configuration for any provider that is not explicitly configured.

The alias option:

Although, arguments are specific to the provider being configured, there is one “meta-argument” that are defined by Terraform itself and available for all provider blocks: alias which is used for using the same provider with different configurations for different resources.

You can optionally define multiple configurations for the same provider, and select which one to use on a per-resource or per-module basis.

The provider block without alias set is known as the default provider configuration.

When Terraform needs the name of the provider configuration, it always expects a reference of the form <PROVIDER_NAME>.<ALIAS>. In the above example, aws.west would refer to the provider with the us-west-2 region.

Multiple provider blocks can exist if a Terraform configuration is composed of multiple providers, which is a common situation.

When alias is set, it creates an additional provider configuration. For providers that have no required configuration arguments, the implied empty configuration is considered to be the default provider configuration.

Selecting Alternate TF Providers

By default, resources use a default provider configuration inferred from the first word of the resource type name. For example, a resource of type aws_instance uses the default (un-aliased) aws provider configuration unless otherwise stated.

To select an aliased provider for a resource or data source, set its provider meta-argument to a <PROVIDER-NAME>.<ALIAS> reference. For example,

resource "aws_instance" "foo" {
  provider = aws.west  # ...
}

To select aliased providers for a child module, use its providers meta-argument to specify which aliased providers should be mapped to which provider names inside the module. For example,

module "aws_vpc" {
  source = "./aws_vpc"
  providers = {
    aws = aws.west
  }
}

Modules have some special requirements when passing in providers; see Providers within Modules for more details. In most cases, only root modules should define provider configurations, with all child modules obtaining their provider configuration from their parents.

Dependency Lock File

A dependency lock file is a file that belongs to the configuration as a whole, rather than to each separate module in the configuration. For that reason, Terraform creates it and expects to find it in your current working directory when you run Terraform, which is also the directory containing the .tf files for the root module of your configuration.

The lock file is always named .terraform.lock.hcl, and this name is intended to signify that it is a lock file for various items that Terraform caches in the .terraform subdirectory of your working directory.

Terraform automatically creates or updates the dependency lock file each time you run the terraform init command. You should include this file in your version control repository.

The Resources in TF

What is a TF Resource?

The resource block defines a resource of given type and name representing one or more infrastructure objects. A resource might be a physical component such as an EC2 instance, or it can be a logical resource such as a Heroku application.

For example,

resource "aws_instance" "web" {
  ami           = "ami-a1b2c3d4"
  instance_type = "t2.micro"
}

The resource block has two strings before opening the block: the resource type and the resource name. The name is used to refer to this resource from elsewhere in the same Terraform module, but has no significance outside of the scope of a module. Each resource type in turn belongs to a provider. The resource type and a name together serve as an identifier for a given resource and so must be unique within a module.

Within the resource block (between { and }) resides the configuration for that resource. This is dependent on each resource provider. There are also some meta-arguments that are defined by Terraform itself and apply across all resource types.

The <RESOURCE_TYPE>.<NAME> represents a managed resource of a given type and name.

TF Resource and its state

When Terraform creates a new infrastructure object represented by a resource block, the identifier for that real object is saved in Terraform's state, allowing it to be updated and destroyed in response to future changes. For resource blocks that already have an associated infrastructure object in the state, Terraform compares the actual configuration of the object with the arguments given in the configuration and, if necessary updates the object to match the configuration.

The State:

Terraform requires some sort of database to map Terraform config to the real world. When you have a resource resource "aws_instance" "foo" in your configuration, Terraform uses this map to know that instance i-abcd1234 is represented by that resource. For some providers like "aws", Terraform could theoretically use something like AWS tags, but not all resources support tags, and not all cloud providers support tags. Therefore, for mapping configuration to resources in the real world, Terraform uses its own state structure.

Alongside the mappings between resource and remote objects, Terraform must also track metadata such as resource dependencies. Terraform typically uses the configuration to determine dependency order.

In addition to basic mapping, Terraform stores a cache of the attribute values for all resources in the state. This is the most optional feature of Terraform state and is done as a performance improvement.

When running terraform plan, Terraform must know the current state of resources in order to effectively determine the changes that it needs to make to reach your desired configuration. For small infrastructures, Terraform can query your providers and sync the latest attributes from all your resources. But for large infrastructures, querying every resource is too slow. Larger users of Terraform make heavy use of -refresh=false flag as well as -target flag in order to work around this. In these scenarios, the cached state is treated as the record of truth.

Remote State:

The state is stored by default in a local file named terraform.tfstate (in JSON format, but do not modify it manually), but it can also be stored remotely, which works better in a team environment. The state file should not be stored in source control.

By default, Terraform uses the “local” backend.

With remote state, Terraform writes the state data to a remote data store, which can then be shared between all members of a team. Terraform supports storing state in Terraform Cloud, HashiCorp Consul, Amazon S3, etc.

Remote state gives you more than just easier version control and safer storage. It also allows you to delegate the outputs to other teams. This allows your infrastructure to be more easily broken down into components that multiple teams can access.

Remote state is a feature of backends. Configuring and using remote backends is easy and you can get started with remote state quickly.

For fully-featured remote backends, Terraform can also use state locking to prevent concurrent runs of Terraform against the same state.

Remote state allows you to share output values with other configurations. This allows your infrastructure to be decomposed into smaller components. For example, check terraform_remote_state data source.

Backend Configuration:

Whenever a configuration’s backend changes, you must run terraform init again to validate and configure the backend before you can perform any plans, applies, or state operations. When changing backends, Terraform will give you the option to migrate your state to the new backend. This lets you adopt backends without losing any existing state. To be extra careful, it is always recommended to manually backing up your state as well. You can do this by simply copying your terraform.tfstate file to another location. The initialization process should create a backup as well, but it never hurts to be safe!

You do not need to specify every required argument in the backend configuration. Omitting certain arguments may be desirable if some arguments are provided automatically by an automation script running Terraform. When some or all of the arguments are omitted, we call this a partial configuration.

With a partial configuration, the remaining configuration arguments must be provided as part of the initialization process. There are several ways to supply the remaining arguments:

File: A configuration file may be specified via the init command line. To specify a file use the -backend-config=PATH option when running terraform init. If the file contains secrets it may be kept in a secure data store, such as Vault, in which it must be downloaded to a local disk before running Terraform.
Command-line key-value pairs: Key-value pairs can be specified via the init command line. Note that many shells retain command-line flags in a history file, so this isn't recommended for secrets. To specify a single key-value pair one at a time, use the -backend-config="KEY=VALUE" option when running terraform init.
Interactively: Terraform will interactively ask you for the required values unless interactive input is disabled. Terraform will not prompt for optional values.

If backend settings are provided in multiple locations, the top-level settings are merged such that any command-line options override the settings in the main configuration, and then the command-line options are processed in order, with later options overriding values set by earlier options. The final, merged configuration is stored on disk in the .terraform directory, which should be ignored from version control.

When using a partial configuration, Terraform requires at a minimum that an empty backend configuration is specified in one of the root Terraform configuration files, to specify the backend type. For example,

terraform {
  backend "consul" {}
}

A backend configuration file has the contents of the backend block as top-level attributes, without the need to wrap it in another terraform or backend block:

address = "demo.consul.io"
path    = "example_app/terraform_state"
scheme  = "https"

The same settings can alternatively be specified on the command line as follows:

$ terraform init \
    -backend-config="address=demo.consul.io" \
    -backend-config="path=example_app/terraform_state" \
    -backend-config="scheme=https"

The Consul backend also requires a Consul access token. Per the recommendation above of omitting credentials from the configuration and using other mechanisms, the Consul token would be provided by setting either the CONSUL_HTTP_TOKEN or CONSUL_HTTP_AUTH environment variables.

Manual State Pull/Push:

You can manually retrieve the state from the remote state using the terraform state pull command. This will load your remote state and output it to stdout.

You can manually write state with terraform state push. This is extremely dangerous and should be avoided if possible. This will overwrite the remote state. This can be used to do manual fixups if necessary.

State Locking:

If supported by your backend, Terraform will lock your state for all operations that could write state. This prevents others from acquiring the lock and potentially corrupting your state.

State locking happens automatically on all operations that could write state.

TF Resource Dependencies

Some resources must be processed after other specific resources; sometimes this is because of how the resource works, and sometimes the resource’s configuration just requires information generated by another resource.

Most resource dependencies are handled automatically. Terraform analyses any expressions within a resource block to find references to other objects, and treats those references as implicit ordering requirements when creating, updating, or destroying resources

However, some dependencies cannot be recognized implicitly in configuration. For example, if Terraform must manage access control policies and take actions that require those policies to be present, there is a hidden dependency between the access policy and a resource whose creation depends on it. In these rate cases, the depends_on meta-argument can explicitly specify a dependency.

TF Resource Meta-Arguments

Terraform CLI defines the following meta-arguments, which can be used with any resource type to change the behavior of resources:

depends_on: Used for specifying hidden dependencies (the one which Terraform cannot know by itself, such as the dependency of a table within a Bigquery view query) that Terraform can't automatically infer. Explicitly specifying a dependency is only necessary when a resource or module relies on some other resource's behavior but doesn't access any of that resource's data in its arguments.
count: Used for creating multiple resource instances according to a count (its value can be 0, in which case the resource will not be created).
for_each: Used to create multiple instances according to a map, or set of things.
provider: Used for selecting a non-default provider configuration.
lifecycle: Used for lifecycle customizations.
provisioner and connection: Used for taking extra actions after resource creation.

Local-only TF Resources

While most resource types correspond to an infrastructure object type that is managed via a remote network API, there are certain specialized resources types that operate only within Terraform itself (such as generating private keys, issuing self-signed TLS certificates, generating random ids, etc.), calculating some results and saving those results in the state of future use.

The behavior of local-only resources is the same as all other resources, but their result data exists only within the Terraform state. “Destroying” such a resource means only to remove it from the state, discarding its data.

TF Operation Timeouts

Some resource types provide a special timeouts nested block argument that allows you to customize how long certain operations are allowed to take before being considered to have failed. For example,

resource "aws_db_instance" "example" {
  # ...  timeouts {
    create = "60m"
    delete = "2h"
  }
}

Transferring Resource State Into Modules

When refactoring an existing configuration to split code into child modules, moving resource blocks between modules causes Terraform to see the new location as an entirely different resource from the old. Always check the execution plan after moving code across modules to ensure that no resources are deleted by surprise.

If you want to make sure an existing resource is preserved, use the terraform state mv command to inform Terraform that it has moved to a different module.

Authoritative vs Non-Authoritative Resources

If you are attempting to manage roles using the authoritative resource (such as google_project_iam_policy and google_project_iam_binding), then every entity in that project/role must be defined within that block. Every time a user or service account is added to a role, you will have to modify your .tf script and add the entiry as a member in the role, otherwise next time you run terraform apply, it will wipe off any changes done manually or via some other resource call within the same codebase, or via some other tool.

For example, let’s look at GCP IAM Policy for Projects:

google_project_iam_policy: Authoritative for the project. The google_project_iam_policy cannot be used in conjunction with google_project_iam_binding, google_project_iam_member, or google_project_iam_audit_config or they will fight over what your policy should be.
google_project_iam_binding: Authoritative for a given role. The google_project_iam_binding resources can be used in conjunction with google_project_iam_member resources only if they do not grant privilege to the same role.
google_project_iam_member: Non-authoritative.
google_project_iam_audit_config: Authoritative for a given service. Updates the IAM policy to enable audit logging for the given service.

Data Sources in TF

What is a TF Data Source?

Data sources allow data to be fetched (such as fetching data from provider) or computed for use elsewhere in Terraform configuration. The use of data sources allows a Terraform configuration to make use of information defined outside of Terraform, or defined by another separate Terraform configuration.

Each provider may offer data sources alongside its set of resource types.

How a data source is accessed?

A data source is accessed via a special kind of resource known as “data resource”, declared using a data block:

data "aws_ami" "alx" {                      # read from data source "aws_ami" and export the result to local name "alx"
  most_recent = true
  owners = ["self"]  filters {
    name = "state"
    values = ["available"]
  }  tags = {
    Name   = "app-server"
    Tested = "true"
  }
}resource "aws_instance" "ex" {
  ami = "data.aws_ami.alx.id"               # using an attribute of data resource
  instance_type = "t2.micro"
}output "aws_public_ip" {
  value = "aws_instance.ex.public_dns"      # using an attribute of data resource
}

A data block requests that Terraform reads from a given data source (for example, "aws_ami") and export the result under the given local name (for example, "alx"). The name is used to refer to this resource from elsewhere in the same Terraform module, but has no significance outside of the scope of a module. The data source and name together serve as an identifier for a given resource and so must be unique within a module.

Within a block body (between { and }) and query constraints defined by the data source. Most arguments in this section depend on the data source. However, there are some "meta-arguments" that are defined by Terraform itself and apply across all data sources.

Each data instance will export one or more attributes, which can be used in other resources as reference expressions of the form data.<TYPE>.<NAME>.<ATTRIBUTE>. For example,

# Find the latest available AMI that is tagged with Component = web
data "aws_ami" "web" {
  filter {
    name   = "state"
    values = ["available"]
  }  filter {
    name   = "tag:Component"
    values = ["web"]
  }  most_recent = true
}resource "aws_instance" "web" {
  ami           = data.aws_ami.web.id
  instance_type = "t1.micro"
}

Multiple Resource Instance:

Data resources support count and for_each meta-arguments as defined for managed resources, with the same syntax and behavior.

Selecting a non-default provider configuration:

Data resources support the providers meta-argument as defined for managed resources, with the same syntax and behavior.

When Info from Data Resource is Read?

If the query constraint arguments for a data resource refer only to constant values or values that are already known, the data resource will be read and its state updated during Terraform’s refresh phase, which runs prior to creating a plan. This ensures that the retrieved data is available for use during planning and so Terraform’s plan will show the actual values obtained.

Query constraints arguments may refer to values that cannot be determined until after configuration is applied, such as id of a managed resource that has not been created yet. In this case, reading from the data source is deferred until the apply phase, and any references to the results of the data resource elsewhere in configuration will themselves be unknown until after the configuration has been applied.

TF Data Resource Dependencies

Data resources have the same dependency resolution as defined for managed resources. In particular, the depends_on meta-argument is also available within data blocks, with the same meaning and syntax as in resource blocks.

However, due to the data resource behavior of deferring the read until the apply phase when depending on values that are not yet known, using depends_on with data resources will force the read to always be deferred to the apply phase, and therefore a configuration that uses depends_on with a data resource can never converge. Due to this behavior, we do not recommend using depends_on with data resources.

Terraform Pull Request Automation: Atlantis

Atlantis is Terraform’s pull request automation.

Atlantis is deployed as a standalone application into your infrastructure. It listens to Github, GitLab or Bitbucket webhooks about Terraform pull requests. It then runs terraform plan and comments with the output back on the pull request. When you want to apply, comment atlantis apply on the pull request, and Atlantis will run terraform apply and comment back with the output.

An atlantis.yaml file specified at the root of the Terraform repo allows you to instruct Atlantis on the structure of your repo and set custom workflows.

Managing Infra with Terraform

Table of Contents

Terraform References

Getting Started with Terraform

Terraform Installation

Terraform CLI

Terraform Help

The `terraform init`

The `terraform plan`

The `terraform refresh`

The `terraform apply`

The `terraform destroy`

The `terraform import`

The `terraform state`

Terraform Commands Workflow

Terraform Configuration

Terraform Settings

TF Modules

The Providers in TF

The Resources in TF

Data Sources in TF

Terraform Pull Request Automation: Atlantis

Written by Munish Goyal