VMwire

Terraform is a great framework to use to start developing and working with infrastructure-as-code to manage resources. It provides awesome benefits such as extremely fast deployment through automation, managing configuration drift, adding configuration changes and destroying entire environments with a few key strokes. Plus it supports many providers so you can easily use the same code logic to deploy and manage different resources, for example on VMware clouds, AWS or Azure at the same time.

For more information if you haven’t looked at Terraform before, please take a quick run through HashiCorp’s website:

https://www.terraform.io/

Getting started with Terraform is really quite simple when the environment that you are starting to manage is green-field. In that, you are starting from a completely fresh deployment on Day-0. If we take AWS as an example, this is as fresh as signing up to the AWS free-tier with a new account and having nothing deployed in your AWS console.

Terraform has a few simple files that are used to build and manage infrastructure through code, these are the configuration and the state. The basic building blocks of Terraform. There are other files and concepts that could be used such as variables and modules, but I won’t cover these in much detail in this post.

How do you bring in infrastructure that is already deployed into Terraform’s management?

This post will focus on how to import existing infrastructure (brown-field) into Terraform’s management. Some scenarios where this could happen is that you’ve already deployed infrastructure and have only recently started to look into infrastructure as code and maybe you’ve tried to use PowerShell, Ansible and other tools but none are quite as useful as Terraform.

Assumptions

First lets assume that you’ve deployed Terraform CLI or are already using Terraform Cloud, the concepts are pretty much the same. I will be using Terraform CLI for the examples in this post together with AWS. I’m also going to assume that you know how to obtain access and secret keys from your AWS Console.

By all means this import method works with any supported Terraform provider, including all the VMware ones. For this exercise, I will work with AWS.

My AWS environment consists of the following infrastructure, yours will be different of course and I’m using this infrastructure below in the examples.

You will need to obtain the AWS resource IDs from your environment, use the AWS Console or API to obtain this information.

But I used CloudFormation to deploy my infrastructure…

If you used CloudFormation to deploy your infrastructure and you now want to use Terraform, then you will need to update the CloudFormation deletion policy to retain before bringing any resources into Terraform. This is important as any accidental deletion or change with CloudFormation stack would impact your Terraform configuration and state. I recommend setting this policy before importing resources with Terraform.

This link has some more information that will help you enable the deletion policy on all resources.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-deletionpolicy.html

For example to change a CloudFormation configuration with the deletion policy enabled, the code would look like this:

Resources:


    VPC:
      Type: AWS::EC2::VPC
      DeletionPolicy: Retain
      Properties:
        CidrBlock: 10.0.0.0/16
        InstanceTenancy: default
        EnableDnsSupport: 'true'
        EnableDnsHostnames: 'true'

Lets get started!

Set up your main.tf configuration file for a new project that will import an existing AWS infrastructure. The first version of our main.tf file will look like this, with the only resource that we will import being the VPC. Its always good to work with a single resource first to ensure that your import works before going all out and importing all the rest.

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "3.28.0"
    }
  }
}

provider "aws" {
  # Configuration options
  region = "eu-west-1"
  access_key = "my_access_key"
  secret_key = "my_secret_key"
}

resource "aws_vpc" "VPC" {
  # (resource arguments)
}

Run the following to initialize the AWS provider in Terraform.

terraform init

Import the VPC resource with this command in your terminal

terraform import aws_vpc.VPC vpc-02d890cacbdbaaf87

You can then review the terraform state file, it should be named terraform.tfstate, and it will look something like this. (Open it in a text editor).

{
  "version": 4,
  "terraform_version": "0.14.6",
  "serial": 13,
  "lineage": "xxxx",
  "outputs": {},
  "resources": [    {
  "mode": "managed",
      "type": "aws_vpc",
      "name": "VPC",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "schema_version": 1,
          "attributes": {
            "arn": "xxxx",
            "assign_generated_ipv6_cidr_block": false,
            "cidr_block": "10.0.0.0/16",
            "default_network_acl_id": "acl-067e11c10e2327cc9",
            "default_route_table_id": "rtb-0a55b9e1683991242",
            "default_security_group_id": "sg-0db58c5c159b1ebf9",
            "dhcp_options_id": "dopt-7d1b121b",
            "enable_classiclink": false,
            "enable_classiclink_dns_support": false,
            "enable_dns_hostnames": true,
            "enable_dns_support": true,
            "id": "vpc-02d890cacbdbaaf87",
            "instance_tenancy": "default",
            "ipv6_association_id": "",
            "ipv6_cidr_block": "",
            "main_route_table_id": "rtb-0a55b9e1683991242",
            "owner_id": "xxxxxxx",
            "tags": {
              "Name": "VPC",
              "environment": "aws",
              "project": "Imported by Terraform"
            }
          },
          "sensitive_attributes": [],
          "private": "xxxxxx"
        }
      ]
    }
  ]
}

Notice that the VPC and all of the VPC settings have now been imported into Terraform.

Now that we have successfully imported the VPC, we can continue and import the rest of the infrastructure. The remaining AWS services we need to import are detailed in Table 1. AWS Resource IDs.

To import the remaining infrastructure we need to add the code to the main.tf file to import the other resources. Edit your main.tf so that it looks like this. Notice that all of the thirteen resources are defined in the configuration file and the resource arguments are all empty. We will update the resource arguments later, initially we just need to import the resources into the Terraform state and then update the configuration with the known state.

Terraform does not support automatic creation of a configuration out of a state.

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "3.28.0"
    }
  }
}

provider "aws" {
  # Configuration options
  region = "eu-west-1"
  access_key = "my_access_key"
  secret_key = "my_secret_key"
}

resource "aws_vpc" "SAVPC" {
  # (resource arguments)
}

resource "aws_subnet" "PublicSubnetA" {
  # (resource arguments)
}

resource "aws_subnet" "PublicSubnetB" {
  # (resource arguments)
}

resource "aws_subnet" "PrivateSubnetA" {
  # (resource arguments)
}

resource "aws_subnet" "PrivateSubnetB" {
  # (resource arguments)
}

resource "aws_internet_gateway" "IGW" {
  # (resource arguments)
}

resource "aws_network_acl" "NACL" {
  # (resource arguments)
}

resource "aws_route_table" "PublicRoute" {
  # (resource arguments)
}

resource "aws_route_table" "PrivateRoute" {
  # (resource arguments)
}

resource "aws_instance" "Instance1" {
  # (resource arguments)
}

resource "aws_elb" "elb-UE360LJ7779C" {
  # (resource arguments)
}

resource "aws_security_group" "ELBSecurityGroup" {
  # (resource arguments)
}

resource "aws_security_group" "AppServerSecurityGroup" {
  # (resource arguments)
}

Run the following commands in your terminal to import the remaining resources into Terraform.

terraform import aws_subnet.PublicSubnetA subnet-0f6d45ef0748260c6
terraform import aws_subnet.PublicSubnetB subnet-092bf59b48c62b23f
terraform import aws_subnet.PrivateSubnetA subnet-03c31081bf98804e0
terraform import aws_subnet.PrivateSubnetB subnet-05045746ac7362070
terraform import aws_internet_gateway.IGW igw-09056bba88a03f8fb
terraform import aws_network_acl.NACL acl-0def8bcfeff536048
terraform import aws_route_table.PublicRoute rtb-082be686bca733626
terraform import aws_route_table.PrivateRoute rtb-0d7d3b5eacb25a022
terraform import aws_instance.Instance1 i-0bf15fecd31957129
terraform import aws_elb.elb-158WU63HHVD3 elb-158WU63HHVD3
terraform import aws_security_group.ELBSecurityGroup sg-0b8f9ee4e1e2723e7
terraform import aws_security_group.AppServerSecurityGroup sg-031fadbb59460a776

Now that all thirteen resources are imported you will need to manually update the configuration file, in our case main.tf with the resource arguments that correspond to the current state of all the resources that were just imported. The easiest way to do this is to first take a look at the Terraform provider for AWS documentation to find the mandatory fields that are needed. Lets use the aws_subnet as an example:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/subnet

From the documentation we need two things

cidr_block – (Required) The CIDR block for the subnet.

vpc_id – (Required) The VPC ID.

We know that we need these two as a minimum, but what if there are other configuration items that were done in the AWS Console or CloudFormation before you started to work with Terraform. An example of this is of course tags and other configuration parameters. You want to update your main.tf file with the same configuration as what was just imported into the state. This is very important.

To do this, do not use the terraform.tfstate but instead run the following command.

terraform show

You’ll get an output of the current state of your AWS environment that you can then copy and paste the resource arguments into your main.tf configuration.

I won’t cover how to do all thirteen resources in this post so I’ll again use our example for one of the aws_subnet resources. Here is the PublicSubnetA aws_subnet resource information copy and pasted straight out of the terraform show command.

# aws_subnet.PublicSubnetA:
resource "aws_subnet" "PublicSubnetA" {
    arn                             = "arn:aws:ec2:eu-west-1:xxxx:subnet/subnet-0f6d45ef0748260c6"
    assign_ipv6_address_on_creation = false
    availability_zone               = "eu-west-1a"
    availability_zone_id            = "euw1-az2"
    cidr_block                      = "10.0.0.0/24"
    id                              = "subnet-0f6d45ef0748260c6"
    map_customer_owned_ip_on_launch = false
    map_public_ip_on_launch         = true
    owner_id                        = "xxxx"
    tags                            = {
        "Name"        = "PublicSubnetA"
        "environment" = "aws"
        "project"     = "my_project"
    }
    vpc_id                          = "vpc-02d890cacbdbaaf87"

    timeouts {}
}

Not all resource arguments are needed, again review the documentation. Here is an example of my changes to the main.tf file with some of the settings taken from the output of the terraform show command.

resource "aws_subnet" "PublicSubnetA" {
    assign_ipv6_address_on_creation = false
    cidr_block                      = var.cidr_block_PublicSubnetA
	map_public_ip_on_launch         = true
    tags                            = {
        Name        = "PublicSubnetA"
        environment = "aws"
        project     = "my_project"
    }
    vpc_id                          = var.vpc_id

    timeouts {}
}

Notice that I have turned the value for cidr_block and vpc_id into a variables.

Using Variables

Using variables simplifies a lot of your code. I’m not going to explain what these are in this post, you can read up on these with this link:

https://www.terraform.io/docs/language/values/variables.html

However, the contents of my terraform.tfvars file looks like this:

cidr_block = "10.0.0.0/16"
vpc_id = "vpc-02d890cacbdbaaf87"
cidr_block_PublicSubnetA = "10.0.0.0/24"
cidr_block_PublicSubnetB = "10.0.1.0/24"
cidr_block_PrivateSubnetA = "10.0.2.0/24"
cidr_block_PrivateSubnetB = "10.0.3.0/24"
instance_type = "t2.micro"
ami_id = "ami-047bb4163c506cd98"
instance_port = "80"
instance_protocol = "http"
lb_port = "80"
lb_protocol = "http"

Just place your terraform.tfvars file in the same location as your main.tf file. Terraform automatically links to the default or you can reference a different variable file, again refer to the documentation.

Finalizing the configuration

Once you’ve updated your main.tf configuration with all the correct resource arguments, you can test to see if what is in the configuration is the same as what is in the state. To do this run the following command:

terraform plan

If you copied and pasted and updated your main.tf correctly then you would get output from your terminal similar to the following:

terraform plan
[ Removed content to save space ]

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

Congratulations, you’ve successfully imported an infrastructure that was built outside of Terraform.

You can now proceed to manage your infrastructure with Terraform. For example changing the terraform.tfvars parameters for

lb_port = "443"
lb_protocol = "https"

And then running plan and apply will update the elastic load balancer elb-158WU63HHVD3 from health check on port 80 to port 443 instead.

terraform plan
[ removed content to save space ]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_elb.elb-158WU63HHVD3 will be updated in-place
  ~ resource "aws_elb" "elb-158WU63HHVD3" {
      ~ health_check {
          ~ target              = "TCP:80" -> "TCP:443"           
        }
    }

terraform apply
[ content removed to save space] 

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate

And that’s how you import existing resources into Terraform, I hope you find this post useful. Please comment below if you have a better method or have any suggestions for improvements. And feel free to comment below if you have questions and need help.