Updating Terraform Cloud State with existing resources

Let's assume one of two scenarios

1. You have all the infra required for your project created already and now you would like to maintain it using terraform.
2. You have the infra managed by terraform but you had to create just one resource directly in the cloud provider console and now you want to add it to your terraform config.

In either cases, when you add the corresponding terraform script and run terraform plan, the plan output will first show that terraform will create the necessary resource but the apply operation will fails with an error that the required resource already exists.

The output will look something like this if you are using Terraform Cloud to create a s3 bucket which already exists in AWS with the same name.

This happens because terraform maintains a state and there is no entry within the terraform state that the S3 bucket already exists. So, it will always assume that the bucket doesn't exist and will attempt to recreate it. The solution to this problem is to import the resource into terraform state manually using the import statement. For import an S3 bucket resource into terraform state, the command would be this

terraform import aws_s3_bucket.bucket bucket-name

You can find the import statements for all terraform resources on their respective registry pages.

This is pretty straight forward if you have access to terraform store. But what if the terraform workspace is on Terraform Cloud. Then the process involves few more steps. Let's go through them.

Create a local workspace with remote storage

First step is to create a terraform workspace locally and use Remote State feature to point the state storage to Terraform cloud

Add the following snippet to your terraform config files

 terraform {
    backend "remote" {
        organization = "pavan-playground" # org name 
        workspaces {
            name = "terraform-cloud-import" # workspace on terraform cloud
        }
    }
}

Login to terraform cloud if you are already not logged in using terraform login. You will be redirected to the login page where you will have to login, create a token and copy that token back into the console.

Let's assume one of two scenarios

1. You have all the infra required for your project created already and now you would like to maintain it using terraform.
2. You have the infra managed by terraform but you had to create just one resource directly in the cloud provider console and now you want to add it to your terraform config.

In either cases, when you add the corresponding terraform script and run terraform plan, the plan output will first show that terraform will create the necessary resource but the apply operation will fails with an error that the required resource already exists.

The output will look something like this if you are using Terraform Cloud to create a s3 bucket which already exists in AWS with the same name.

This happens because terraform maintains a state and there is no entry within the terraform state that the S3 bucket already exists. So, it will always assume that the bucket doesn't exist and will attempt to recreate it. The solution to this problem is to import the resource into terraform state manually using the import statement. For import an S3 bucket resource into terraform state, the command would be this

terraform import aws_s3_bucket.bucket bucket-name

You can find the import statements for all terraform resources on their respective registry pages.

This is pretty straight forward if you have access to terraform store. But what if the terraform workspace is on Terraform Cloud. Then the process involves few more steps. Let's go through them.

Create a local state but link it with the state on terraform cloud.

First step is to create a terraform workspace locally and use Remote State feature to point the state storage to Terraform cloud

Add the following snippet to your terraform config files

 terraform {
    backend "remote" {
        organization = "pavan-playground" # org name 
        workspaces {
            name = "terraform-cloud-import" # workspace on terraform cloud
        }
    }
}

Login to terraform cloud if you are already not logged in using terraform login. You will be redirected to the login page where you will have to login, create a token and copy and paste that token back into the console.

Once done, run terraform init to initiate the workspace

Import the required resource into state

The resources we need to import are the following.

resource "aws_s3_bucket" "b" {
  bucket = "pavan-terraform-import-bucket"
}

resource "aws_s3_bucket_acl" "example" {
  bucket = aws_s3_bucket.b.id
  acl    = "private"
}

So we need to run the following two commands to import them into terraform state.

terraform import aws_s3_bucket.b pavan-terraform-import-bucket
terraform import aws_s3_bucket_acl.example pavan-terraform-import-bucket

Please note that you will have to be logged into AWS from your terminal before you can import the resources successfully.

If everything goes well, you should see something like this

And on terraform cloud, two new state updation entries should appear.

Thats it. Now when we run plan again, we should see that the infra is matching our terraform config (unless your configuration differs from the infra already created).

The same approach can be followed to migrate your existing infra to terraform.