How to Share Terraform State in Terraform Cloud and Scalr

Terraform state is a direct record of your deployed infrastructure, mapping your configuration code to real cloud resources. It tracks all resource IDs and their attributes, allowing Terraform to accurately plan, create, and modify your environment. Sharing this state is essential for teams and interconnected deployments, ensuring everyone has an up-to-date, consistent view of infrastructure, preventing conflicts, and enabling complex dependencies between different parts of your system. In this blog, we are going to review how to share state in remote backends, such as Scalr or Terraform Cloud.

New to state sharing in general? Check out the intro blog on sharing outputs and states in Terraform.

Why Share Terraform State?

Platforms like Scalr and Terraform Cloud simplify the learning process for Terraform state management by providing a centralized, managed backend. Users should leverage their workspace capabilities for state sharing because these platforms natively handle secure remote storage, robust state locking to prevent conflicts, and versioning for easy rollbacks. This integration becomes truly powerful when combined with run triggers: a feature that automatically initiates a run in a dependent "downstream" workspace whenever its "upstream" counterpart successfully completes an apply. This chaining of deployments, where outputs from one state file are automatically consumed by the next, creates an automated and reliable pipeline for your infrastructure.

How to Share State Between Workspaces

Both Terraform Cloud and Scalr provide solutions for managing and sharing Terraform state, making collaboration much smoother. They abstract away many of the complexities of manual state management through the concept of workspaces.

In both Terraform Cloud and Scalr, a workspace is a fundamental organizational unit that acts as an isolated object for managing a specific set of infrastructure. The workspace is where the state and all of the artifacts related to the deployment are stored.

The orchestration of state sharing and triggering runs from one succesful deployment to another are all dependent on workspaces.

Terraform Cloud

Terraform Cloud is a managed service by HashiCorp that offers a collaborative workflow for Terraform. State management is a core feature. Here are the basic concepts:

1. Workspaces: In Terraform Cloud, each Terraform configuration is associated with a workspace. Each workspace has its own remote state.
2. Permissions: Terraform Cloud offers granular permissions on workspaces, allowing you to control who can read, write, and administer each workspace, ensuring secure state sharing.
3. Workspace Access: Terraform Cloud gives users the ability to share state across the entire organization or to select specific workspaces, regardless of the projects they are in, to share with.

Once it has been decided to share state, users can add a remote state datasource snippet to their code to pull the outputs from one workspace into another.

Sharing State with terraform_remote_state: To share outputs from one workspace to another, you use the terraform_remote_state data source:

# In Workspace A (e.g., networking)
output "vpc_id" {
  value = aws_vpc.main.id
}

output "public_subnet_ids" {
  value = aws_subnet.public.*.id
}

# In Workspace B (e.g., compute), consuming outputs from Workspace A
data "terraform_remote_state" "network" {
  backend = "remote"
  config = {
    organization = "your-organization-name"
    workspaces {
      name = "workspace-a-name" # The workspace containing the network resources
    }
  }
}

resource "aws_instance" "web" {
  ami           = "ami-0abcdef1234567890"
  instance_type = "t2.micro"
  subnet_id     = data.terraform_remote_state.network.outputs.public_subnet_ids[0]
  vpc_security_group_ids = [aws_security_group.web.id]
  # ... other configurations
}

Terraform Cloud handles the secure retrieval of state from workspace-a-name and provides its outputs to workspace-b-name.

Scalr

Scalr is another powerful Terraform automation platform that provides enterprise-grade features for Terraform, including robust state management and sharing.

1. Environments and Workspaces: Similar to Terraform Cloud, Scalr organizes your deployments into Environments and Workspaces. Each Workspace stores its own remote state. A environment is the parent of a workspace.
2. Permissions: Scalr provides extensive RBAC to define granular permissions on workspaces and environments, allowing you to control who can access and interact with state files. This is crucial for maintaining a secure and compliant IaC environment.
3. Federated Environments: Scalr gets more fine-grained with how state outputs are shared with the Federated Environments feature. Federated environments are a powerful feature that enables secure and controlled interaction between Terraform workspaces that reside in different Scalr environments. By default, Scalr environments are designed to be isolated from one another for security and separation of concerns. This means that a workspace in your "Development" environment typically cannot directly see or interact with a workspace in your "Production" environment. Federated environments break down this isolation in a controlled, one-way manner.

Once a workspace is granted permission to pull outputs from another workspaces state, the users can define the remote state datasource.

Scalr also leverages the terraform_remote_state data source for sharing. The configuration is very similar to Terraform Cloud.

# In Scalr Workspace A (e.g., base-infrastructure)
output "vpc_id" {
  value = aws_vpc.main.id
}

# In Scalr Workspace B (e.g., application-deployment), consuming outputs from Workspace A
data "terraform_remote_state" "base_infra" {
  backend = "remote"
  config = {
    organization = "your-scalr-organization-id" # Scalr-specific organization ID
    workspaces {
      name = "base-infrastructure-workspace-name" # The name of the workspace in Scalr
    }
  }
}

resource "aws_instance" "app" {
  ami           = "ami-0abcdef1234567890"
  instance_type = "t2.micro"
  subnet_id     = data.terraform_remote_state.base_infra.outputs.public_subnet_ids[0]
  # ...
}

Note: The organization value in terraform_remote_state for Scalr will be your Scalr environment ID. Refer to the Scalr documentation for the precise value.

Workspace Orchestration with Run Triggers

Run triggers go hand in hand with sharing outputs from one state file into another. Normally, a successful Terraform apply will change the state file outputs, which then needs to be shared automatically with the workspaces that consume it.

Run triggers, for Terraform Cloud and Scalr, are a feature that allows you to chain Terraform workspace runs together. They create an explicit dependency where the successful completion of a Terraform run (specifically, a successful terraform apply) in one workspace, known as the "upstream" or "source" workspace, automatically initiates a new run in another "downstream" or "target" workspace.

Run triggers allow you to fully orchestrate deployments and make it possible to avoid a monolithic deployment, instead breaking deployments down into smaller workspaces that depend on each other, which is much more manageable and a better developer experience.

Summary

Scalr and Terraform Cloud offer managed, secure environments for Terraform state. Their workspaces centralize state storage, enforce state locking, and enable controlled sharing of outputs via the terraform_remote_state data source. This integrated state management, combined with run triggers, allows for automated pipelines where a successful infrastructure deployment in one workspace can automatically initiate a dependent deployment in another, streamlining complex, interconnected infrastructure operations.

Try it for free in Scalr!