Terraform Workspaces Explained

Managing multiple different environments for infrastructure is a common challenge in Infrastructure as Code (IaC). Terraform uses "workspaces" to help, but the term means different things in open-source Terraform versus platforms like Terraform Cloud and Scalr. This blog clarifies these differences, offering insights into best practices for using each.

Terraform CLI Workspace Basics

In open-source Terraform using the CLI, the "Terraform workspace" command allows you to manage multiple state files from a single set of Terraform configurations. This means you can have separate instances for different environments like development, staging, and production without duplicating your Terraform code. Each workspace maintains its own state file, preventing these different workspaces from interfering with each other. This is a simple way to manage parallel environments. If you do not create a new workspace, operations run in the default workspace. 

Key Terraform CLI workspace commands include:

• terraform workspace new <workspace name>: Creates a new workspace with a specified workspace name. 
• terraform workspace list: Lists existing different Terraform workspaces. 
• terraform workspace select <workspace name>: Switches to a different workspace using its workspace name. 
• terraform workspace show: Displays the current workspace name. 
• terraform workspace delete <workspace name>: Removes a Terraform workspace by its workspace name. 

The terraform.workspace interpolation allows terraform configurations to dynamically reference the current workspace name.This helps in making adjustments, like changing instance sizes based on the environment (e.g., 'dev' vs. 'prod'), which will be reflected when you run a terraform plan. 

CLI workspaces are best when different environments are very similar or have minor differences in their Terraform configurations. They are often used for testing, parallel development by team members, or simple environment tiers. Their main function is state isolation, not comprehensive environment management. They don't offer built-in features for managing different variable sets per workspace in a structured way (beyond tfvars files), access controls, or audit trails. One of the best practices is to use external files for environment-specific variables rather than hardcoding them. 

Their utility decreases if infrastructure configurations diverge significantly between different environments, as managing this through interpolations within the same Terraform code can become complex. When multiple team members work on the same workspace, state locking for remote backends is a crucial best practice.

Remote Backend Workspaces

As infrastructure grows and the number of team members involved increases, the limitations of CLI workspaces become apparent. Managing many different environments with CLI commands and conditional logic in Terraform configurations can be error-prone. The lack of built-in access controls, policy enforcement, and centralized visibility creates governance and security issues. 

TACOs (Terraform automation and collaboration software) like Terraform Cloud and Scalr developed more advanced Terraform workspace solutions to address these gaps. They treat workspaces as comprehensive management units, adding features for remote operations, collaboration among team members, governance, and automation.

CLI Workspaces vs. Remote Workspaces

Open-source CLI workspaces are for local state file isolation for a singular terraform configuration, helping manage different environments from the same terraform code. They are simple and suitable for individual developers or small team members with uncomplicated, similar different environments. The default workspace is used if no other is selected.  

Terraform Cloud and Scalr workspaces are more comprehensive. They are management units within larger platforms, offering remote execution (including terraform plan), collaboration tools for team members, governance (RBAC, policy enforcement for Terraform configurations), VCS integration, and more. They address the complexities of managing IaC for multiple different workspaces and terraform projects at scale.  

Choosing the Right Approach (Best Practices):

• CLI Workspaces: Good for individuals, learning, small teams with simple, homogenous different environments, or temporary feature branch testing where each new workspace is ephemeral. 
• Scalr 0r Terraform Cloud: Suitable for teams needing to scale their execution, execute runs centrally with enterprise security controls

The choice depends on team size, complexity of terraform configurations for different environments, existing tools, compliance needs, governance approach, and budget.

How do the Remote Workspaces Compare?

In the section below we'll dive into how the Terraform Cloud and Scalr workspaces are the same, but different.

Similarities of Scalr and TFC Workspaces

In Terraform Cloud (TFC) and Scalr, workspaces are where the deployments happen. They are distinct collections of infrastructure, each representing one of possibly many different environments, with its own isolated state, variables, run history, and settings for its terraform configurations. 

Key features of TFC and Scalr workspaces include:

• Workspace Sources: Both support the ability to use the Terraform CLI, a VCS integration, or No Code deployments as sources to kick off runs.
• Secure Remote State Storage: They both securely store Terraform state files, versions them for rollbacks, and encrypts them at rest. 
• VCS Integration: Scalr and TFC both integrate with Version Control Systems (VCS) like GitHub, GitLab, Azure DevOps and Bitbucket, enabling automated Terraform runs (which include a terraform plan phase) based on changes to the terraform code. This supports GitOps practices.  
• Remote Execution: Both products run terraform plan and apply operations on their own cloud infrastructure, ensuring a consistent environment and abstracting credential management from individual users. Runs can also be executed on self-hosted agents as needed.
• Collaboration and RBAC: Role-Based Access Control (RBAC) allows administrators to grant specific permissions to team members and teams per workspace.
• Policy Enforcement: Both products integrate with Open Policy Agent (OPA) to enforce governance and compliance standards on Terraform configurations. 
• Private Module Registry: Team members can publish, share, and version custom Terraform modules within a private registry, promoting reuse in Terraform configurations. Workspaces can be created directly from the module registry.

Terraform Cloud and Scalr Workspace Differences

Terraform Cloud and Scalr offer different approaches to workspace management, particularly for handling different environments and Terraform projects.

Key Differences:

• GitOps Options:
  • TFC supports the merge before apply workflow.
  • Scalr supports the merge before apply workflow or apply before merge workflow (Atlantis).
• Structure:
  • TFC uses Terraform Projects to group Workspaces.
  • Scalr has a stricter Account > Environment > Workspace hierarchy, allowing for inherited configurations and policies across different environments. 
• State Backend:
  • TFC uses its own backend.
  • Scalr allows using its backend, customer-owned blob storage, or any standard Terraform backend for Terraform configurations. 
• Tool Support:
  • TFC is Terraform-only.
  • Scalr supports Terraform, OpenTofu, and Terragrunt, catering to diverse Terraform projects.  
• Pricing:
  • TFC's pricing has been based on Resources Under Management (RUM).
  • Scalr uses a run-based model.

This list is focused on the specific workspace feature; see a full list of differences here.

Workspace Dependencies

Workspace dependencies in Terraform Cloud and Scalr allow you to create execution order relationships between workspaces, ensuring that infrastructure is provisioned in the correct sequence. When you set up dependencies, a "parent" workspace must complete its run successfully before "child" workspaces can execute their plans and applies.

This feature is particularly useful for scenarios where one workspace creates foundational infrastructure (like networking or shared resources) that other workspaces need to reference or build upon. For example, a networking workspace might need to run first to create VPCs and subnets before application workspaces can deploy resources that depend on those network components.

Dependencies help prevent race conditions and ensure that infrastructure is built in a logical, coordinated manner across multiple workspaces, making it easier to manage complex, multi-tier architectures where resources have interdependencies.

Scalr Workspace Dependencies

In Scalr, sharing state and outputs between workspaces, especially in the context of dependencies, is facilitated by its architectural design and specific features:

1. Federated Environments for Cross-Environment Dependencies: Scalr's federated environments feature is designed to allow users to establish dependencies between workspaces that reside in different environments. When you create such a dependency, the downstream workspace often needs to consume outputs (which are part of the state) from the upstream workspace. Federation helps manage the visibility and access required for this.  
2. Run Triggers for Orchestration: Scalr supports run triggers, which enable you to configure a workspace to initiate a run in another workspace.This is crucial for orchestrating deployments where, for example, a network workspace must complete successfully and its outputs (like VPC IDs or subnet IDs) are needed by an application workspace. The triggered (downstream) workspace would typically be configured to fetch these necessary outputs.  
3. Using terraform_remote_state Data Source: Since Scalr supports standard Terraform operations and can use either its own managed backend or any backend natively supported by Terraform, you can use the standard terraform_remote_state data source in your Terraform configurations. This allows one workspace to read the outputs from the state file of another workspace.  
   • To use this, you would configure the terraform_remote_state data source in the dependent workspace to point to the backend and workspace name of the source workspace.
   • Scalr's federated access system would govern whether a workspace (and the team/user executing its runs) has the necessary permissions to read the state from another workspace. 

Scalr combines its platform features like Federated Environments and Run Triggers with standard Terraform capabilities (like terraform_remote_state) under its RBAC and hierarchical governance model to enable secure and controlled sharing of state and outputs between workspaces.

Terraform Cloud Workspace Dependencies

In Terraform Cloud, workspace dependencies are managed primarily through:

1. Sharing Outputs with terraform_remote_state or tfe_outputs:
   • terraform_remote_state data source: Allows a workspace to read the output values from another workspace's state file. The producer workspace defines outputs, and the consumer workspace uses this data source to fetch them.
   • tfe_outputs data source: A more secure alternative from the hashicorp/tfe provider that directly fetches specified outputs from a remote workspace without needing access to the entire state file. This is generally the preferred method over terraform_remote_state for security.
   • Permissions: Access is controlled via Remote State Sharing settings in the producer workspace, where you can specify which workspaces can access its outputs.
2. Automating with Run Triggers:

To orchestrate a series of dependent deployments, Run Triggers automatically queue a run in a consumer workspace after a successful apply in a source (producer) workspace. This ensures that dependent infrastructure is updated when its underlying dependencies change. Run triggers are configured in the consumer workspace's settings.

You define what data (outputs) a "producer" workspace should share. A "consumer" workspace then uses data sources (ideally tfe_outputs or terraform_remote_state) to access this data. To automate this process, you set up Run Triggers so that changes in the producer automatically trigger updates in the consumer. This allows for modular infrastructure where different teams or components can manage their parts independently while still being able to reference and react to each other's resources.

Summary

Workspaces are an important abstraction in IaC, helping manage Terraform configurations for different environments. Their evolution from simple CLI state separators (where each new workspace gets a unique workspace name and state) to components of sophisticated platforms like Terraform Cloud and Scalr reflects the growing complexity of infrastructure management. Selecting the right workspace model involves matching its philosophy and features, including how it manages Terraform code for different workspaces and Terraform projects, to an organization's specific operational needs and goals for its team members.

Appendix

How to use the Terraform CLI Workspace Command

1. Terraform Configuration (main.tf):

Create a Terraform configuration with this basic example:

# main.tf

# Define a variable for an instance name prefix
variable "instance_name_prefix" {
  type    = string
  default = "my-instance"
}

# Define a local variable to change tags based on the workspace
locals {
  common_tags = {
    Provisioner = "Terraform"
    Workspace   = terraform.workspace # This will be 'default', 'dev', or 'prod'
  }

  # Example: Add an environment-specific tag
  environment_specific_tag = terraform.workspace == "prod"? { Environment = "Production" } : { Environment = "Development" }

  # Merge common tags with environment-specific tags
  all_tags = merge(local.common_tags, local.environment_specific_tag)
}

# Example resource (could be any resource, e.g., an AWS instance, Azure VM, etc.)
# For this example, we'll just output the tags to demonstrate the concept
# as creating actual cloud resources requires provider configuration and credentials.

output "instance_tags" {
  value       = local.all_tags
  description = "The tags that would be applied to an instance in this workspace."
}

output "current_workspace_name" {
  value       = terraform.workspace
  description = "The name of the currently active Terraform workspace."
}

2. Shell Commands and Expected Output:

Save the code above into a file named main.tf. Then, open your terminal in that directory and run the following commands:

# Initialize Terraform
terraform init

(Output will show initialization of backend and plugins) 

# Check Current Workspace
terraform workspace show

Expected Output:

default

# Apply in Default Workspace
terraform apply -auto-approve

Expected Output (will include):

Outputs:

current_workspace_name = "default"
instance_tags = {
  "Environment" = "Development"
  "Provisioner" = "Terraform"
  "Workspace"   = "default"
}

# Create a New Workspace (e.g., 'dev')
terraform workspace new dev

Expected Output:

Created and switched to workspace "dev"!

# List Workspaces
terraform workspace list

Expected Output:

  default
* dev

# Apply in 'dev' Workspace
terraform apply -auto-approve

Expected Output (will include):

Outputs:

current_workspace_name = "dev"
instance_tags = {
  "Environment" = "Development"
  "Provisioner" = "Terraform"
  "Workspace"   = "dev"
}

# Create Another Workspace (e.g., 'prod')
terraform workspace new prod

Expected Output:

Created and switched to workspace "prod"!

# Apply in 'prod' Workspace
terraform apply -auto-approve

Expected Output (will include):

Outputs:

current_workspace_name = "prod"
instance_tags = {
  "Environment" = "Production"
  "Provisioner" = "Terraform"
  "Workspace"   = "prod"
}

# Switch Back to 'dev' Workspace
terraform workspace select dev

Expected Output:

Switched to workspace "dev".

# Delete a Workspace (e.g., 'dev')
# First, switch to a different workspace (you cannot delete the active one, except 'default' under specific conditions)
terraform workspace select default
terraform workspace delete dev

Expected Output:

Switched to workspace "default".
Deleted workspace "dev"!