Using Terraform in Azure DevOps Pipelines

This guide explains the basic steps to getting started with Terraform and Azure DevOps Pipelines

Ryan Fee

• 6 min read

1. Introduction: IaC and the Power of Pairing IaC with Azure DevOps

Maintaining infrastructure consistency and minimizing configuration drift is paramount. Trying to manage cloud resources manually is a recipe for chaos, leading to slow deployments and unreliable environments.

This is where Infrastructure as Code (IaC) becomes non-negotiable. IaC allows you to manage and provision your cloud infrastructure using code and software engineering practices.

Why Terraform/OpenTofu? Both Terraform and its open-source fork, OpenTofu, are leading declarative IaC tools. They allow you to define the desired state of your Azure infrastructure—from Resource Groups and Virtual Networks to App Services—in HashiCorp Configuration Language (HCL). This code can be version-controlled, reviewed, and tested.

Why Azure DevOps Pipelines? Azure DevOps provides a robust, native CI/CD platform to act as the orchestration engine. By integrating your IaC code into YAML pipelines, you can automate the entire lifecycle: validating changes, creating execution plans, and applying those changes across your environments in a safe, repeatable manner. The result is infrastructure deployments that are auditable and traceable.

2. Prerequisites: Getting Your Environment Ready

Before diving into the pipeline code, several foundational elements must be in place.

Infrastructure Code

Your Terraform or OpenTofu files (.tf) should be committed to an Azure Repos Git repository. Ensure your code is configured to use the Azure Provider (azurerm) or any other provider you might be using.

Remote State Management (Crucial for CI/CD)

The state file tracks the real-world resources managed by your configuration. In a CI/CD environment, this file must be stored securely and centrally to enable collaboration and state locking. You have two main approaches:

  • Option 1: Azure Blob Storage (Self-Managed)
    • Requirement: You must use Azure Blob Storage for your remote backend.
    • Setup: Manually create a dedicated Azure Storage Account and a container to host your state files before your pipeline runs. Add the AzureRM backend block to your code.
  • Option 2: Scalr (Centralized Governance)
    • Scalr provides a centralized platform for managing Terraform/OpenTofu state, runs, policy enforcement, and drift detection. This is a powerful alternative for organizations requiring advanced governance features. It's not just a backend for state storage; it orchestrated the entire Terraform/OpenTofu ecosystem.
    • When using Scalr, you authenticate the agent by providing a dedicated API token. This is typically achieved using the terraform login or tofu login command in an initial script step before the init command runs. The token must be stored securely as an Azure DevOps secret variable.

Scalr Configuration Example:

terraform {
  backend "remote" {
    hostname     = "your-account.scalr.io"
    organization = "your-organization-name"
    workspaces {
      name = "project-name-environment-name"
    }
  }
}

Permissions and Authentication

Your pipeline needs permission to create and manage resources in your Azure subscription.

  • Service Connection: Create an Azure Resource Manager Service Connection in Azure DevOps. This Service Principal (SPN) is the identity that the IaC tool will use to authenticate with Azure.
  • Security Principle: Strictly adhere to the principle of least privilege. Grant the SPN only the permissions it needs (e.g., Contributor rights to a specific subscription or Resource Group).

The Pipeline Agent

For deployment, we will utilize Microsoft-hosted agents (e.g., vmImage: 'ubuntu-latest' or 'windows-latest'). These agents come pre-installed with many necessary tools, though we will ensure the IaC tool is available if needed.

3. Designing the Pipeline: A Multi-Stage CI/CD Workflow

The safest way to run IaC in a pipeline is to separate the process into two stages: Plan and Apply. This split guarantees that a human operator can review the proposed changes before they are executed.

We recommend using native script: blocks with standard CLI commands (e.g., terraform init) instead of relying exclusively on marketplace tasks. This gives you maximum control and ensures compatibility across both Terraform and OpenTofu.

4. Full YAML Pipeline Example

To visualize the multi-stage workflow, review the full azure-pipelines.yml provided below. It showcases the necessary steps for initialization, planning, artifact publishing, and application using variables for secure backend configuration.

# This YAML defines a multi-stage pipeline for deploying infrastructure using Terraform or OpenTofu.
# It enforces a best practice: separate stages for Plan and Apply,
# with an optional manual approval gate between them.

trigger:
- main

# Define variables needed for configuration and authentication
variables:
  # The name of the folder containing your .tf files in the repository root
  - name: terraformDirectory
    value: '$(System.DefaultWorkingDirectory)/infra'
  # The version of Terraform/OpenTofu to use (e.g., 'latest' or '1.7.0')
  - name: iacVersion
    value: 'latest'

# Link to a variable group for secrets like storage account name, container name, etc.
# IMPORTANT: Secrets for Azure authentication (ARM_CLIENT_ID, etc.) are usually
# provided securely via the Service Connection (see Stage 1/Job 1).
# Replace 'tf-backend-secrets' with your actual variable group name.
- group: tf-backend-secrets

stages:
# ------------------------------
# Stage 1: Terraform Plan (CI)
# ------------------------------
- stage: Plan
  displayName: '1. Plan Infrastructure Changes'
  jobs:
  - job: Plan
    displayName: 'Generate Execution Plan'
    pool:
      vmImage: 'ubuntu-latest' # Use a Microsoft-hosted agent

    steps:
    # This task automatically downloads and installs the specified version of Terraform/Tofu.
    - task: TerraformInstaller@1
      displayName: 'Install Terraform/Tofu'
      inputs:
        terraformVersion: $(iacVersion)

    # 2. Terraform/OpenTofu Init
    # Initializes the backend configuration (Azure Storage Account)
    - script: |
        # Use 'terraform' or 'tofu' as needed
        terraform init \
          -backend-config="resource_group_name=$(RG_NAME)" \
          -backend-config="storage_account_name=$(SA_NAME)" \
          -backend-config="container_name=$(CONTAINER_NAME)" \
          -backend-config="key=production.tfstate"
      displayName: 'Terraform/Tofu Init'
      workingDirectory: $(terraformDirectory)

    # 3. Terraform/OpenTofu Plan & Save
    # Creates the execution plan file (tfplan)
    - script: |
        # Use 'terraform' or 'tofu' as needed
        terraform plan \
          -out=tfplan
      displayName: 'Terraform/Tofu Plan'
      workingDirectory: $(terraformDirectory)

    # 4. Publish the Plan as a Pipeline Artifact
    # The plan artifact is required for the Apply stage.
    - task: PublishPipelineArtifact@1
      displayName: 'Publish Plan Artifact'
      inputs:
        targetPath: '$(terraformDirectory)/tfplan'
        artifact: 'tfplan-artifact'

# ------------------------------
# Stage 2: Terraform Apply
# ------------------------------
- stage: Apply
  displayName: '2. Apply Infrastructure Changes'
  # This targets a protected environment, triggering a Manual Approval Check if configured
  environment: 'production-infra'
  dependsOn: Plan

  jobs:
  - job: Apply
    displayName: 'Execute Apply'
    pool:
      vmImage: 'ubuntu-latest' # Use the same agent pool as the Plan stage

    steps:
    # 1. Install Terraform/OpenTofu (again, required for the new job)
    - task: TerraformInstaller@1
      displayName: 'Install Terraform/OpenTofu'
      inputs:
        terraformVersion: $(iacVersion)

    # 2. Download the Plan Artifact
    - task: DownloadPipelineArtifact@2
      displayName: 'Download Plan Artifact'
      inputs:
        artifactName: 'tfplan-artifact'
        # The artifact is downloaded to $(Pipeline.Workspace)/tfplan-artifact
        targetPath: '$(System.DefaultWorkingDirectory)/tfplan-artifact'
    
    # 3. Terraform/OpenTofu Init
    # Re-initialize the backend to ensure connectivity for the apply command
    - script: |
        # Use 'terraform' or 'tofu' as needed
        terraform init \
          -backend-config="resource_group_name=$(RG_NAME)" \
          -backend-config="storage_account_name=$(SA_NAME)" \
          -backend-config="container_name=$(CONTAINER_NAME)" \
          -backend-config="key=production.tfstate"
      displayName: 'Terraform/Tofu Init for Apply'
      workingDirectory: $(terraformDirectory)
    
    # 4. Terraform/OpenTofu Apply
    # Apply the downloaded plan file.
    - script: |
        # Use 'terraform' or 'tofu' as needed
        terraform apply -auto-approve \
          $(Pipeline.Workspace)/tfplan-artifact/infra/tfplan
      displayName: 'Terraform/Tofu Apply'
      workingDirectory: $(terraformDirectory)

5.Validate and Plan

This stage’s primary goal is to validate the code, generate the execution plan, and secure that plan as an artifact for the next stage.

Step

Command/Action

Purpose

1. Initialize

terraform init or tofu init

Initializes the working directory and configures the Azure Backend, connecting to the remote state file.

2. Validate Code

terraform validate or tofu validate

Performs a quick syntax and configuration check.

3. Static Analysis (Best Practice)

tflint or similar

Runs security and best-practice checks against the code.

4. Generate Plan

terraform plan -out=tfplan or tofu plan -out=tfplan

Creates the execution plan file (tfplan) which contains the exact changes to be made.

5. Output Management

PublishPipelineArtifact@1

Publishes the generated tfplan file as a pipeline artifact, making it available to the Apply stage.

6. Manual Approval Gate

After the plan is generated, your pipeline should pause. To prevent accidental resource destruction or unintended changes when deploying infrastructure with Terraform, a manual approval check should be implemented. This process involves a designated DevOps Engineer reviewing the tfplan output, typically by executing terraform show tfplan in a pre-approval job or via a pull request comment integration, before the infrastructure is applied. This critical control is configured in Azure DevOps by defining a protected environment and adding the manual approval check to it, ensuring that the Terraform Apply stage is explicitly configured to target this protected environment.

7. Continuous Deployment - Apply

Once the plan is approved, this stage executes the change.

Step 1 Download Artifacts: Use the DownloadPipelineArtifact@2 task to retrieve the saved tfplan file from Stage 1. This helps to ensure idempotence and immutability between stages.

Step 2: Apply Plan: The agent executes the apply command using the previously generated plan file.

Step 3 Output Management: If your IaC creates resources whose outputs are needed by downstream application deployments (e.g., a database connection string), you can extract them:

8. Best Practices for Production Readiness

To truly productionize your IaC workflow, consider these best practices:

  • Secret Management: NEVER hardcode secrets in your repo or YAML. Use Azure Key Vault integration with Azure DevOps Variable Groups to securely fetch and inject sensitive data (like authentication tokens or environment-specific values) at runtime.
  • Workspace Strategy: For true isolation between development and production, use separate configuration directories or utilize Terraform/OpenTofu Workspaces (terraform workspace new <environment>) combined with dynamic pipeline variables. This ensures that a bug in your dev code cannot accidentally affect your prod state.
  • GitFlow Integration: Configure a pipeline trigger on pull requests. Whenever a developer opens a PR with infrastructure changes, automatically run Stage 1 (validate and plan). Displaying the plan output directly in the PR helps reviewers understand the impact before any code is merged to the main branch.

9. Conclusion and Next Steps

Integrating Terraform or OpenTofu with Azure DevOps Pipelines transforms infrastructure management from a manual chore into a reliable, automated engineering discipline. By following the multi-stage plan/apply workflow, you ensure every infrastructure change is reviewed, tested, and deployed with confidence.