The Developer's Guide to HCL, Part 3: Common Challenges

Solve frequent HCL issues. Learn when to use for_each vs. count, master dynamic blocks, troubleshoot errors, and manage variables and secrets effectively.

Developers working with HCL, particularly within Terraform, often encounter recurring challenges. This section delves into these common issues, drawing insights from community discussions, and offers practical solutions. Part 1 covered the basics, and Part 2 explained the syntax.

Looping: for_each vs. count

Choosing between for_each and count for creating multiple resources is a critical decision.

  • count: Creates a specified number of resources, tracked by a numeric index (count.index).
    • Pitfall: If you remove an item from the middle of a list used with count, every subsequent resource will be destroyed and recreated because their indices shift. This is highly disruptive.
    • Best Use: Use count only for creating a specific number of identical, interchangeable resources, or for conditionally creating a single resource (count = var.enabled ? 1 : 0).
  • for_each: Iterates over a map or a set of strings, creating a resource for each item. Each resource is tracked by the map key or set value, providing a stable identity.
    • Benefit: Removing an item only affects that specific resource; all others remain untouched. This is much safer and more predictable for managing dynamic collections.
    • Best Use: Prefer for_each for almost all scenarios involving multiple resources. It provides stable identity and avoids the dangerous pitfalls of count.

Table 3: count vs. for_each: Key Differences

Feature

count

for_each

Iteration Basis

Integer

Map or Set of strings

Resource Identity in State

Numeric index (e.g., aws_instance.server[0])

Map key or Set value (e.g., aws_instance.server["web-1"])

Behavior on Modification

Removing an item from a list's middle can cause recreation of subsequent items.

Removing an item only affects that specific instance. Others are untouched.

Verdict

Use with extreme caution. Good for a simple on/off switch (count = var.enabled? 1 : 0).

The default, preferred choice for creating multiple resources from a collection.

Dynamic Blocks: Generating Configuration Programmatically

Dynamic blocks are a DRY (Don't Repeat Yourself) mechanism for programmatically generating multiple nested blocks, like ingress rules in a security group.

  • How it works: A dynamic block uses a for_each expression to iterate over a collection. Inside its content block, it defines the structure of each generated nested block.

Example: Dynamic AWS Security Group Rules

variable "ingress_ports" {
  type    = list(number)
  default = [80, 443, 8080]
}

resource "aws_security_group" "web_sg" {
  # ... other config ...

  dynamic "ingress" {
    for_each = toset(var.ingress_ports) # Use a set for the for_each
    content {
      from_port   = ingress.value
      to_port     = ingress.value
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
    }
  }
}

This is far cleaner than writing three separate static ingress blocks.

Errors fall into two main categories:

  1. HCL Syntax Errors: Caught by terraform validate. These are mistakes in the code itself, like missing braces {}, unclosed quotes "", or misspelled keywords. Use your IDE's HCL extension and run terraform validate frequently to catch these early.
  2. Provider/Runtime Errors: Occur during terraform plan or apply. These are not HCL errors, but failures from the cloud provider's API (e.g., "Insufficient Permissions," "Invalid Subnet ID," API rate limiting). The error message from Terraform will typically include details from the provider, which is your key to debugging the issue.

A frustrating issue is when terraform apply fails after a plan looked correct. This can be caused by external infrastructure changes, API eventual consistency, or provider bugs.

Managing Variables and Locals Effectively

  • Variables (variable blocks): These are the parameters of your configuration.
    • Best Practices: Always define a type, add a description, and set a default value if the variable should be optional. Use sensitive = true for secrets to prevent them from being displayed in logs. Add validation blocks to enforce constraints.
  • Local Values (locals block): These are intermediate values or complex expressions given a name to improve readability and avoid repetition within a module.
    • Best Practices: Use locals to create derived values, format complex strings, or simplify conditional logic. This keeps your resource blocks cleaner.

Secrets Management in HCL Configurations

🧠
Never hardcode secrets in your .tf or .tfvars files and commit them to version control.
  • The Right Way:
    1. Use a Secrets Manager: Store secrets in a dedicated service like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault.
    2. Use Data Sources: Use a Terraform data source (e.g., data "aws_secretsmanager_secret_version" "db_creds") to fetch the secret at runtime. The secret value lives only in Terraform's memory during the run and is never stored in the state file or configuration.
    3. Use Environment Variables: For CI/CD, inject secrets as environment variables (e.g., TF_VAR_api_key). This is a secure way to pass credentials to Terraform without writing them to disk.
    4. Mark Variables as sensitive: Always mark variables that will hold secret data with sensitive = true. This prevents Terraform from showing the value in plan or apply output.

Next up, best practices:

The Developer’s Guide to HCL, Part 4: Best Practices
Write clean, scalable HCL. Learn best practices for project structure, module design, state management, code formatting, validation, and testing your IaC.

Key Sources