The Developer's Guide to HCL, Part 3: Common Challenges

Developers working with HCL, particularly within Terraform, often encounter recurring challenges. This section delves into these common issues, drawing insights from community discussions, and offers practical solutions. Part 1 covered the basics, and Part 2 explained the syntax.

Looping: for_each vs. count

Choosing between for_each and count for creating multiple resources is a critical decision.

• count: Creates a specified number of resources, tracked by a numeric index (count.index).
  • Pitfall: If you remove an item from the middle of a list used with count, every subsequent resource will be destroyed and recreated because their indices shift. This is highly disruptive.
  • Best Use: Use count only for creating a specific number of identical, interchangeable resources, or for conditionally creating a single resource (count = var.enabled ? 1 : 0).
• for_each: Iterates over a map or a set of strings, creating a resource for each item. Each resource is tracked by the map key or set value, providing a stable identity.
  • Benefit: Removing an item only affects that specific resource; all others remain untouched. This is much safer and more predictable for managing dynamic collections.
  • Best Use: Prefer for_each for almost all scenarios involving multiple resources. It provides stable identity and avoids the dangerous pitfalls of count.

Table 3: count vs. for_each: Key Differences

Dynamic Blocks: Generating Configuration Programmatically

Dynamic blocks are a DRY (Don't Repeat Yourself) mechanism for programmatically generating multiple nested blocks, like ingress rules in a security group.

• How it works: A dynamic block uses a for_each expression to iterate over a collection. Inside its content block, it defines the structure of each generated nested block.

Example: Dynamic AWS Security Group Rules

variable "ingress_ports" {
  type    = list(number)
  default = [80, 443, 8080]
}

resource "aws_security_group" "web_sg" {
  # ... other config ...

  dynamic "ingress" {
    for_each = toset(var.ingress_ports) # Use a set for the for_each
    content {
      from_port   = ingress.value
      to_port     = ingress.value
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
    }
  }
}

This is far cleaner than writing three separate static ingress blocks.

Navigating HCL Syntax Errors and Runtime Issues

Errors fall into two main categories:

1. HCL Syntax Errors: Caught by terraform validate. These are mistakes in the code itself, like missing braces {}, unclosed quotes "", or misspelled keywords. Use your IDE's HCL extension and run terraform validate frequently to catch these early.
2. Provider/Runtime Errors: Occur during terraform plan or apply. These are not HCL errors, but failures from the cloud provider's API (e.g., "Insufficient Permissions," "Invalid Subnet ID," API rate limiting). The error message from Terraform will typically include details from the provider, which is your key to debugging the issue.

A frustrating issue is when terraform apply fails after a plan looked correct. This can be caused by external infrastructure changes, API eventual consistency, or provider bugs.

Managing Variables and Locals Effectively

• Variables (variable blocks): These are the parameters of your configuration.
  • Best Practices: Always define a type, add a description, and set a default value if the variable should be optional. Use sensitive = true for secrets to prevent them from being displayed in logs. Add validation blocks to enforce constraints.
• Local Values (locals block): These are intermediate values or complex expressions given a name to improve readability and avoid repetition within a module.
  • Best Practices: Use locals to create derived values, format complex strings, or simplify conditional logic. This keeps your resource blocks cleaner.

Secrets Management in HCL Configurations

• The Right Way:
  1. Use a Secrets Manager: Store secrets in a dedicated service like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault.
  2. Use Data Sources: Use a Terraform data source (e.g., data "aws_secretsmanager_secret_version" "db_creds") to fetch the secret at runtime. The secret value lives only in Terraform's memory during the run and is never stored in the state file or configuration.
  3. Use Environment Variables: For CI/CD, inject secrets as environment variables (e.g., TF_VAR_api_key). This is a secure way to pass credentials to Terraform without writing them to disk.
  4. Mark Variables as sensitive: Always mark variables that will hold secret data with sensitive = true. This prevents Terraform from showing the value in plan or apply output.

Next up, best practices:

The Developer’s Guide to HCL, Part 4: Best Practices

Write clean, scalable HCL. Learn best practices for project structure, module design, state management, code formatting, validation, and testing your IaC.

Scalr Learning CenterSebastian Stadil