Terraform in CI/CD & Testing

Why this matters

Running terraform apply from your laptop is fine for learning, but it's exactly the un-auditable, "works on my machine" pattern IaC was supposed to kill. Real teams run Terraform from a pipeline: a push opens a PR, the pipeline runs fmt/validate/plan and posts the plan for review, and a merge runs apply with credentials no human holds. This module bridges toward track 10 (CI/CD and GitOps) by showing the pipeline shape and the quality gates — formatting, validation, linting, and policy checks — that belong in front of every apply.

Concepts

Why Terraform belongs in a pipeline

The same reasons declarative IaC beat imperative az (module 00) compound when you automate them:

• The reviewed plan is the applied plan. With plan -out=tfplan → apply tfplan (module 01), the pipeline applies exactly what a reviewer saw in the PR — no drift between review and execution.
• No human holds prod credentials. The pipeline authenticates as a service principal (or, better, via OIDC), so applying to production doesn't require anyone to have standing admin rights on their laptop.
• Every change is gated and logged. fmt/validate/lint/policy checks run automatically; the run log is the audit trail.

The canonical pipeline stages

A Terraform pipeline is almost always these stages, in order:

1. terraform fmt -check — fails if any file isn't canonically formatted. Cheap, catches style drift.
2. terraform init — with the remote backend (module 05) and the committed lock file for reproducible provider versions.
3. terraform validate — syntax and internal-reference check (module 00).
4. Lint / policy — tflint for provider-specific best practices, checkov (or tfsec) for security/policy scanning of the plan or config.
5. terraform plan -out=tfplan — on a PR, this is posted for human review; nothing is applied.
6. terraform apply tfplan — only on merge to the main branch, applying the exact reviewed plan.

The split matters: plan runs on the PR, apply runs on merge. A reviewer approves the plan; the merge triggers the apply of that same plan.

Authenticating a pipeline to Azure

A pipeline has no az login session, so it uses non-interactive auth via the ARM_* environment variables the azurerm provider reads:

• Service principal with a secret: ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_TENANT_ID, ARM_SUBSCRIPTION_ID. Simple, but the secret must be stored in the CI system and rotated.
• OIDC / workload identity federation (preferred): the pipeline gets a short-lived token from its CI provider, exchanged for an Azure token — no stored secret at all. Set ARM_USE_OIDC=true alongside ARM_CLIENT_ID/ARM_TENANT_ID/ARM_SUBSCRIPTION_ID. This is the same OIDC-federation idea track 7's CI module used for pushing to ACR, applied to Terraform. You'll master the mechanics in track 10; here, recognize that "no long-lived secret in CI" is the goal.

A GitHub Actions skeleton

Conceptually (you'll build the real thing in track 10), a plan-on-PR job looks like this — note it uses HashiCorp's setup-terraform action and authenticates via OIDC:

name: terraform-plan
on: pull_request
permissions:
  id-token: write     # required for OIDC
  contents: read
jobs:
  plan:
    runs-on: ubuntu-latest
    env:
      ARM_USE_OIDC: "true"
      ARM_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
      ARM_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
      ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      - run: terraform fmt -check
      - run: terraform init
      - run: terraform validate
      - run: terraform plan -out=tfplan

The apply job is nearly identical but triggers on: push to main and runs terraform apply -auto-approve tfplan (applying the saved plan; -auto-approve is acceptable here precisely because a human already reviewed the plan on the PR).

terraform validate vs fmt vs lint vs policy

Four checks that people conflate but that catch different things:

• terraform fmt — formatting only (indentation, alignment). Says nothing about correctness.
• terraform validate — internal consistency: syntax, argument names, references resolve. Doesn't contact Azure, doesn't judge quality.
• tflint — best-practice/lint: deprecated syntax, invalid instance/VM types, provider-specific pitfalls validate won't catch.
• checkov / tfsec — security & policy: "this storage account allows public blob access," "this NSG rule opens 22 to the world." These scan for misconfigurations, not syntax.

You want all four, and they run cheaply before any apply.

A taste of terraform test

Terraform 1.6+ ships a native testing framework: .tftest.hcl files with run blocks that execute plan (or apply) against your config and assert on the results:

# tests/network.tftest.hcl
run "vnet_has_expected_cidr" {
  command = plan

  assert {
    condition     = azurerm_virtual_network.hub.address_space[0] == "10.0.0.0/16"
    error_message = "Hub VNet CIDR must be 10.0.0.0/16"
  }
}

terraform test runs these. A plan-based test is fast and free (no resources created); an apply-based test creates real resources, asserts, then destroys them — powerful but billable, so use plan tests for most logic checks. This is the direction infrastructure testing is heading; you're seeing the shape, not mastering it, here.

Command reference

Flag breakdown — terraform fmt -check -recursive:

• -check — exit non-zero if files would be reformatted, but don't rewrite them. This is the CI-appropriate form (fail the build; don't silently mutate).
• -recursive — also check subdirectories (so child modules are covered).

Flag breakdown — terraform apply tfplan (vs interactive apply):

• Passing a saved plan file makes apply skip re-computation and the confirmation prompt — it applies precisely that plan. Combined with plan -out on the PR, it guarantees the applied change equals the reviewed change.

Hands-on exercises

Mostly free — you'll run the local checks that a pipeline runs. A couple of optional steps install external tools; the plan/validate/fmt steps need only Terraform.

1. Set up a small config to gate. New directory with a resource group + VNet (reuse module 02's shape):

   mkdir -p ~/tf-labs/07-cicd && cd ~/tf-labs/07-cicd
   export ARM_SUBSCRIPTION_ID="$(az account show --query id -o tsv)"
   

   Create main.tf (provider skeleton + RG + a VNet), then terraform init.

2. Run the pipeline's format gate. Deliberately misformat main.tf (mangle indentation), then run the check form:

   terraform fmt -check -recursive
   

   Verify: it exits non-zero and names the unformatted file — this is what fails a CI build. Fix it with terraform fmt, then rerun terraform fmt -check and confirm it now exits 0 (clean).

3. Run the validate gate. Introduce a typo'd argument (e.g. adress_space), run:

   terraform validate
   

   Verify: it fails with the unexpected-argument error. Fix the typo and confirm validate reports success. This is CI stage 3.

4. Produce and inspect a saved plan. Generate the artifact a reviewer would approve:

   terraform plan -out=tfplan
   terraform show tfplan | head -40
   terraform show -json tfplan > plan.json
   

   Verify: tfplan (binary) and plan.json exist. The JSON is what policy tools like checkov scan. Note: you'd never commit tfplan — add it to .gitignore (it may contain sensitive values).

5. Apply the exact saved plan. Prove plan→apply fidelity:

   terraform apply tfplan
   

   Verify: it applies with no confirmation prompt and no re-computation — exactly the reviewed plan. This is why CI uses -out then apply <file>.

6. Write and run a terraform test. Create tests/vnet.tftest.hcl:

   run "vnet_cidr_is_correct" {
     command = plan
     assert {
       condition     = azurerm_virtual_network.hub.address_space[0] == "10.0.0.0/16"
       error_message = "Hub VNet CIDR must be 10.0.0.0/16"
     }
   }
   

   (Adjust the resource local name to match yours.) Run:

   terraform test
   

   Verify: it reports the test passed. Now break it — change the assert's expected CIDR to 10.99.0.0/16 and rerun; confirm the test fails with your error message. This plan-based test created nothing in Azure. Revert the assertion.

7. Diagnose and fix: a policy check catches a real risk. Add an NSG rule that opens SSH to the whole internet (a genuine misconfiguration):

   security_rule {
     name                       = "allow-ssh-anywhere"
     priority                   = 100
     direction                  = "Inbound"
     access                     = "Allow"
     protocol                   = "Tcp"
     source_port_range          = "*"
     destination_port_range     = "22"
     source_address_prefix      = "*"     # <-- the problem: open to the world
     destination_address_prefix = "*"
   }
   

   (Attach it to an NSG resource.) If you have checkov installed (pip install checkov), run checkov -d .; otherwise reason it through.

   Verify / reason: checkov flags a check like "Ensure NSG does not allow SSH (22) from the internet." Fix by scoping source_address_prefix to a specific admin CIDR (or a Bastion subnet) instead of * — the same track 5 NSG discipline, now enforced automatically by policy scanning rather than a reviewer's memory. This is the class of bug CI policy checks exist to stop before apply.

8. Optional: run tflint. If you install tflint (brew/download), run tflint --init && tflint in the directory.

   Verify / reason: it surfaces provider best-practice issues (e.g. an invalid VM size, deprecated arguments) that validate passes over. Note where each of your four gates would catch a different bug.

9. Sketch the pipeline (no apply). In a README or comment in your repo, write the six pipeline stages in order and mark which run on PR (plan) versus merge (apply), and note that the pipeline would authenticate via ARM_USE_OIDC=true with no stored secret. You'll build this for real in track 10 — the goal here is that you can describe it from memory.

10. Clean up. Destroy the small stack and remove local plan artifacts:

    terraform destroy
    rm -f tfplan plan.json
    

    Verify: terraform destroy removes the RG/VNet/NSG; confirm with az group list -o table. Nothing here was expensive, but the habit stands.

Independent challenge

Without copying the exercises, take the module-04 network module (or your module-06 infra config) and add a complete local "pre-merge gate" script — a single shell script that runs, in order, terraform fmt -check -recursive, terraform init, terraform validate, a terraform test with at least one meaningful plan-based assertion about your network (e.g. that the number of subnets equals the length of your input list), and finally terraform plan -out=tfplan — and that exits non-zero if any stage fails, exactly as CI would. Deliberately introduce one failure of each kind (a format issue, a bad argument, a wrong CIDR the test catches) and confirm your script stops at the right stage each time. This integrates module 04 (modules), module 02 (fmt/validate), module 01 (plan -out), and this module's testing — and is the exact gate you'll lift into a real pipeline in track 10.

Common mistakes & troubleshooting

• Running apply on the PR instead of plan. The PR should only plan (for review); apply belongs on merge. Applying on PR means unreviewed changes hit real infrastructure.
• Re-planning at apply time instead of applying a saved plan. If the merge job runs a fresh plan+apply rather than apply tfplan, the applied change can differ from what was reviewed (state or a dependency moved in between). Use plan -out → apply <file>.
• Storing a long-lived SP secret in CI. It's the thing most likely to leak. Prefer OIDC (ARM_USE_OIDC=true) with no stored secret; you'll set this up in track 10.
• Treating fmt/validate as sufficient quality gates. They catch formatting and syntax, not insecure or wasteful configuration. Add tflint and checkov/tfsec for the bugs that actually cost money or open security holes.
• Committing tfplan/plan.json. These can contain sensitive values and are ephemeral CI artifacts. .gitignore them.
• Cost pitfall: an apply-based terraform test creates real resources (then destroys them) — a test suite full of apply-tests against AKS-shaped resources bills every run. Keep most tests command = plan (free), and reserve apply-tests for the few things that genuinely need a live resource. A misconfigured CI that runs apply on every PR push, rather than only on merge, can also create/leak resources fast.

Checkpoint quiz

Write down your answer to each question before expanding it — checking without attempting first is the single easiest way to fool yourself into thinking you've learned this.

1. Why does running Terraform from a pipeline beat running it from your laptop — give two reasons tied to earlier modules.
2. List the canonical Terraform pipeline stages in order, and mark which run on a PR versus on merge.
3. What's the point of plan -out=tfplan followed by apply tfplan instead of just running apply?
4. How does a pipeline authenticate to Azure with no az login, and what's the preferred credential-less method?
5. Distinguish what fmt, validate, tflint, and checkov each catch.
6. What does a command = plan terraform test cost in Azure resources, and when would you use command = apply instead?
7. -auto-approve is dangerous on your laptop but acceptable in the CI apply job. Why is it acceptable there?
8. A checkov run fails on an NSG rule. What kind of problem is it likely catching that terraform validate would have passed?

Next

08 — Capstone project: combine everything — a multi-module config with remote state provisioning a VNet+NSG and AKS+ACR, variables and outputs used sensibly — then tear it all down and prove nothing's left.