Fit tfpolicy into a multi-stage Terraform pipeline
This covers where the policy work belongs in a pipeline that already has validate, plan, and apply stages, and what each gate is honestly capable of blocking.
Read the limitation section first. It changes how you design the rest.
What the policy job can and cannot block
The tfpolicy CLI ships three subcommands: validate, test, and version. HashiCorp documents it as a tool for validating and testing policies locally. There is no command that takes a terraform plan and judges it, and the terraform plan command has no flag for handing it a policy directory either.
So a CI job running tfpolicy cannot block a plan that violates a policy. What it can do is prove that the policies parse, and that their tests still describe the behaviour they claim to, so a broken or dishonest policy never reaches the place where enforcement really happens.
That place is HCP Terraform, once the policy repository is registered as a policy set. Policies evaluate against the real plan there, and enforcement_level = "mandatory" stops the run.
The two jobs are doing different work:
| Gate | Runs where | Blocks | Catches |
|---|---|---|---|
tfpolicy validate and test | Your CI | A broken or vacuous policy set | Policies that do not parse, or no longer fire on the cases their tests describe |
| Policy set evaluation | HCP Terraform | A non-compliant plan | Configuration that violates a mandatory policy |
If you read the CI policy job as a deployment gate, you will believe you are protected when you are not. Say that out loud in the pull request that introduces it, because the green check is persuasive and wrong.
Get the policy set to the workspaces that need it
Your CI pipeline tests the policies. It does not deliver them. Delivery is a separate path, and it is the one that decides whether a control actually applies to a consumer.
Policies are hosted in a version control repository connected to HCP Terraform and registered there as a policy set. The policy set is then scoped, and the scoping options are what you design against:
| Scope | Use it for |
|---|---|
| Global | Controls that apply to the whole organisation, such as approved regions |
| Project | Controls for one product or business unit |
| Workspace | A control that applies to one deployment only, usually a temporary exception |
| Workspace tag | Controls that follow a classification, such as every workspace tagged production (tag scoping is in beta) |
You can also exclude specific projects, workspaces, or tags from a set, which is how a documented exception gets recorded as configuration rather than as a policy edit.
Tag-based scoping is the one worth designing around early. Scoping by workspace means somebody has to remember to attach the set every time a workspace is created, and that is the step that gets skipped. Scoping by tag means a workspace inherits the control the moment it is labelled, so the enforcement follows a property of the workspace rather than somebody's memory.
Two consequences to plan for.
A consumer does not get your policies by calling your module. Modules and policy sets are separate artifacts on separate distribution channels, so a team using your hardened module in a workspace with no policy set attached is unguarded, and a team in a scoped workspace is guarded even if they never touch your module. If you publish internal modules, attach the matching policy set at workspace creation time rather than expecting teams to opt in. See the discussion of forked and in-house modules for why that is the closest available thing to shipping policy inside a module.
Anyone running Terraform outside HCP Terraform is outside this enforcement path. That does not have to mean no enforcement at all: Gate a Terraform plan with tfpolicy without HCP Terraform builds a resource-policy gate out of terraform show -json and tfpolicy test, which runs in any pipeline. It does not cover module or provider policies, and it does not reach an apply run from someone's laptop, which is where Azure Policy stays the backstop.
The job layout
Four jobs. Pull requests and pushes to main run validate, policy, and plan. Apply is deliberately outside that path.
name: Terraform
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
inputs:
run_apply:
description: 'Type "apply" to run terraform apply against Azure. Anything else plans only.'
required: true
default: 'plan-only'
type: string
permissions:
contents: read
id-token: write # required for Azure OIDC federated login
pull-requests: write # required to post the plan back onto the PR
env:
# The policy framework needs Terraform 1.16 or later, which is still in beta.
TERRAFORM_VERSION: 1.16.0-beta2
TFPOLICY_VERSION: 0.1.0
WORKING_DIR: examples/compliant
concurrency:
group: terraform-${{ github.ref }}
cancel-in-progress: false
The concurrency group keys on the ref with cancel-in-progress: false. Cancelling a Terraform job partway through is how you get a stale state lock, so let runs queue instead.
Keep the policy job free of cloud credentials
Give policy its own job rather than bolting it onto validate or plan. It needs no Azure access at all, which means it runs on a pull request from a fork, and it gives feedback in under a minute instead of waiting behind an OIDC login.
policy:
name: Policy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install tfpolicy
run: |
set -euo pipefail
base="https://releases.hashicorp.com/tfpolicy/${TFPOLICY_VERSION}"
zip="tfpolicy_${TFPOLICY_VERSION}_linux_amd64.zip"
curl -fsSL -o "$zip" "$base/$zip"
curl -fsSL -o SHA256SUMS "$base/tfpolicy_${TFPOLICY_VERSION}_SHA256SUMS"
sha256sum -c --ignore-missing SHA256SUMS
unzip -q "$zip"
install -m 0755 tfpolicy /usr/local/bin/tfpolicy
tfpolicy version
- name: Validate policies
run: tfpolicy validate --policies=policies/
- name: Run policy tests
run: tfpolicy test --policies=policies/ --tests=tests/
Verify the download against the published checksums. There is no package manager distribution for a beta binary, so the pipeline is fetching an executable over the network on every run and then running it. sha256sum -c is the only thing standing between you and whatever that URL served today.
Pin TFPOLICY_VERSION explicitly. Beta tools change behaviour between patch releases, and a policy suite that silently starts passing after an unpinned upgrade is the failure this whole page is about.
Order the gates by cost
plan:
name: Plan
runs-on: ubuntu-latest
needs: [validate, policy]
Validate and policy run in parallel with no credentials. Plan waits for both. Nothing that touches Azure starts until the cheap checks have passed, so a formatting error never burns an OIDC login or a state lock.
Validate is the fast syntax gate:
validate:
name: Validate
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: ${{ env.TERRAFORM_VERSION }}
terraform_wrapper: false
- name: Check formatting
run: terraform fmt -check -recursive -diff
- name: Init without backend
working-directory: ${{ env.WORKING_DIR }}
run: terraform init -backend=false -input=false
- name: Validate
working-directory: ${{ env.WORKING_DIR }}
run: terraform validate
terraform init -backend=false is what keeps this job credential-free. It resolves modules and providers without touching remote state.
Skip rather than fail when the cloud is not wired up
A fresh clone of a repository should not show a broken pipeline. Make the cloud-dependent jobs skip themselves until the variables exist:
if: vars.AZURE_CLIENT_ID != ''
Then say so in the run summary, so the skip is visible instead of mysterious:
- name: Note if Azure is not configured
if: vars.AZURE_CLIENT_ID == ''
run: |
echo "### Plan skipped" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "No AZURE_CLIENT_ID repository variable is set, so the plan and apply jobs were skipped." >> "$GITHUB_STEP_SUMMARY"
echo "Policies and their tests still ran. See the README for the Azure OIDC setup steps." >> "$GITHUB_STEP_SUMMARY"
Policies and tests still run in that state, which is the useful property. Someone can fork the repository, change a policy, and get real feedback without any Azure access.
Produce the plan once and apply that exact file
The plan job writes a plan file and uploads it as an artifact:
- name: Plan
id: plan
working-directory: ${{ env.WORKING_DIR }}
env:
ARM_USE_OIDC: true
ARM_CLIENT_ID: ${{ vars.AZURE_CLIENT_ID }}
ARM_TENANT_ID: ${{ vars.AZURE_TENANT_ID }}
ARM_SUBSCRIPTION_ID: ${{ vars.AZURE_SUBSCRIPTION_ID }}
run: |
set -euo pipefail
set +e
terraform plan -input=false -detailed-exitcode -out=tfplan
code=$?
set -e
case $code in
0) echo "has_changes=false" >> "$GITHUB_OUTPUT" ;;
2) echo "has_changes=true" >> "$GITHUB_OUTPUT" ;;
*) exit $code ;;
esac
terraform show -no-color tfplan > plan.txt
- name: Upload plan
uses: actions/upload-artifact@v4
with:
name: tfplan
path: |
${{ env.WORKING_DIR }}/tfplan
${{ env.WORKING_DIR }}/plan.txt
retention-days: 5
-detailed-exitcode returns 0 for no changes, 2 for changes, and anything else for a real error. Handling those three cases separately is what lets a downstream job know whether there is anything to apply, without parsing plan text.
The apply job downloads that artifact rather than re-planning. What gets reviewed is what gets built. A fresh plan at apply time can differ from the one a human approved, because the world moved in between.
Make apply need two independent things
apply:
name: Apply
runs-on: ubuntu-latest
needs: plan
if: github.event_name == 'workflow_dispatch' && inputs.run_apply == 'apply'
environment: production
Two conditions have to hold, and neither is reachable by merging code. The run must have been started by hand from the Actions tab, and the person starting it must have typed apply into the confirmation box. A merge to main never reaches this job.
The environment: production line adds a third gate on top, since a GitHub environment with required reviewers holds the job until somebody approves it.
Then refuse to apply without remote state:
if [ -n "${{ vars.TFSTATE_RESOURCE_GROUP }}" ]; then
terraform init -input=false \
-backend-config="resource_group_name=${{ vars.TFSTATE_RESOURCE_GROUP }}" \
-backend-config="storage_account_name=${{ vars.TFSTATE_STORAGE_ACCOUNT }}" \
-backend-config="container_name=${{ vars.TFSTATE_CONTAINER }}" \
-backend-config="key=${{ vars.TFSTATE_KEY }}"
else
echo "::error::Apply needs remote state. Set the TFSTATE_* repository variables first."
exit 1
fi
The plan job falls back to ephemeral local state when those variables are missing, which is fine for previewing a change. Applying with throwaway state creates real Azure resources that nothing tracks afterwards, so apply stops instead.
Authenticate with OIDC, not a stored secret
The workflow logs in with federated credentials, so no long-lived secret sits in the repository:
- name: Log in to Azure
uses: azure/login@v2
with:
client-id: ${{ vars.AZURE_CLIENT_ID }}
tenant-id: ${{ vars.AZURE_TENANT_ID }}
subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}
Create the app registration and add a federated credential for the branch you deploy from:
az ad app federated-credential create --id <APP_ID> --parameters '{
"name": "main-branch",
"issuer": "https://token.actions.githubusercontent.com",
"subject": "repo:<org>/<repo>:ref:refs/heads/main",
"audiences": ["api://AzureADTokenExchange"]
}'
AZURE_CLIENT_ID, AZURE_TENANT_ID, and AZURE_SUBSCRIPTION_ID are repository variables rather than secrets. They are identifiers, not credentials, and the if: vars.AZURE_CLIENT_ID != '' skip condition needs to read one of them, which a secret would not allow.
Post the plan onto the pull request
Reviewers should not have to open a job log to see what will change:
- name: Comment plan on the pull request
if: github.event_name == 'pull_request'
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const plan = fs.readFileSync('${{ env.WORKING_DIR }}/plan.txt', 'utf8');
const clipped = plan.length > 60000
? plan.slice(0, 60000) + '\n\n... truncated, see the job log for the full plan.'
: plan;
await github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `### Terraform plan\n\n<details><summary>Show plan</summary>\n\n\`\`\`terraform\n${clipped}\n\`\`\`\n\n</details>`
});
Clip the output. GitHub rejects comments over its size limit, and a plan for a landing zone will exceed it, which turns a helpful step into a failing one at the worst moment.
Where this maps onto Azure DevOps
If your pipelines are Azure DevOps rather than GitHub Actions, the gate structure carries over directly. The implementation above is the one I have actually run, so treat this as the mapping rather than a tested translation:
| GitHub Actions | Azure DevOps |
|---|---|
Job with needs: | Stage with dependsOn |
environment: production with required reviewers | Environment with an approval check |
workflow_dispatch input | Runtime parameter on a manually queued run |
upload-artifact and download-artifact | publish and download pipeline artifacts |
vars.AZURE_CLIENT_ID | Variable group or pipeline variable |
azure/login@v2 with OIDC | Azure Resource Manager service connection with workload identity federation |
The property to preserve is that the policy stage carries no service connection. If the stage that checks your guardrails needs cloud credentials, it stops running on contributions from outside the team, which is where guardrails matter most.
Running more than one policy set
When you split policies by domain rather than keeping one flat directory, fan the policy job out:
strategy:
fail-fast: false
matrix:
domain: [storage, networking, identity]
Then point the commands at the matching directories:
tfpolicy validate --policies=policies/${{ matrix.domain }}/
tfpolicy test --policies=policies/${{ matrix.domain }}/ --tests=tests/${{ matrix.domain }}/
fail-fast: false so one broken domain does not cancel the others. When three policy sets are failing, you want all three in one run rather than discovering them one merge at a time.
Splitting by domain also sidesteps the flat input namespace. tfpolicy reads overrides from TFPOLICY_INPUT_<name> and loads every file under --policies together, so two files declaring the same input name are ambiguous. Separate invocations keep separate namespaces.