How We Use Terraform

What Terraform actually is

Terraform is an open-source infrastructure-as-code (IaC) tool from HashiCorp. You declare the cloud resources you want in .tf files written in HCL (HashiCorp Configuration Language), and Terraform talks to each cloud's API to create, update, or delete resources until the real world matches your code.

The loop has three verbs you'll use constantly:

• terraform init - download the provider plugins your code needs and connect to where your state is stored. Run it once per stack, and again whenever you add a provider.
• terraform plan - compare your code to reality and print exactly what would change. Nothing is applied. This is your safety net; read it every time.
• terraform apply - make the changes from the plan. Terraform records what it created in a state file so it knows what it owns next time.

The thing that makes it declarative: you don't write "create a bucket." You write "a bucket named X should exist," and Terraform decides whether that means create it, change it, or do nothing. Here's the absolute minimum - a single AWS S3 bucket:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "example" {
  bucket = "my-globally-unique-bucket-name"
}

terraform init && terraform apply, and the bucket exists. Change the code and apply again, and only the difference is applied. Delete the block and apply, and Terraform deletes the bucket. That shape - declare, plan, apply - stays the same whether you're managing one bucket or an entire multi-cloud edge.

Why it's great (and where it bites)

What's good

• Your infrastructure is reviewable. Every change is a diff in a pull request. You can see exactly what's about to change to production before it happens - and so can a reviewer. (See Git & Version Control and how we wire that into CI.)
• It's reproducible. The same code builds the same infra in a new account or region. Disaster recovery becomes "re-run apply" instead of "remember what we clicked two years ago."
• One language, many clouds. AWS, GCP, Azure, Cloudflare, GitHub, Datadog - 4,000+ providers, all driven from the same HCL and the same plan/apply workflow. Our site spans four providers in one repo.
• It documents itself. The .tf files are the source of truth for what exists. No stale wiki describing a console nobody remembers configuring.
• Drift is visible. terraform plan tells you if someone hand-edited a resource in the console. That's a config-management feature and a security feature - unexpected drift can be the first sign of a problem.

Where it bites

• State is the whole ballgame. Terraform tracks what it owns in a state file. Lose it, corrupt it, or let two people apply at once, and you get a bad day. Remote, locked, versioned state is not optional (we'll show ours).
• State can contain secrets. Generated passwords, keys, and connection strings land in state in plaintext. State must be treated as sensitive and access-controlled.
• The plan is a promise, not a guarantee. Most of the time apply matches plan. Occasionally an API rejects a change mid-apply and you're left in a partial state. Small, frequent applies hurt less than big-bang ones.
• Provider upgrades can surprise you. A major-version provider bump can rename or deprecate arguments. Pin versions and read the changelog (we use ~> constraints and commit the lock file).
• Destroy is real. terraform destroy - or removing a resource block and applying - deletes live infrastructure. plan first, every time, and read the "destroy" lines carefully.

A note on licensing & OpenTofu

Anatomy of a Terraform stack

Open infra/terraform/aws/versions.tf in another tab and follow along - it's short and shows the three pieces that configure Terraform itself.

The terraform {} block - configuring the tool

This block configures Terraform itself, not any cloud. It pins versions and says where state lives, so every laptop and every CI run behaves identically:

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"     # any 5.x, never 6.0
    }
  }

  backend "gcs" {            # where state is stored
    bucket = "csoh-org-495800-tfstate"
    prefix = "csoh/aws"
  }
}

Three things to notice: required_version refuses to run on an old CLI; the ~> ("pessimistic") constraint takes bug-fix releases but never a breaking major bump; and the backend says state lives in a Google Cloud Storage bucket. Yes - our AWS stack keeps its state in GCS. That's deliberate: one state store to secure instead of one per cloud (the rationale is commented at the top of the file).

Providers - the plugins that talk to each cloud

A provider is the plugin that translates HCL into a specific platform's API calls. You declare which ones you need (above) and configure them (below):

provider "aws" {
  region              = var.aws_region
  allowed_account_ids = [var.aws_account_id]   # safety tripwire
}

Notice what's not here: credentials. They come from the environment - your local AWS profile, or in CI a short-lived OIDC token. That's why no secrets appear in the file. allowed_account_ids is a guardrail we love: if the credentials Terraform happens to be using point at the wrong account, the apply aborts before touching anything.

Variables, resources, and outputs

The other three building blocks, each in its own file by convention:

• variable (in variables.tf) - named inputs with a type and an optional default, read elsewhere as var.name. Ours all have defaults, so the stack runs with zero manual input.
• resource (in s3.tf, etc.) - one real cloud object to create and manage. resource "TYPE" "NAME" { ... }; refer to it elsewhere as TYPE.NAME.attribute.
• output (in outputs.tf) - values to surface after apply (a bucket name, a role ARN). We feed these into GitHub repo Variables that the deploy workflow reads.

References between blocks (var.bucket_name, aws_s3_bucket.site.arn) do double duty: they pass values around and they tell Terraform the dependency order, so it builds the bucket before the role that points at it. You never sequence things by hand.

A tour of CSOH's four stacks

The entire site is static, served active/active from three clouds behind Cloudflare. That shape is described in How We Deploy; here we're reading the Terraform that builds it. The code lives in four independent stacks under infra/terraform/:

infra/terraform/
  aws/          S3 (private) + CloudFront/OAC + GitHub OIDC role
  azure/        Storage account + $web static website + federated cred
  gcp/          Cloud Run + Artifact Registry + Workload Identity Federation
  cloudflare/   Load Balancer + pool/monitor + header/redirect/cache rules

Each directory is its own stack with its own state - you cd in (or use -chdir) and run init/plan/apply there. All four keep state in the same GCS bucket under separate prefixes (csoh/aws, csoh/azure, csoh/prod, csoh/cloudflare): one secured store, no per-cloud bootstrapping.

Why split into four stacks instead of one?

A smaller blast radius. A bad apply to the Cloudflare stack can't accidentally delete the AWS bucket, because AWS isn't in that state file. Each cloud also authenticates differently, plans faster on its own, and can be destroyed independently. The trade-off is coordinating outputs between stacks by hand - which for a four-stack, solo-maintained repo is a fine price.

Keyless deploys: the security centerpiece

The file worth reading twice is aws/oidc.tf. It sets up deploys with no long-lived cloud secret in the repo at all. Instead, GitHub Actions proves its identity to AWS with a short-lived, signed OIDC token, and AWS hands back temporary credentials. The trust is scoped tighter than most people's first attempt:

condition {
  test     = "StringEquals"
  variable = "token.actions.githubusercontent.com:sub"
  values   = ["repo:${var.github_owner}/${var.github_repo}:environment:production"]
}

Read that condition out loud: only a workflow in this repo, running in the production environment, may assume the deploy role. A token from any other repo, any fork, or any other environment is rejected. The role's permissions are equally tight - write to one bucket and invalidate one CloudFront distribution, nothing else. So even if a token leaked, it could touch exactly one site's files and nothing else in the account. GCP does the same thing with Workload Identity Federation in gcp/wif.tf; Azure with a federated credential in azure/identity.tf.

A resource, start to finish

From aws/s3.tf, the private origin bucket and one of its settings:

resource "aws_s3_bucket" "site" {
  bucket = var.bucket_name
}

resource "aws_s3_bucket_versioning" "site" {
  bucket = aws_s3_bucket.site.id      # depends on the bucket above
  versioning_configuration {
    status = "Enabled"                # cheap rollback / forensics trail
  }
}

Two idioms worth internalizing. First, AWS bucket settings are split across several small resources (versioning, encryption, public-access block, policy) that each point back at the one bucket via bucket = aws_s3_bucket.site.id - that reference both targets the bucket and orders the work. Second, versioning here is a deliberate security choice: a bad publish can be rolled back object-by-object, and accidental deletes are recoverable.

Concepts that bite newcomers

1. State is shared, so it must be remote and locked

By default Terraform writes terraform.tfstate to your local disk. That's fine for a tutorial and a disaster for a team or CI: two applies at once race and corrupt it. The fix is a remote backend (our gcs backend) so state is centralized, and state locking so a second apply waits for the first to finish. Never commit a state file to git, and never let two people run apply against the same state simultaneously.

2. State can hold secrets - treat it as sensitive

Any secret a resource generates (a database password, an access key, a TLS private key) is stored in state in plaintext, even if you marked the output sensitive (that only hides it from the console). So the bucket holding state must be encrypted, access-controlled, and versioned - exactly the properties our GCS state bucket is configured with at bootstrap.

3. Always plan before apply

Read the plan. + is create, ~ is change-in-place, -/+ is destroy and recreate (downtime!), and - is destroy. The recreate and destroy lines are where production incidents come from. In CI, save the plan and apply that exact plan so what you reviewed is what runs.

4. Pin providers and commit the lock file

terraform init writes .terraform.lock.hcl, which records the exact provider versions and their checksums. Commit it. It's the supply-chain equivalent of a package-lock.json: it guarantees every machine and every CI run uses byte-identical provider binaries, and it'll catch a tampered provider via checksum mismatch. Ours are committed in each stack (e.g. aws/.terraform.lock.hcl).

5. Drift: the console fights your code

If someone changes a resource by hand in the cloud console, your code and reality "drift." The next plan shows it - Terraform wants to put things back the way your code says. The discipline: change infrastructure in the code, not the console. We even tag every AWS resource managedBy = "terraform" as a visible reminder.

6. count and for_each - don't copy-paste resources

Need three near-identical buckets? Don't write three blocks. for_each over a map (or count over a number) makes one block manage many instances. The gotcha: count indexes by position, so removing the middle item renumbers everything after it and Terraform plans destroy/recreate. Prefer for_each with stable keys when items can be added or removed.

7. terraform -chdir and per-directory state

Because our four stacks are separate directories, we drive them without cd-ing around: terraform -chdir=infra/terraform/aws plan. Each directory is an isolated root module with its own state and its own init. Forgetting which directory you're in is the most common "why is the plan empty?" moment.

8. import existing resources instead of recreating them

Already built something by hand and want Terraform to manage it? terraform import (or an import {} block) brings an existing cloud object under management without destroying it. The alternative - letting Terraform create a duplicate - is how people accidentally end up with two load balancers.

Security baseline

Terraform is a security tool when you use it well and a footgun when you don't. From day one:

1. Remote, encrypted, versioned, access-controlled state. No local state for anything real. Lock it. Assume it contains secrets, because it can.
2. No secrets in .tf or .tfvars files. Pull credentials from the environment, a secrets manager, or - best of all - don't use long-lived credentials at all. Add *.tfvars and .terraform/ to .gitignore.
3. Prefer keyless / OIDC for CI. Our deploys carry zero static cloud keys; GitHub federates short-lived tokens to each cloud. If you take one idea from this page, take this one. See oidc.tf.
4. Least-privilege everything. The deploy role can write one bucket and invalidate one CDN - not "PowerUserAccess." Scope provider credentials and resource permissions to exactly the job.
5. Pin provider versions and commit .terraform.lock.hcl. Supply-chain hygiene; checksums catch tampering.
6. Scan your IaC. Static analyzers - Trivy, Checkov, tfsec - catch "this bucket is public" or "this SG is 0.0.0.0/0" before apply. Our deploy pipeline already runs Trivy on the GCP container image; IaC scanning is the same idea, one step earlier.
7. Guardrails against the wrong account. allowed_account_ids (AWS) and equivalent checks abort an apply pointed at the wrong place. Cheap insurance.
8. Review the plan like a code review. Treat a surprising -/+ or - the way you'd treat a surprising rm -rf in a PR.

Using our repo as a learning resource

The point of this page is that our Terraform is real, in-production, and heavily commented for beginners - every non-obvious line has a one-to-three-line explanation. A suggested reading order:

1. Start with the tool config: aws/versions.tf. The terraform{} block, provider pinning, and the remote backend in one short file.
2. Then the inputs: aws/variables.tf. How values are declared once and reused as var.x.
3. Then a real resource: aws/s3.tf and cloudfront.tf. Resources, references, and how settings split across small blocks.
4. Then the security set-piece: aws/oidc.tf. Trust policy vs permissions policy, scoped to one repo and environment.
5. Finally the edge: the cloudflare/ stack - a different provider entirely (Load Balancer, pools, header/redirect/cache rules) driven by the same workflow. Proof that the shape transfers across clouds.

The infra/README.md ties it together: the architecture diagram, the bootstrap commands, the cutover/rollback plan, and the cost table (~$8-12/mo across three clouds). Fork the repo, swap in your own account IDs and bucket names, and you have a working multi-cloud static-hosting setup to learn from.

Further reading

• Terraform documentation - the official source of truth.
• State and backends - read these early; state is where the sharp edges are.
• Terraform Registry - providers and reusable modules. Check the source before you trust a module.
• OpenTofu - the open-source, Linux Foundation fork. Drop-in for everything here.
• Trivy / Checkov - scan IaC for misconfigurations before apply.
• TFLint and terraform-docs - linting and auto-generated docs.
• On this site: How We Use GitHub Actions (the CI that runs this Terraform), How We Deploy (the architecture it builds), and Git & Version Control (the workflow that reviews every change).