Monorepo consolidation — merge terraform-aws-cli + terraform-azure-cli (draft plan)

[bgauduch]

Monorepo refactor plan — consolidate the Terraform + cloud-CLI images

Status: Draft proposal — backlog. No ADR and no roadmap phase number are
assigned yet: the ADR is written when this issue is actually handled, not
now. Tracked from the modernization tracking issue (#106).

Scope: merge the sibling image repositories
terraform-aws-cli and
terraform-azure-cli into a
single monorepo, structured so additional providers (e.g. a future
terraform-gcp-cli) drop in without rework.

Validated direction: promote terraform-aws-cli in place as the
monorepo home · cover aws + azure now, extensible to more · execute
after the single-repo modernization (roadmap Phases 1–8) lands.

Why

terraform-aws-cli and terraform-azure-cli are near-identical projects: a
minimalist, multi-stage Debian image bundling the Terraform CLI plus a
single cloud CLI, GPG/checksum-verified, published to Docker Hub. They share
~80% of their Dockerfile, their entire CI/CD topology, their release ritual, and
their maintenance cadence (bump Terraform, bump the cloud CLI, bump the base
image, cut a release). Maintaining two repos means doing every piece of work
twice and letting them drift.

Once the modernization (docs/roadmap.md, Phases 0–8) turns terraform-aws-cli
into a hardened reference (release-please, supply-chain security, agent
framework, ADR discipline), we fold Azure into it and apply that framework
once across both images instead of rebuilding it in Azure.

Goals

• One repository, one CI/CD topology, one governance/agent framework for all
  Terraform + cloud-CLI images.
• No drift: shared logic (Terraform builder stage, security-verification
  chain, base-image policy, workflows, skills) lives in exactly one place.
• Preserve git history of both source repositories.
• Preserve published image names so consumers are not broken:
  bgauduch/terraform-aws-cli and bgauduch/terraform-azure-cli keep shipping
  as Docker Hub repositories, regardless of source layout.
• Independent release cadence per image — an AWS-CLI bump ships an aws
  release without forcing an azure release, and vice versa.
• Adding gcp (or another provider) = a new directory + matrix entry, not a
  structural change.

Non-goals

• Merging the published images into one fat image — each stays single-cloud
  and minimal (the product's whole point).
• A shared version for the images — they version independently.
• Re-doing the modernization work — Phases 1–8 are a prerequisite.
• Bundling extra tools (python3, etc.) — out of scope per docs/roadmap.md.

Target repository layout

terraform-cloud-cli/                 # (repo promoted in place; rename is a late, optional step)
├── images/
│   ├── aws/
│   │   ├── Dockerfile                # AWS-CLI stage + final assembly
│   │   ├── supported_versions.json   # awscli_versions (+ tf range reference)
│   │   ├── security/                 # awscliv2.asc, *.sig
│   │   ├── tests/
│   │   │   └── container-structure-tests.yml.template
│   │   └── README.md                 # Docker Hub description (per image)
│   └── azure/
│       ├── Dockerfile                # Azure-CLI stage + final assembly
│       ├── supported_versions.json   # azurecli_versions (+ tf range reference)
│       ├── security/
│       ├── tests/
│       └── README.md
├── shared/
│   ├── terraform.Dockerfile          # the identical Terraform builder stage, defined once
│   ├── security/
│   │   └── hashicorp.asc             # shared HashiCorp signing key + tf SHA256SUMS/.sig
│   └── supported_terraform.json      # the single Terraform version matrix, shared by all images
├── docker-bake.hcl                   # build targets, version matrix, shared stage wiring
├── .release-please-manifest.json     # per-image versions
├── release-please-config.json        # per-image components (aws, azure), tag prefixes
├── AGENTS.md / CLAUDE.md             # agent SSOT + adapter, provider-parametrized
└── dev.sh                            # local build helper, now provider-aware

Principles

• shared/ holds anything byte-identical across images today: the Terraform
  builder stage, the HashiCorp signing key + Terraform SHA256SUMS, the single
  Terraform version list, the base-image pin policy.
• images/<provider>/ holds only what differs: the cloud-CLI download/verify
  stage, that CLI's version list and signing material, the per-image Docker Hub
  README, and provider-specific structure tests.
• Adding a provider = new images/<provider>/ + one matrix entry. No edits to
  shared logic.

Build system: one Terraform stage, per-provider assembly

Both Dockerfiles today contain an identical terraform builder stage
(download → import hashicorp.asc → verify .sig → sha256sum --check →
unzip). It is extracted to shared/terraform.Dockerfile and consumed by every
image, so the verification chain is defined once.

Proposed orchestration: Docker Bake (docker buildx bake) — a group builds
all images, one target per image, a matrix over the supported version
tuples (Terraform × cloud-CLI), with shared args / cache-from / cache-to /
platforms / tags inherited (this also lands roadmap Phase 4's cache + multi-
arch harmonisation for all images at once). dev.sh becomes provider-aware
(./dev.sh --image aws --tf 1.11.x --cli 2.17.x) and delegates to Bake so local
and CI builds share one definition.

Each image's final stage still produces a single-cloud, minimal image. Bake
is a build-orchestration choice; it does not change the shipped artifacts.

Versioning, releases, registries

Independent versions in one repo — release-please manifest mode:

• release-please-config.json declares two components (images/aws,
  images/azure), each with its own changelog, version, and tag prefix
  (aws-vX.Y.Z, azure-vX.Y.Z);
• .release-please-manifest.json tracks each component's current version;
• Conventional-Commit scopes route changes — feat(aws): … bumps only aws;
  fix(shared): … (or a shared/** touch) fans out to every image, since a
  Terraform-stage or base-image change rebuilds all of them.

Registries & tags — consumer continuity is non-negotiable: published Docker
Hub repos do not change (bgauduch/terraform-aws-cli,
bgauduch/terraform-azure-cli keep receiving images). The GHCR mirror stays
per-image. The full tag strategy (latest, edge, vX.Y.Z, vX.Y, fully-
pinned vX.Y.Z_tf-A.B.C_<cli>-D.E.F) applies independently per image.

The source collapses to one repo; the products stay separate. Map: one repo
→ N components → N Docker Hub repos.

CI/CD topology

One matrixed, path-filtered set of workflows:

• Path filters decide what rebuilds: images/aws/** → AWS only;
  images/azure/** → Azure only; shared/**, docker-bake.hcl, base-image
  policy, or .github/workflows/** → all images.
• Matrix builds — build-test, lint-dockerfile, publish workflows run over
  strategy.matrix.image: [aws, azure], each pulling its own
  supported_versions.json and structure-test template.
• adr-check.yml keeps guarding structural paths — now including shared/**,
  docker-bake.hcl, release-please-config.json.
• Security gates (Trivy, SBOM, SLSA provenance, cosign) run per matrix entry,
  so every image is scanned, attested, and signed independently.

History preservation

• AWS content is moved, not copied: git mv the current top-level files
  into images/aws/ and shared/. History follows the tree move
  (git log --follow).
• Azure merged with full history via git subtree:

  git remote add azure https://github.com/bgauduch/terraform-azure-cli
  git fetch azure
  git subtree add --prefix=images/azure azure/<default-branch>

  Then a follow-up commit lifts Azure's identical Terraform-stage / HashiCorp
  material into shared/ (dedup) and reshapes the rest to the
  images/<provider>/ contract. git-filter-repo is the fallback if subtree's
  merge geometry is awkward.

No history is rewritten on the default branch — every step lands as new commits.

Agent framework & docs reconciliation

• supported_versions.json split — Terraform list → shared/; each image
  keeps only its cloud-CLI list. Skills and validate-supported-versions read
  both.
• Skills become provider-parametrized — bump-terraform-version operates on
  the shared list (fans out to all images); bump-awscli-version /
  bump-azurecli-version are per-image; a new provider adds a bump-<cli>-version
  skill, nothing else.
• AGENTS.md / CLAUDE.md gain a short "images & shared layout" section; hard
  rules, roles/tiers, ADR discipline unchanged.
• ADRs carry over untouched; the consolidation ADR is written when this work
  is picked up.

Deprecating the old repository

• terraform-aws-cli is the monorepo (promoted in place) — keeps stars,
  issues, forks, SEO. A repo rename to terraform-cloud-cli is an optional,
  late step (GitHub auto-redirects; image names are decoupled).
• terraform-azure-cli is archived after the merge: README → pointer to the
  monorepo, open issues/PRs triaged into the monorepo tracker, repo set
  read-only. Its Docker Hub repo keeps being published from the monorepo, so
  Azure consumers are unaffected.

Execution — sub-steps (one reviewable PR each)

Delivered after the Phases 1–8 modernization; phase number assigned when
scheduled.

Step	Title	Outcome
a	Reshape AWS in place	git mv current files into images/aws/ + shared/; introduce shared/terraform.Dockerfile; AWS still builds & publishes byte-identically. No Azure yet.
b	Bake + matrixed CI	Add docker-bake.hcl; convert the six workflows to a single-entry matrix (image: [aws]) with path filters. Pure topology change, AWS-only.
c	Merge Azure (history-preserving)	git subtree add Azure into images/azure/; dedupe its Terraform/HashiCorp material into shared/; conform to the contract.
d	Two-image matrix	Extend CI matrix + Bake group to [aws, azure]; per-image structure tests, security gates, Docker Hub READMEs green.
e	Per-image releases	release-please manifest mode with aws/azure components, tag prefixes, scope→component routing; first independent release of each.
f	Archive & document	Archive terraform-azure-cli with a redirect README; triage its issues; migration notes; (optional) rename repo → terraform-cloud-cli.

Each step is independently shippable and leaves the default branch releasable.

Risks & mitigations

Risk	Mitigation
Breaking consumers' docker pull	Published image names preserved; only source layout changes. Verify publish targets in step b before adding Azure.
Azure history lost in the merge	git subtree add (full history), git-filter-repo fallback; verify git log --follow before archiving the source.
shared/ change silently breaks one image	Path filter routes shared/** to all images; matrix build + structure tests + Trivy must be green for every image before merge.
Version-matrix explosion (TF × CLI)	Bake matrix limited to supported tuples from the JSON files; not a full cross-product.
Wrong component released	release-please dry-run in CI on the PR; Conventional-Commit scope mandatory + lint-enforced.
Doing it on a moving base	Sequenced after Phases 1–8 — the reference repo is hardened first.
Repo rename surprises	Rename is the last, optional step; GitHub redirects the old slug; image names decoupled.

Open decisions (at kickoff)

1. Final repo name — keep terraform-aws-cli or rename to
   terraform-cloud-cli / terraform-cli-images? (Recommend rename in the last
   step, relying on GitHub redirects.)
2. Build orchestrator — Docker Bake (recommended) vs. a workflow matrix
   calling plain docker buildx build.
3. GCP timing — design now, add images/gcp/ only when a real need exists.
4. Issue migration — bulk transfer vs. manual triage for
   terraform-azure-cli's open issues.

---

The ADR ratifying this consolidation, its roadmap phase number, and the
docs/-committed plan are produced when this issue is scheduled/handled — not
before. This issue is the working draft until then.