Skip to content

Make the Makefile do what CI does, and name the scenarios - #52

Merged
sethbergman merged 3 commits into
mainfrom
chore/makefile-congruent-with-ci
Aug 29, 2026
Merged

Make the Makefile do what CI does, and name the scenarios#52
sethbergman merged 3 commits into
mainfrom
chore/makefile-congruent-with-ci

Conversation

@sethbergman

Copy link
Copy Markdown
Owner

The problem

make lint could pass while CI failed — the worst state for a
convenience target, because it teaches people to distrust it and push
instead. Three divergences:

Makefile did CI does
ansible ansible-lint ansible/ || true — failures swallowed fails the build, plus --syntax-check
shellcheck scripts/*.sh only also every test harness and fake-bin shim, plus the executable-bit invariant
markdownlint, shell invariants, docs-index generator no target at all three separate jobs

Also: every target already carried a ## annotation and nothing rendered
them. make now prints a grouped list and is the default goal.

Suites are discovered, not listed

ALL_SUITES comes from tests/*/run-tests.sh. The workflow has to
enumerate its jobs; this file does not, and make check-ci-coverage
turns "a suite exists that no CI job runs" — the gap CLAUDE.md warns
about — into a failure rather than an omission. It currently reports all
17 suites wired.

make test changed meaning

It ran the DR restore drill alone. It now runs every suite that needs no
cluster, and the drill is make dr-drill.

CLAUDE.md called the old narrowness deliberate, so this is a real
deviation rather than a tidy-up, and CLAUDE.md now records it. The
reasoning: a test target that runs one drill is surprising in the one
way a test target must not be — somebody who ran it and saw green had
tested almost nothing.

Operational scenarios get targets

deploy-min, deploy-full, status (now showing each node's seal and
HA state rather than only node 0), peers, logs, snapshot,
verify-audit, rotate-secret-id ROLE=, test-suite SUITE=, and the
three cloud checks that cost nothing: tf-test, preflight-static,
emulated-apply.

Verified without make

make is not installed on the authoring machine, so rather than assume:

  • the help awk was run directly against the Makefile and renders all
    33 targets in their groups
  • suite discovery, check-ci-coverage and check-exec were each run as
    standalone shell — 17 suites found, all wired, all scripts 100755
  • checked for the classic fault: 85 recipe lines, all tab-indented, no
    space-indented recipe

lint-markdown says out loud that CI pins the action's version while
npx resolves whatever is current, so a local pass is evidence and not a
guarantee.

🤖 Generated with Claude Code

sethbergman and others added 3 commits August 29, 2026 15:35
`make lint` could pass while CI failed, which is the worst state for a
convenience target to be in: it teaches people to distrust it and push
instead. Three reasons it diverged, all fixed:

  - `ansible-lint ansible/ || true` swallowed every failure
  - shellcheck ran over scripts/*.sh; CI also lints every test harness and
    every fake-bin shim, and asserts the executable bit
  - markdownlint, the shell-invariants suite and the docs-index generator
    check had no target at all

The `##` annotations were already on every target and nothing rendered
them, so `make` now prints a grouped list and is the default goal.

Suites are discovered from tests/*/run-tests.sh rather than enumerated.
The workflow has to list its jobs, but this file does not, and
check-ci-coverage turns "a suite exists that no CI job runs" -- the gap
CLAUDE.md warns about -- into a failure rather than an omission.

`make test` changed meaning, deliberately. It ran the DR restore drill
alone; it now runs every suite that needs no cluster, and the drill is
`make dr-drill`. The old behaviour was surprising in the one way a `test`
target should not be: somebody who ran it and saw green had tested almost
nothing. CLAUDE.md records the change.

Operational scenarios get targets rather than remembered flags:
deploy-min, deploy-full, status showing each node's seal and HA state,
peers, snapshot, verify-audit, rotate-secret-id, and the three cloud
checks that cost nothing -- tf-test, preflight-static, emulated-apply.

lint-markdown says out loud that CI pins the action's version while npx
resolves whatever is current, so a local pass is evidence rather than a
guarantee.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Running `make deploy-full` on a machine without the Compose V2 plugin
produced this:

  unknown flag: --project-directory
  Usage:  docker [OPTIONS] COMMAND [ARG...]

Compose V2 is a plugin to the docker CLI, so `docker` can exist and work
while `docker compose` does not resolve at all. When that happens docker
cannot match the subcommand, falls back to parsing the remainder as
global flags, and reports the first one it does not recognise -- naming
neither compose nor the actual problem.

The script had already generated TLS material by then. Its header claimed
"Requirements: docker compose, curl, jq" and it checked none of them,
which is the convention every other script here follows and this one
did not.

Now checked first: docker, curl and jq on PATH, `docker compose version`
as a subcommand rather than by looking for a docker-compose binary, and a
reachable daemon. The compose failure prints what to do about it on both
Docker Desktop with WSL integration and a native install.

Verified by running it with docker absent, and against a stub docker
whose `compose` subcommand fails the way the real one did. Neither run
creates anything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The lock files carried h1: hashes for three platforms, all of them
amd64-shaped, because that is what had ever been developed or built on.

The registry zh: hashes cover every published platform, so `init` works
anywhere regardless -- but it then appends an h1: for whatever platform
you are on and leaves the lock file modified in `git status`. That reads
as "I broke something" to whoever hits it, and the person most likely to
hit it is on an architecture nobody here has used before. CI runs
linux_amd64 only, so it can never surface there.

Both files now cover linux_amd64, linux_arm64, darwin_amd64, darwin_arm64
and windows_amd64.

Not windows_arm64: neither hashicorp/aws nor hashicorp/random publishes a
build for it, and `providers lock` is all-or-nothing -- naming one
unavailable platform fails the whole command and writes nothing. That is
also why a Windows-on-ARM machine runs Terraform inside WSL and locks
linux_arm64 rather than a native Windows target.

CLAUDE.md gains the command and the reasoning, including that adding a
platform to the list is a claim the registry has to agree with.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sethbergman

Copy link
Copy Markdown
Owner Author

Two commits landed after the description above was written

The PR body covers the Makefile only. These two came out of actually
running it on a machine unlike CI's, and neither was caused by the
Makefile — it just made someone walk a path where they already lived.

7d09ea2 — the bootstrap script checks its tools first

make deploy-full on a machine without the Compose V2 plugin produced:

unknown flag: --project-directory
Usage:  docker [OPTIONS] COMMAND [ARG...]

Compose V2 is a plugin to the docker CLI, so docker can work perfectly
while docker compose does not resolve at all. When that happens docker
cannot match the subcommand, falls back to parsing the remainder as
global flags, and reports the first one it does not recognise — naming
neither compose nor the actual problem.

The script had already generated TLS material by then. Its header claimed
"Requirements: docker compose, curl, jq" and it checked none of them,
against a convention CLAUDE.md states explicitly:

a command -v check for each external tool

Now checked before anything is created: docker, curl, jq on PATH,
docker compose version as a subcommand rather than by looking for a
docker-compose binary, and a reachable daemon.

Verified against a stub docker whose compose subcommand fails the way
the real one did, and with docker absent entirely. Neither run creates
anything.

4d76a23 — providers locked for five platforms

The lock files carried h1: hashes for three platforms, all amd64-shaped,
because that is what had ever been built on.

The registry zh: hashes cover every published platform, so init works
anywhere regardless — it then appends an h1: for your platform and
leaves the lock file dirty in git status. That reads as "I broke
something"
, and the person most likely to hit it is on an architecture
nobody here has used before.

CI runs linux_amd64 only, so this can never surface there. It can
only ever fail for a person, on their machine.

Both files now cover linux_amd64, linux_arm64, darwin_amd64,
darwin_arm64, windows_amd64 — five h1: per provider, all four
providers.

Not windows_arm64: neither hashicorp/aws nor hashicorp/random
publishes a build for it, and providers lock is all-or-nothing, so
naming one unavailable platform fails the whole command and writes
nothing. I got that wrong first time round and the registry corrected me.
It is also why a Windows-on-ARM machine runs Terraform inside WSL and
locks linux_arm64 rather than a native Windows target.

CLAUDE.md carries the command and the reasoning, including the part worth
keeping: adding a platform to that list is a claim the registry has to
agree with.


27/27 green, MERGEABLE / CLEAN. The cluster jobs — smoke test, DR
drill, both OIDC logins, integration — all run through the new preflight,
so a check that was too strict would have reddened seven jobs at once.

@sethbergman
sethbergman merged commit 3286360 into main Aug 29, 2026
27 checks passed
@sethbergman
sethbergman deleted the chore/makefile-congruent-with-ci branch August 29, 2026 23:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant