From 3b3cea0f0c6aa6ed8132e048bb62d1b9b78d3eb1 Mon Sep 17 00:00:00 2001 From: refinery costing Date: Mon, 24 Aug 2026 04:55:24 +0000 Subject: [PATCH 1/2] fix(scripts): raise both native-dependency ceilings the beads v1.2.1 bump breached (gc-na313) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `Preflight / static checks` has been red on main since 255d35ae0 (beads pin -> v1.2.1, #138): native dependency guard: module graph has 740 modules; max is 727 Attribution is exact: the graph was 727 at 255d35ae0^ — sitting precisely on the ceiling — and 740 at the bump. Nothing since has moved it. All 13 new module paths are graph-only artifacts, not new native surface: none appears in our go.mod, `go mod why -m` reports "main module does not need module X" for all 13, and each has zero reach in `go list -deps ./cmd/gc`. Eleven are requirements declared in beads' own go.mod (the oapi-codegen OpenAPI generator toolchain); `cloud.google.com/go/pubsub/v2` and `github.com/zeebo/errs` ride in on version bumps of Google modules already present. They are in the build list only because Go's module-graph pruning admits every requirement declared in a directly-required module's go.mod, including its build tooling — so there is nothing here for us to trim short of `exclude` directives that fight upstream. Raising the ceiling is the correct call, and it follows upstream's own precedent: the beads v1.0.4 -> v1.1.0 bump raised it 725 -> 727 in the same commit. The bead reported the module ceiling as the only breach. It is not — that was measured without running the guard to completion. The script is a straight-line `set -e` sequence, so the binary is never built until the module check passes, and a second breach was hidden behind the first: native dependency guard: gc binary is 270733176 bytes; max is 270000000 That one is also the bump's doing. Measured: the same pre-bump commit built 267,809,008 bytes on CI (go 1.26.5) and 268,260,480 locally (go 1.26.6), so the environment spread is 0.17%; the bump adds 2,472,696 bytes (0.92%), putting CI at a projected ~270,281,704 — over the cap. Raising `max_modules` alone would have handed back a differently-red board. The two ceilings get different treatment on purpose. Module count is deterministic, so it stays an exact ratchet at 740. Binary size is not reproducible across toolchains, so an exact cap would go red on a Go patch bump alone; it becomes a round 280,000,000, a few percent above today's 270.7M. Both now carry a comment saying which kind of number they are. Also lands engdocs/contributors/beads-version-bump-anchors.md. The beads-bump anchor list has now been lost twice — first to a session scratchpad under /tmp (reaped), then into the notes of a closed bead (gc-lbfmj) — and both of the traps recorded there fail somewhere that never mentions beads. This puts it in the repo with the two ceilings added as traps 3 and 4, including the note that the binary cap eroded from 14% headroom (2026-05-31) to 0.8% (2026-08-14) without anyone noticing. Validation: `bash scripts/check-native-dependency-surface.sh` now runs to completion — modules=740 aws=25 azure=9 dolthub=15 googleapi=1 binary_bytes=270733176, exit 0. `make check-docs` passes (docsync link-validates engdocs/). No Go source is touched. --- .../beads-version-bump-anchors.md | 136 ++++++++++++++++++ engdocs/contributors/index.md | 3 + scripts/check-native-dependency-surface.sh | 15 +- 3 files changed, 152 insertions(+), 2 deletions(-) create mode 100644 engdocs/contributors/beads-version-bump-anchors.md diff --git a/engdocs/contributors/beads-version-bump-anchors.md b/engdocs/contributors/beads-version-bump-anchors.md new file mode 100644 index 0000000000..e2112351cb --- /dev/null +++ b/engdocs/contributors/beads-version-bump-anchors.md @@ -0,0 +1,136 @@ +# Beads Version Bump Anchors + +Bumping the `github.com/steveyegge/beads` pin is a coordinated multi-file +edit, not a `go get`. The version appears in go.mod, in `deps.env`, in +Dockerfiles, in a dozen workflow assignments, in checksum pins, and in two +guards that fail only *after* the obvious edits look complete. Miss one and +the board goes red in a place that does not name beads. + +This page is the anchor list. It exists because the list has twice been lost: +first to a session scratchpad under `/tmp` (reaped), then to the notes of a +closed bead (`gc-lbfmj`). Add anchors here as they are discovered. + +## Anchors + +### A — library / current cell + +| File | Field | +|---|---| +| `deps.env` | `BD_CURRENT_VERSION` (pseudo-version) | +| `deps.env` | `BD_CURRENT_REF` (full 40-char commit SHA) | +| `go.mod` + `go.sum` | the `require` entry | +| `contrib/k8s/Dockerfile.agent` | `ARG BD_SOURCE_REF` (full SHA) | +| `contrib/k8s/Dockerfile.agent` | `ARG BD_BUILD` (`BD_SOURCE_REF[:10]`) | + +### B — installable default + +| File | Field | +|---|---| +| `deps.env` | `BD_VERSION` | +| `contrib/k8s/Dockerfile.agent` | `ARG BD_VERSION` | +| `.github/scripts/install-bd-archive.sh` | per-platform archive SHA pins | +| `.github/workflows/*` | every `BD_VERSION:` assignment | +| `.devcontainer/README.md` | the quoted current version | + +`BD_PREV_VERSION` is the compatibility-matrix floor and does **not** move +with a bump. Neither do `bdMinVersion`, `bdReadyProjectionMinVersion`, or +the `bd_compatibility` enum — floors are separate decisions. + +## The traps + +These are the anchors that are not covered by `TestBDVersionPins`, which +compares version *strings* only. Each fails somewhere that does not mention +beads. + +### 1. Archive checksum — `ARG BD_SOURCE_SHA256` + +`contrib/k8s/Dockerfile.agent` verifies the source archive with +`sha256sum --check --strict`. The pin test never looks at it, so a stale +checksum passes every Go test and fails the image build. Recompute it +against the archive at the new ref. + +### 2. gRPC version — `ARG GRPC_VERSION` + +The Dockerfile pins a gRPC version with a `go get` and then asserts the +result equals that pin. When beads at the new ref already ships a *newer* +gRPC, leaving the old pin makes the `go get` a **downgrade** — inverting the +block's purpose — and the assertion still passes, because it checks for +exactly the pinned version. The build goes green while shipping the older, +possibly vulnerable, library. + +Set `GRPC_VERSION` to the version beads already ships. Do not delete the +block: the `go get` becomes a no-op and the assertion keeps guarding future +regressions. + +### 3. Native dependency module ceiling — `max_modules` + +`scripts/check-native-dependency-surface.sh` caps the total size of the +module graph at an exact count. A beads bump routinely grows that graph, and +the guard is not part of the unit-test suite — it fails in CI under +**Preflight / static checks**, with a message that names no dependency: + + native dependency guard: module graph has 740 modules; max is 727 + +Raise `max_modules` to the new exact count **in the bump commit**. The +precedent is upstream's own bumps: v1.0.4 → v1.1.0 raised the ceiling +725 → 727 in the same commit. + +Before raising it, confirm the growth is what you think it is. The ceiling +guards *native dependency surface*, so the question is whether the new +modules are real surface or graph noise: + +```bash +# Which module paths are new, against the pre-bump commit +git worktree add /var/tmp/pre --detach ^ +(cd /var/tmp/pre && go list -m all) | awk '{print $1}' | LC_ALL=C sort -u > /var/tmp/pre.paths +go list -m all | awk '{print $1}' | LC_ALL=C sort -u > /var/tmp/post.paths +LC_ALL=C comm -13 /var/tmp/pre.paths /var/tmp/post.paths + +# For each new path: does anything we build actually import it? +go mod why -m # "main module does not need" => graph-only +go list -deps ./cmd/gc | grep -c '^' # 0 => links nothing into gc +``` + +A module that is absent from our `go.mod`, reports *"main module does not +need module X"*, and has zero reach in `./cmd/gc` adds no native dependency +surface. It is in the graph only because Go's module-graph pruning admits +every requirement declared in a directly-required module's own `go.mod` — +including that dependency's build tooling. We have no lever to remove those +short of `exclude` directives that fight upstream; raising the ceiling is +correct, and the binary-size cap remains the check that measures real +growth. + +If instead a new module *is* reachable, treat it as genuine growth and +decide whether to accept or avoid it before touching the number. + +### 4. Native dependency binary ceiling — `max_binary_bytes` + +The same script also caps the size of the built `gc` binary, and a beads +bump grows that too — the v1.2.1 bump cost about 2.5 MB. + +**This one hides behind the module check.** The script is a straight-line +`set -e` sequence: the module comparison runs first and the binary is not +even built until it passes. So a bump that breaches both ceilings reports +only the module one, and raising `max_modules` alone turns a red board into +a *differently* red board. After changing `max_modules`, always run the +guard to completion before believing it is fixed. + +Unlike the module count, binary size is **not** reproducible across +environments — the same commit measured 267,809,008 bytes on CI (go 1.26.5) +and 268,260,480 locally (go 1.26.6), a 0.17% spread. Pinning it to the exact +current size would go red on a Go patch bump alone, so this ceiling is +deliberately a round number with a few percent of headroom, and it is the +one number here you should *not* ratchet tight. + +Watch the headroom rather than just the pass/fail: the cap sat at +270,000,000 from 2026-05-31, when the binary was 236,197,920 (14% headroom), +and by 2026-08-14 the binary had reached 267,809,008 (0.8%) without anyone +noticing. A guard that is one dependency bump from tripping is not +protecting anything. If a bump leaves under a couple of percent of room, +raise the ceiling *and* say so in the PR, so the growth trend gets looked at +rather than absorbed a second time. + +## After the bump + +Land the bump as its own commit with its own full gate run, including +`make check-native-dependency-surface`. diff --git a/engdocs/contributors/index.md b/engdocs/contributors/index.md index ad55457e73..b5eb0897dd 100644 --- a/engdocs/contributors/index.md +++ b/engdocs/contributors/index.md @@ -21,6 +21,9 @@ description: The shortest path for new contributors to get productive in Gas Cit - [Release Gate Criteria Conventions](release-gate-criteria-conventions.md) when signing off the "Tests pass" criterion on a `release-gates/*.md` deploy gate — it must cite the CI jobs `ci-required` actually gates on +- [Beads Version Bump Anchors](beads-version-bump-anchors.md) when moving the + `github.com/steveyegge/beads` pin — the version lives in a dozen files, and + three of them fail in places that never mention beads - [`CONTRIBUTING.md`](https://github.com/gastownhall/gascity/blob/main/CONTRIBUTING.md) - [`TESTING.md`](https://github.com/gastownhall/gascity/blob/main/TESTING.md) diff --git a/scripts/check-native-dependency-surface.sh b/scripts/check-native-dependency-surface.sh index 00be1345a0..d28402d622 100644 --- a/scripts/check-native-dependency-surface.sh +++ b/scripts/check-native-dependency-surface.sh @@ -1,8 +1,19 @@ #!/usr/bin/env bash set -euo pipefail -max_modules="${GC_NATIVE_DEP_MAX_MODULES:-727}" -max_binary_bytes="${GC_NATIVE_DEP_MAX_BINARY_BYTES:-270000000}" +# Exact ratchet, not headroom: this is the module count of the graph today, so +# any growth trips the guard and gets reviewed rather than absorbed silently. +# Raise it in the same commit that causes the growth -- a beads pin bump is the +# usual cause, and the failure message reports the new count. Full anchor list: +# engdocs/contributors/beads-version-bump-anchors.md. +max_modules="${GC_NATIVE_DEP_MAX_MODULES:-740}" +# Headroom, not a ratchet: unlike the module count, this number is not +# reproducible across environments -- the same commit built 267,809,008 bytes on +# CI (go 1.26.5) and 268,260,480 locally (go 1.26.6), so an exact cap would go +# red on a toolchain bump alone. Keep it a round number a few percent above the +# real size: big enough to absorb that spread, small enough that sustained +# growth still has to be reviewed. Today's build is ~270.7M. +max_binary_bytes="${GC_NATIVE_DEP_MAX_BINARY_BYTES:-280000000}" max_aws_modules="${GC_NATIVE_DEP_MAX_AWS_MODULES:-25}" max_azure_modules="${GC_NATIVE_DEP_MAX_AZURE_MODULES:-9}" max_dolthub_modules="${GC_NATIVE_DEP_MAX_DOLTHUB_MODULES:-15}" From 22eb082ced96a5c49884a7f2e670cc3c56c45bc4 Mon Sep 17 00:00:00 2001 From: refinery costing Date: Mon, 24 Aug 2026 04:59:06 +0000 Subject: [PATCH 2/2] docs: correct the anchor-doc trap count and tighten its recipes (gc-na313) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three precision fixes to the page added in the previous commit, all found re-reading it end to end: - The index entry said three anchors fail somewhere that never mentions beads. There are four — the binary ceiling was added to the page after that bullet was written. - The intro claimed the version "appears in ... two guards". It does not: the guards measure what the new pin drags in, they do not name it. - The reach check used `grep -c '^'`, which matches any module whose path merely starts with that prefix. Anchored to `(/|$)`, so a sibling module cannot report a false non-zero reach and talk someone out of a ceiling raise that was actually correct. The worktree recipe now removes the worktree it creates, and says why it compares paths and not raw `go list -m all` lines (every version bump reads as add+remove otherwise). `make check-docs` passes. --- engdocs/contributors/beads-version-bump-anchors.md | 12 +++++++----- engdocs/contributors/index.md | 2 +- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/engdocs/contributors/beads-version-bump-anchors.md b/engdocs/contributors/beads-version-bump-anchors.md index e2112351cb..8364981385 100644 --- a/engdocs/contributors/beads-version-bump-anchors.md +++ b/engdocs/contributors/beads-version-bump-anchors.md @@ -2,8 +2,8 @@ Bumping the `github.com/steveyegge/beads` pin is a coordinated multi-file edit, not a `go get`. The version appears in go.mod, in `deps.env`, in -Dockerfiles, in a dozen workflow assignments, in checksum pins, and in two -guards that fail only *after* the obvious edits look complete. Miss one and +Dockerfiles, in a dozen workflow assignments, and in checksum pins — and two +size guards then fail on what the new pin drags in behind it. Miss one and the board goes red in a place that does not name beads. This page is the anchor list. It exists because the list has twice been lost: @@ -80,15 +80,17 @@ guards *native dependency surface*, so the question is whether the new modules are real surface or graph noise: ```bash -# Which module paths are new, against the pre-bump commit +# Which module paths are new, against the pre-bump commit. Compare paths, not +# the raw lines: every version bump would otherwise read as add+remove. git worktree add /var/tmp/pre --detach ^ (cd /var/tmp/pre && go list -m all) | awk '{print $1}' | LC_ALL=C sort -u > /var/tmp/pre.paths go list -m all | awk '{print $1}' | LC_ALL=C sort -u > /var/tmp/post.paths LC_ALL=C comm -13 /var/tmp/pre.paths /var/tmp/post.paths +git worktree remove --force /var/tmp/pre # For each new path: does anything we build actually import it? -go mod why -m # "main module does not need" => graph-only -go list -deps ./cmd/gc | grep -c '^' # 0 => links nothing into gc +go mod why -m # "main module does not need" => graph-only +go list -deps ./cmd/gc | grep -cE '^(/|$)' # 0 => links nothing into gc ``` A module that is absent from our `go.mod`, reports *"main module does not diff --git a/engdocs/contributors/index.md b/engdocs/contributors/index.md index b5eb0897dd..cfefe18838 100644 --- a/engdocs/contributors/index.md +++ b/engdocs/contributors/index.md @@ -23,7 +23,7 @@ description: The shortest path for new contributors to get productive in Gas Cit deploy gate — it must cite the CI jobs `ci-required` actually gates on - [Beads Version Bump Anchors](beads-version-bump-anchors.md) when moving the `github.com/steveyegge/beads` pin — the version lives in a dozen files, and - three of them fail in places that never mention beads + four of them fail in places that never mention beads - [`CONTRIBUTING.md`](https://github.com/gastownhall/gascity/blob/main/CONTRIBUTING.md) - [`TESTING.md`](https://github.com/gastownhall/gascity/blob/main/TESTING.md)