From 59a781546536314abc6107ae8a2bbb414ea899ea Mon Sep 17 00:00:00 2001 From: "Vitaly D." Date: Fri, 31 Jul 2026 17:13:32 +0300 Subject: [PATCH 1/2] Derive the command reference from the binary that ships MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Comparing what is written with what is true cannot notice a command nobody wrote about, and the product is adding them. This adds the half that can. One release is described at a time and product/pin.json names it. Bumping the tag and running `npm run sync:product` downloads that release, verifies it against its published checksums, asks the binary what it accepts, and rewrites the marked blocks in the commands page from the answer. Prose outside the markers is never touched: the reference is derived, the explanation stays written. The diff of a pin bump is the product's changelog as far as this site is concerned, which makes it the list of prose worth re-reading. Every command the binary accepts must now have a section on that page or an entry in the pin saying why not, and an entry that outlives its command fails too. A command added to the product cannot pass through here unnoticed; leaving one out stays allowed, leaving one out silently does not. Deriving it immediately paid for itself. The hand-written flag table paraphrased all four of its descriptions — "override detection" for "codex or claude-code; omit to detect", among others — and documented one command's flags where the binary has five commands that take them. The page now carries what the binary prints, with one normalization: Go writes `-repo` and every other page here, including the product's own install prompt, writes `--repo`. Two things went differently than planned. The first run reported that every command takes no flags, because Go's flag package writes usage to standard error and reading stdout on success returns nothing — silently, which is the failure this repository exists to be unhappy about. And `--check` was going to run only on the daily schedule; the pull request that bumps the pin is the one place it matters most, so it runs in CI as well. --- .github/workflows/ci.yml | 5 + .github/workflows/drift.yml | 13 +- README.md | 36 ++++-- package.json | 3 +- product/pin.json | 24 ++++ product/surface.json | 172 +++++++++++++++++++++++++ public/docs/commands.md | 41 +++++- public/llms-full.txt | 41 +++++- scripts/check-claims.mjs | 97 ++++++++++++-- scripts/product-reference.mjs | 68 ++++++++++ scripts/sync-product.mjs | 232 ++++++++++++++++++++++++++++++++++ 11 files changed, 703 insertions(+), 29 deletions(-) create mode 100644 product/pin.json create mode 100644 product/surface.json create mode 100644 scripts/product-reference.mjs create mode 100644 scripts/sync-product.mjs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 75ca3f2..d99355b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -33,6 +33,11 @@ jobs: # Lifts the API rate limit off the runner's shared address. GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: npm run check:claims + # Downloads the pinned release and asks it what it accepts. The pull + # request that bumps the pin is the one where this matters most, which is + # why it runs here and not only on the daily schedule. + - name: Check the reference against the pinned release + run: npm run sync:product -- --check # `npm run build` runs `tsc -b` before Vite, so this is the typecheck too. - run: npm run build # The deployed artifact is the image, not the directory, so the image is diff --git a/.github/workflows/drift.yml b/.github/workflows/drift.yml index 6187442..ec40f7d 100644 --- a/.github/workflows/drift.yml +++ b/.github/workflows/drift.yml @@ -39,7 +39,7 @@ jobs: cache: npm - run: npm ci --no-audit --no-fund - - name: Check the claims that came from the product + - name: Check the claims and the reference against the product shell: bash env: # Lifts the API rate limit off the runner's shared address. @@ -47,8 +47,15 @@ jobs: run: | # Without pipefail the exit status would be tee's, which is always # zero — the failure this job exists to report would be invisible. - set -euo pipefail - npm run check:claims 2>&1 | tee "${RUNNER_TEMP}/claims.log" + set -uo pipefail + + # Both run even when the first fails: the issue this opens should + # carry everything that is wrong, not the first thing. + status=0 + npm run check:claims 2>&1 | tee "${RUNNER_TEMP}/claims.log" || status=1 + npm run sync:product -- --check 2>&1 \ + | tee -a "${RUNNER_TEMP}/claims.log" || status=1 + exit "${status}" # A red run in a repository nobody is watching is not a notification. # An issue is, and it survives until someone closes it. diff --git a/README.md b/README.md index e11825d..d73c184 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ npm run dev ```sh npm run lint # oxlint npm run check:claims # what this page says about the product, against the product +npm run sync:product # rewrite the generated reference from the pinned release npm run build # verify:wallets, the llms.txt generator, tsc -b, vite build ``` @@ -21,13 +22,31 @@ The build writes `public/llms.txt` and `public/llms-full.txt` from the pages in a second step. It also refuses to run if a sponsorship address fails its own checksum. -`check:claims` is the one that exists because of what this repository is. It -describes a product it does not contain, so its failure mode is drift rather -than a broken build: the install prompt is compared with the product's README, -and the `gr doctor` capture with the newest published release, so a release is -what forces a re-capture rather than someone noticing. It needs the network and -fails loudly without it — a check that passes when it did not run is the promise -it was meant to replace. +## Keeping up with the product + +This repository describes a product it does not contain, so its failure mode is +drift rather than a broken build. Three things hold against it. + +**One release is described at a time.** `product/pin.json` names it. Bumping the +tag and running `npm run sync:product` is what pulls this site forward: the +script downloads that release, verifies it against its published checksums, asks +the binary what it accepts, and rewrites the `` blocks in +`public/docs/commands.md` from the answer. Prose around those blocks is written +by hand and never touched — the reference is derived, the explanation is not. +The diff of a pin bump is the product's changelog as far as this site is +concerned, and therefore the list of prose worth re-reading. + +**Nothing new arrives unmentioned.** Every command the binary accepts must have +a section on the commands page or an entry in `product/pin.json` saying why not. +A command added to the product cannot pass through here unnoticed; leaving one +out stays allowed, leaving one out silently does not. + +**A clock, not a pull request.** `check:claims` compares the install prompt with +the product's README and the pin with the newest published release, and +`.github/workflows/drift.yml` runs both daily, because drift here is caused by +the product moving rather than by anything happening in this repository. A +failure opens an issue. Both need the network and fail loudly without it — a +check that passes when it did not run is the promise it was meant to replace. ## Shape @@ -36,7 +55,8 @@ it was meant to replace. | `src/content.ts` | every string the landing page shows | | `src/theme/` | the Dracula/Alucard theme and the typefaces | | `public/docs/*.md` | the documentation source, served raw and rendered by `src/Docs.tsx` | -| `scripts/` | the checks that run before a build: the claims check and the wallet verifier | +| `product/` | which release this site describes, and the surface generated from it | +| `scripts/` | the checks and the generator: claims, wallets, and the product reference | | `nginx.conf` | how the routes are served, including the client-route fallback | | `Dockerfile` | the deployed artifact: nginx with `dist/` inside it | diff --git a/package.json b/package.json index f46b4d4..f11d3a9 100644 --- a/package.json +++ b/package.json @@ -10,7 +10,8 @@ "preview": "vite preview", "llms": "node scripts/build-llms.mjs", "verify:wallets": "node --experimental-strip-types scripts/verify-wallets.mjs", - "check:claims": "node --experimental-strip-types scripts/check-claims.mjs" + "check:claims": "node --experimental-strip-types scripts/check-claims.mjs", + "sync:product": "node scripts/sync-product.mjs" }, "dependencies": { "@astryxdesign/cli": "^0.1.9", diff --git a/product/pin.json b/product/pin.json new file mode 100644 index 0000000..b80048f --- /dev/null +++ b/product/pin.json @@ -0,0 +1,24 @@ +{ + "$comment": [ + "Which release of the product this site describes. Hand-maintained.", + "", + "Bumping this is the act that pulls the site forward: `npm run sync:product`", + "downloads that release, asks the binary what it accepts, and rewrites the", + "generated blocks from the answer. The diff of that bump is the list of what", + "changed in the product, which is also the list of prose worth re-reading.", + "", + "`undocumented` is the other half. Every command the binary accepts must", + "either have a section in public/docs/commands.md or appear here with a", + "reason, and the check fails otherwise — so a command added to the product", + "cannot pass through this repository unnoticed. Leaving one out is allowed;", + "leaving one out silently is not." + ], + "tag": "v0.1.2", + "undocumented": { + "prepare": "Wrapper lifecycle, for benchmark and owner-driven runs; named as a group under 'What this page leaves out'.", + "inspect": "Wrapper lifecycle, as above.", + "start": "Wrapper lifecycle, as above.", + "finish": "Wrapper lifecycle, as above.", + "health": "The superseded name of `gr doctor`, which the doctor section already names." + } +} diff --git a/product/surface.json b/product/surface.json new file mode 100644 index 0000000..09ecbd7 --- /dev/null +++ b/product/surface.json @@ -0,0 +1,172 @@ +{ + "$comment": "Generated by scripts/sync-product.mjs from the release named in product/pin.json. Do not edit: run `npm run sync:product`.", + "tag": "v0.1.2", + "commands": [ + { + "name": "init", + "flags": [ + { + "flag": "--confirm-schema-switch", + "effect": "switch an OpenSpec configuration that names another custom schema" + }, + { + "flag": "--fix-gitignore", + "effect": "add the ignore entries the registration and the marker need" + }, + { + "flag": "--repo ", + "effect": "repository to initialize (default \".\")" + }, + { + "flag": "--scaffold ", + "effect": "codex or claude-code; omit to detect" + } + ] + }, + { + "name": "doctor", + "flags": [ + { + "flag": "--json", + "effect": "emit the diagnosis as JSON" + }, + { + "flag": "--repo ", + "effect": "repository to diagnose (default \".\")" + }, + { + "flag": "--scaffold ", + "effect": "codex or claude-code; omit to report every supported scaffold" + }, + { + "flag": "--state-dir ", + "effect": "Goalrail local state directory" + } + ] + }, + { + "name": "update", + "flags": [ + { + "flag": "--discard-local-edits", + "effect": "replace overlay files that differ from the canon, discarding local edits" + }, + { + "flag": "--repo ", + "effect": "repository whose harness to update (default \".\")" + }, + { + "flag": "--state-dir ", + "effect": "Goalrail local state directory" + } + ] + }, + { + "name": "version", + "flags": [] + }, + { + "name": "connect", + "flags": [ + { + "flag": "--scaffold ", + "effect": "codex or claude-code" + }, + { + "flag": "--yes", + "effect": "consent to modifying the scaffold configuration" + } + ] + }, + { + "name": "disconnect", + "flags": [ + { + "flag": "--repo ", + "effect": "repository whose registration to remove (default \".\")" + }, + { + "flag": "--scaffold ", + "effect": "codex or claude-code" + } + ] + }, + { + "name": "health", + "flags": [ + { + "flag": "--repo ", + "effect": "repository to check (default \".\")" + }, + { + "flag": "--scaffold ", + "effect": "codex or claude-code; omit to check all supported scaffolds" + } + ] + }, + { + "name": "prepare", + "flags": [ + { + "flag": "--file ", + "effect": "path to one WorkSpec JSON file" + }, + { + "flag": "--state-dir ", + "effect": "Goalrail local state directory" + } + ] + }, + { + "name": "inspect", + "flags": [ + { + "flag": "--digest ", + "effect": "prepared WorkSpec digest" + }, + { + "flag": "--run ", + "effect": "generated run ID" + }, + { + "flag": "--state-dir ", + "effect": "Goalrail local state directory" + } + ] + }, + { + "name": "start", + "flags": [ + { + "flag": "--adapter ", + "effect": "provider adapter; v0 recognizes codex but is not activated" + }, + { + "flag": "--digest ", + "effect": "prepared WorkSpec digest" + }, + { + "flag": "--state-dir ", + "effect": "Goalrail local state directory" + } + ] + }, + { + "name": "finish", + "flags": [ + { + "flag": "--result ", + "effect": "check result: =[,[,]]" + }, + { + "flag": "--run ", + "effect": "generated run ID" + }, + { + "flag": "--state-dir ", + "effect": "Goalrail local state directory" + } + ] + } + ] +} diff --git a/public/docs/commands.md b/public/docs/commands.md index 578f003..eb17eea 100644 --- a/public/docs/commands.md +++ b/public/docs/commands.md @@ -10,12 +10,14 @@ what it left alone, and the pinned invocation the repository is now driven by. Repeating it changes nothing. Re-running it repairs a registration that is stale, unscoped, or naming an event this arrangement supersedes. + | Flag | Effect | |---|---| -| `--repo ` | repository to initialize, default the working directory | -| `--scaffold ` | override detection | -| `--fix-gitignore` | add the ignore entries the registration and marker need | -| `--confirm-schema-switch` | adopt a configuration that names another custom schema | +| `--confirm-schema-switch` | switch an OpenSpec configuration that names another custom schema | +| `--fix-gitignore` | add the ignore entries the registration and the marker need | +| `--repo ` | repository to initialize (default ".") | +| `--scaffold ` | codex or claude-code; omit to detect | + ## `gr doctor` @@ -73,6 +75,15 @@ itself did not run. `--json` emits the same report for a machine. `gr health` still works and names its successor. + +| Flag | Effect | +|---|---| +| `--json` | emit the diagnosis as JSON | +| `--repo ` | repository to diagnose (default ".") | +| `--scaffold ` | codex or claude-code; omit to report every supported scaffold | +| `--state-dir ` | Goalrail local state directory | + + ## `gr update` Brings this repository's overlay up to what the installed binary carries. @@ -88,6 +99,14 @@ The diagnosis is the only command that reaches the network, and only for the update check described above; initialization, this command, the session hooks, and the escalation loop reaching it would each violate a published requirement. + +| Flag | Effect | +|---|---| +| `--discard-local-edits` | replace overlay files that differ from the canon, discarding local edits | +| `--repo ` | repository whose harness to update (default ".") | +| `--state-dir ` | Goalrail local state directory | + + ## `gr connect` / `gr disconnect` `connect` attaches a scaffold that can only register at user scope; it needs @@ -97,6 +116,20 @@ and points at `gr init`. `disconnect` removes every registration, in whichever scope it lives, and leaves entries it did not add untouched. + +| Flag | Effect | +|---|---| +| `--scaffold ` | codex or claude-code | +| `--yes` | consent to modifying the scaffold configuration | + + + +| Flag | Effect | +|---|---| +| `--repo ` | repository whose registration to remove (default ".") | +| `--scaffold ` | codex or claude-code | + + ## `gr version` The binary's version and the overlay it carries. Nothing about a repository is diff --git a/public/llms-full.txt b/public/llms-full.txt index c821fed..2939f04 100644 --- a/public/llms-full.txt +++ b/public/llms-full.txt @@ -193,12 +193,14 @@ what it left alone, and the pinned invocation the repository is now driven by. Repeating it changes nothing. Re-running it repairs a registration that is stale, unscoped, or naming an event this arrangement supersedes. + | Flag | Effect | |---|---| -| `--repo ` | repository to initialize, default the working directory | -| `--scaffold ` | override detection | -| `--fix-gitignore` | add the ignore entries the registration and marker need | -| `--confirm-schema-switch` | adopt a configuration that names another custom schema | +| `--confirm-schema-switch` | switch an OpenSpec configuration that names another custom schema | +| `--fix-gitignore` | add the ignore entries the registration and the marker need | +| `--repo ` | repository to initialize (default ".") | +| `--scaffold ` | codex or claude-code; omit to detect | + ## `gr doctor` @@ -256,6 +258,15 @@ itself did not run. `--json` emits the same report for a machine. `gr health` still works and names its successor. + +| Flag | Effect | +|---|---| +| `--json` | emit the diagnosis as JSON | +| `--repo ` | repository to diagnose (default ".") | +| `--scaffold ` | codex or claude-code; omit to report every supported scaffold | +| `--state-dir ` | Goalrail local state directory | + + ## `gr update` Brings this repository's overlay up to what the installed binary carries. @@ -271,6 +282,14 @@ The diagnosis is the only command that reaches the network, and only for the update check described above; initialization, this command, the session hooks, and the escalation loop reaching it would each violate a published requirement. + +| Flag | Effect | +|---|---| +| `--discard-local-edits` | replace overlay files that differ from the canon, discarding local edits | +| `--repo ` | repository whose harness to update (default ".") | +| `--state-dir ` | Goalrail local state directory | + + ## `gr connect` / `gr disconnect` `connect` attaches a scaffold that can only register at user scope; it needs @@ -280,6 +299,20 @@ and points at `gr init`. `disconnect` removes every registration, in whichever scope it lives, and leaves entries it did not add untouched. + +| Flag | Effect | +|---|---| +| `--scaffold ` | codex or claude-code | +| `--yes` | consent to modifying the scaffold configuration | + + + +| Flag | Effect | +|---|---| +| `--repo ` | repository whose registration to remove (default ".") | +| `--scaffold ` | codex or claude-code | + + ## `gr version` The binary's version and the overlay it carries. Nothing about a repository is diff --git a/scripts/check-claims.mjs b/scripts/check-claims.mjs index 9b922e0..8c5b5a3 100644 --- a/scripts/check-claims.mjs +++ b/scripts/check-claims.mjs @@ -31,6 +31,13 @@ */ import {readFile} from 'node:fs/promises'; +import { + COMMANDS, + PIN, + SURFACE, + documented, + rewrite, +} from './product-reference.mjs'; import { DOCTOR_CAPTURE, DOCTOR_OUTPUT, @@ -52,9 +59,6 @@ const DOC = 'public/docs/install.md'; /** Where the newest published release announces itself. */ const LATEST = 'https://api.github.com/repos/heurema/goalrail/releases/latest'; -/** The page's copy of the same capture, which must not drift from the page. */ -const COMMANDS = 'public/docs/commands.md'; - /** One sentence stream, however the surface it came from wrapped it. */ function normalize(text) { return text.replace(/\s+/g, ' ').trim(); @@ -219,18 +223,93 @@ report( ` says ${claimed ?? 'none'}; they are one capture and must agree`, ); +// --- the pinned release ---------------------------------------------------- + +/** + * One release is what this site describes, and `product/pin.json` names it. + * + * Everything derived from the product is derived from that one tag: the flag + * tables the generator writes, the capture pasted above it. Bumping the pin is + * the act that pulls the site forward, and the assertions below are what make + * ignoring a release impossible rather than merely unwise. + */ +const pin = JSON.parse(await readFile(PIN, 'utf8')); +const surface = JSON.parse(await readFile(SURFACE, 'utf8')); +const commands = await readFile(COMMANDS, 'utf8'); + if (latest !== null) { report( - `the doctor capture is the current release (${latest})`, - captured === latest + `the pinned release is the newest one published (${latest})`, + pin.tag === latest ? null - : ` the page shows output from ${captured}, and ${latest} is` + - ' published. Re-run the capture against the release rather than' + - ' editing the version string — the other lines move too.', + : ` this site describes ${pin.tag} and ${latest} is published.` + + ' Bump the tag in product/pin.json, run `npm run sync:product`, and' + + ' read the diff: it is the list of what changed in the product, and' + + ' therefore the list of prose worth re-reading.', ); } -const commands = await readFile(COMMANDS, 'utf8'); +report( + 'the doctor capture is the pinned release', + captured === pin.tag + ? null + : ` the capture is from ${captured} and the pin says ${pin.tag}.` + + ' Re-run it against the pinned release rather than editing the version' + + ' string — the other lines move with it.', +); + +report( + 'the recorded surface is the pinned release', + surface.tag === pin.tag + ? null + : ` product/surface.json was generated from ${surface.tag} and the pin` + + ` says ${pin.tag}; run \`npm run sync:product\``, +); + +report( + `the generated blocks in ${COMMANDS} match that surface`, + commands === rewrite(commands, surface) + ? null + : ' a generated block was edited by hand, or the surface moved under' + + ' it. Run `npm run sync:product` and commit the result; the text inside' + + ' those markers belongs to the binary.', +); + +/** + * The one assertion about completeness rather than correctness. + * + * Everything above compares something written with something true, which cannot + * notice a command that was never written about at all — and a product under + * active development grows commands. So every command the binary accepts must + * either have a section on the page or a reason in the pin. Leaving one out + * stays allowed; leaving one out silently does not. + */ +const teaches = documented(commands); +const excused = pin.undocumented ?? {}; +const unexplained = surface.commands + .map(({name}) => name) + .filter((name) => !teaches.has(name) && !(name in excused)); + +report( + 'every command the binary accepts is documented or excused', + unexplained.length === 0 + ? null + : ` ${unexplained.join(', ')} — neither a section in ${COMMANDS} nor` + + ' an entry in product/pin.json. Document it, or record why the page' + + ' does not, so the omission is a decision with a diff.', +); + +const stale = Object.keys(excused).filter( + (name) => !surface.commands.some((entry) => entry.name === name), +); + +report( + 'nothing is excused that no longer exists', + stale.length === 0 + ? null + : ` product/pin.json excuses ${stale.join(', ')}, which ${pin.tag}` + + ' does not accept; drop the entry', +); report( `${COMMANDS} shows the same capture as the page`, diff --git a/scripts/product-reference.mjs b/scripts/product-reference.mjs new file mode 100644 index 0000000..5ffa807 --- /dev/null +++ b/scripts/product-reference.mjs @@ -0,0 +1,68 @@ +/** + * Rendering the product's own surface into the documentation, with no product + * present. + * + * These functions are here rather than in `sync-product.mjs` because both the + * generator and the cheap check need them, and the generator downloads and runs + * a binary at import time. The check runs on every pull request and must not do + * that, so what they share lives where importing it costs nothing. + */ + +/** The marked blocks these functions own, in the pages they own them in. */ +export const COMMANDS = 'public/docs/commands.md'; +export const SURFACE = 'product/surface.json'; +export const PIN = 'product/pin.json'; + +/** The table one generated block holds. */ +export function render(flags) { + if (flags.length === 0) return '_This command takes no flags._'; + return [ + '| Flag | Effect |', + '|---|---|', + ...flags.map(({flag, effect}) => `| \`${flag}\` | ${effect} |`), + ].join('\n'); +} + +/** + * Every `` block, replaced from the surface. + * + * Prose outside the markers is never touched, which is the whole arrangement: + * the reference is derived and the explanation around it is written. + */ +export function rewrite(markdown, surface) { + const missing = []; + const rewritten = markdown.replace( + /(\n)[\s\S]*?()/g, + (whole, open, command, close) => { + const found = surface.commands.find((entry) => entry.name === command); + if (!found) { + missing.push(command); + return whole; + } + return `${open}${render(found.flags)}\n${close}`; + }, + ); + if (missing.length > 0) { + throw new Error( + `${COMMANDS} has a generated block for ${missing.join(', ')},` + + ' which the binary does not accept', + ); + } + return rewritten; +} + +/** + * Which commands the page teaches, read from its own headings. + * + * Derived rather than listed so that documenting a command is one act instead + * of two, and so a list cannot fall out of step with the page it describes. + * One heading may name two commands, as `gr connect` / `gr disconnect` does. + */ +export function documented(markdown) { + const headings = markdown.match(/^## .+$/gm) ?? []; + return new Set( + headings.flatMap((heading) => + [...heading.matchAll(/`gr ([a-z-]+)`/g)].map((match) => match[1]), + ), + ); +} diff --git a/scripts/sync-product.mjs b/scripts/sync-product.mjs new file mode 100644 index 0000000..ee574fe --- /dev/null +++ b/scripts/sync-product.mjs @@ -0,0 +1,232 @@ +/** + * Asks the product what it accepts, and writes the answer into the pages. + * + * The prose on the documentation pages is written by hand and should stay that + * way — nothing generates an explanation of why re-running initialization is + * safe. What does not survive being written by hand is the reference: the list + * of commands and the flags each one takes. That part is already authored, in + * the product, as the help text its own users read. Retyping it here produced + * exactly what retyping produces: the flag table on this site paraphrased four + * descriptions and none of them matched what the binary prints. + * + * So the reference is derived. This script downloads the release named in + * `product/pin.json`, verifies it against the checksums published beside it, + * asks it for its help output, and rewrites the marked blocks in + * `public/docs/commands.md` from what it said. Prose around those blocks is + * untouched. + * + * Two properties are deliberate. + * + * It runs from a pin rather than from the newest release, and it writes files + * that get committed. Generating at build time from whatever is current would + * mean the deployed page could change without a diff anybody reviewed, which is + * the opposite of what this repository is for. Bumping the pin is a pull + * request, and its diff is the product's own changelog as far as this site is + * concerned. + * + * It normalizes exactly one thing and nothing else: Go's flag package prints + * `-repo`, and every other page here — including the install prompt the product + * itself authored — writes `--repo`. Rendering the single dash verbatim would + * contradict the instruction next to it. Descriptions are copied character for + * character. + * + * npm run sync:product rewrite the generated blocks + * npm run sync:product -- --check fail if they are not what the pin produces + * + * Check mode runs in CI and on the daily schedule alike. Running it only on the + * schedule was the first plan and it was wrong: the pull request that bumps the + * pin is the one moment the full check is actually needed, and that plan would + * have verified it the following morning. It downloads and executes a binary, + * which is a real cost, but this repository already will not build without + * reaching the product's README and the releases API. + * + * The cheaper half lives in check-claims.mjs and needs neither: that the blocks + * match the recorded surface, and that every command in it is documented or + * excused. + */ + +import {execFileSync, spawnSync} from 'node:child_process'; +import {createHash} from 'node:crypto'; +import {mkdir, mkdtemp, readFile, writeFile, rm} from 'node:fs/promises'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {COMMANDS, PIN, SURFACE, rewrite} from './product-reference.mjs'; + +const RELEASES = 'https://github.com/heurema/goalrail/releases/download'; + +const check = process.argv.includes('--check'); + +/** What the release archives call this machine. */ +function platform() { + const os = {darwin: 'darwin', linux: 'linux'}[process.platform]; + const arch = {arm64: 'arm64', x64: 'amd64'}[process.arch]; + if (!os || !arch) { + throw new Error( + `no published archive for ${process.platform}/${process.arch};` + + ' the release carries darwin and linux on amd64 and arm64', + ); + } + return `${os}_${arch}`; +} + +async function fetchOrDie(url, as = 'text') { + const response = await fetch(url, {redirect: 'follow'}); + if (!response.ok) { + throw new Error(`${url}: ${response.status} ${response.statusText}`); + } + return as === 'text' + ? response.text() + : Buffer.from(await response.arrayBuffer()); +} + +/** + * The published binary, verified the way the site tells an agent to verify it. + * + * An unverified download would make every claim below rest on whatever answered + * the request, which is a strange foundation for a check about honesty. + */ +async function download(tag, into) { + const want = platform(); + const checksums = await fetchOrDie(`${RELEASES}/${tag}/checksums.txt`); + + const line = checksums + .split('\n') + .map((entry) => entry.trim().split(/\s+/)) + .find(([, name]) => name?.includes(want)); + if (!line) { + throw new Error(`checksums.txt for ${tag} names no archive for ${want}`); + } + const [expected, archive] = line; + + const bytes = await fetchOrDie(`${RELEASES}/${tag}/${archive}`, 'binary'); + const actual = createHash('sha256').update(bytes).digest('hex'); + if (actual !== expected) { + throw new Error( + `${archive} does not match its published checksum\n` + + ` published ${expected}\n downloaded ${actual}`, + ); + } + + const path = join(into, archive); + await writeFile(path, bytes); + execFileSync('tar', ['-xzf', path, '-C', into, 'gr']); + return join(into, 'gr'); +} + +/** + * Help output, whichever stream it came out of and whatever it exited with. + * + * Both halves of that sentence are load-bearing. Go's flag package writes usage + * to standard error, and `gr help --help` exits non-zero while printing exactly + * what was asked for — so reading stdout on success only, which is what the + * obvious call does, returns an empty string for every command and finds no + * flags anywhere. It did, on the first run of this script, silently. + */ +function help(gr, args) { + const run = spawnSync(gr, args, {encoding: 'utf8'}); + if (run.error) throw run.error; + const printed = `${run.stdout ?? ''}${run.stderr ?? ''}`; + if (printed.trim() === '') { + throw new Error(`\`gr ${args.join(' ')}\` printed nothing`); + } + return printed; +} + +/** The commands the binary admits to, from the usage line `gr help` prints. */ +function commandsOf(gr) { + const usage = help(gr, ['help']); + const named = usage.match(/usage: gr <([^>]+)>/)?.[1]; + if (!named) { + throw new Error( + '`gr help` no longer opens with a `usage: gr ` line, so the' + + ' command list cannot be read from it. Teach this script the new shape' + + ' before touching the pages, so the pages stay derived from something.', + ); + } + return named.split('|').map((name) => name.trim()); +} + +/** + * The flags one command takes. + * + * Go's flag package prints a fixed shape: two spaces, the flag, optionally a + * type, then the description on the next line behind a tab. Both help headers in + * this binary — `Usage of x:` and `usage: gr x [flags]` with prose — are + * followed by that same block, so only the block is parsed and the header is + * ignored. + */ +function flagsOf(gr, command) { + const lines = help(gr, [command, '--help']).split('\n'); + const flags = []; + + for (let index = 0; index < lines.length; index += 1) { + const declared = lines[index].match(/^ {2}-(\S+)(?: (\S+))?$/); + if (!declared) continue; + + const description = []; + for (let next = index + 1; next < lines.length; next += 1) { + if (!/^\s+\t/.test(lines[next])) break; + description.push(lines[next].replace(/^\s+\t/, '').trim()); + index = next; + } + flags.push({ + // The one normalization, and it is about this site's own consistency + // rather than about the product: everything here writes `--flag`. + flag: `--${declared[1]}${declared[2] ? ` <${declared[2]}>` : ''}`, + effect: description.join(' '), + }); + } + return flags; +} + +// --- run ------------------------------------------------------------------- + +const pin = JSON.parse(await readFile(PIN, 'utf8')); +const workspace = await mkdtemp(join(tmpdir(), 'goalrail-sync-')); + +let surface; +try { + const gr = await download(pin.tag, workspace); + surface = { + $comment: + 'Generated by scripts/sync-product.mjs from the release named in' + + ' product/pin.json. Do not edit: run `npm run sync:product`.', + tag: pin.tag, + commands: commandsOf(gr).map((name) => ({name, flags: flagsOf(gr, name)})), + }; +} finally { + await rm(workspace, {recursive: true, force: true}); +} + +const markdown = await readFile(COMMANDS, 'utf8'); +const rewritten = rewrite(markdown, surface); +const recorded = `${JSON.stringify(surface, null, 2)}\n`; + +if (!check) { + await mkdir('product', {recursive: true}); + await writeFile(SURFACE, recorded); + await writeFile(COMMANDS, rewritten); + console.log( + `wrote ${SURFACE} and the generated blocks in ${COMMANDS}` + + ` from ${pin.tag} (${surface.commands.length} commands)`, + ); + process.exit(0); +} + +let failed = false; +const committed = await readFile(SURFACE, 'utf8').catch(() => null); + +if (committed !== recorded) { + failed = true; + console.error(`FAIL ${SURFACE} is not what ${pin.tag} reports`); + console.error(' Run `npm run sync:product` and commit the result.'); +} + +if (markdown !== rewritten) { + failed = true; + console.error(`FAIL the generated blocks in ${COMMANDS} are stale`); + console.error(' Run `npm run sync:product` and commit the result.'); +} + +if (failed) process.exit(1); +console.log(`ok the reference matches ${pin.tag} as published`); From fa5e44e7e398d362e47910f78dc0d43da9673c42 Mon Sep 17 00:00:00 2001 From: "Vitaly D." Date: Fri, 31 Jul 2026 17:48:53 +0300 Subject: [PATCH 2/2] Read the prompt at the pinned release too, not at main MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The prompt was the one thing here still compared against the product's `main`, and the reasoning for that — notice the day the product moves, not the day someone bumps a pin — did not survive contact with a moving product. A change landed on `main` that rewrote the prompt for behaviour no release carries, and the check went red on content it had passed an hour earlier. Copying the new wording would have made the site worse rather than better. The prompt's own first instruction is to download `releases/latest`, which serves v0.1.2. Both binaries were run in the same fresh repository to be sure of what that means: the released one applies the registration and prints a notice naming `--fix-gitignore`, which is exactly what the prompt on this site tells the agent to watch for, and the one built from `main` writes that ignore rule itself and prints no notice at all. The new prompt would have sent an agent to fetch one binary and then look for a report only a different one prints. So the prompt is read at the pinned tag, like the flag tables and the capture already are, and the site describes one release consistently. Noticing that the product moved is not this comparison's job and never should have been: the pin failing to be the newest published release is what fires, and bumping it brings the new prompt, the new flags and a new capture in one reviewable diff. The rationale in the file header said the opposite, and a stale explanation inside the check against stale claims is its own small joke. It now says what the code does and why it changed. --- README.md | 19 +++++++++++------ scripts/check-claims.mjs | 46 ++++++++++++++++++++++++++++++---------- 2 files changed, 48 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index d73c184..1c84970 100644 --- a/README.md +++ b/README.md @@ -41,12 +41,19 @@ a section on the commands page or an entry in `product/pin.json` saying why not. A command added to the product cannot pass through here unnoticed; leaving one out stays allowed, leaving one out silently does not. -**A clock, not a pull request.** `check:claims` compares the install prompt with -the product's README and the pin with the newest published release, and -`.github/workflows/drift.yml` runs both daily, because drift here is caused by -the product moving rather than by anything happening in this repository. A -failure opens an issue. Both need the network and fail loudly without it — a -check that passes when it did not run is the promise it was meant to replace. +**Everything is read at the pin, including the prompt.** `check:claims` +compares the install prompt with the README *at the pinned tag*, not at `main`. +That was the other way round for a week, until a change landed on `main` that +rewrote the prompt for behaviour no release carries yet — and the prompt's own +first instruction is to download `releases/latest`. Copying it would have told +an agent to fetch one binary and look for a report only a different one prints. + +**A clock, not a pull request.** What notices the product moving is the pin +failing to be the newest published release, and `.github/workflows/drift.yml` +runs the checks daily, because drift here is caused by the product moving rather +than by anything happening in this repository. A failure opens an issue. All of +it needs the network and fails loudly without it — a check that passes when it +did not run is the promise it was meant to replace. ## Shape diff --git a/scripts/check-claims.mjs b/scripts/check-claims.mjs index 8c5b5a3..79be47c 100644 --- a/scripts/check-claims.mjs +++ b/scripts/check-claims.mjs @@ -21,11 +21,13 @@ * line; those are three renderings of one sentence stream, and re-wrapping any * of them is not drift. * - * The comparison is against the product's `main` rather than a pinned tag on - * purpose: the point is to notice the day the product moves, not the day someone - * remembers to bump a pin. That means this check needs the network, and it fails - * loudly when it cannot reach it — a check that passes when it did not run is - * the promise it was meant to replace. + * Everything compared against the product is compared at the release named in + * `product/pin.json`, for the reason set out above `readmeAt`. What notices the + * product moving is not any of these comparisons but the pin itself failing to + * be the newest published release. + * + * All of it needs the network, and it fails loudly when it cannot reach it — a + * check that passes when it did not run is the promise it was meant to replace. * * Run with `npm run check:claims`. */ @@ -45,8 +47,28 @@ import { INSTALL_PROMPT_VISIBLE, } from '../src/content.ts'; -const README = - 'https://raw.githubusercontent.com/heurema/goalrail/main/README.md'; +/** + * The product's README, at the release this site describes — not at `main`. + * + * This was `main` at first, on the reasoning that the point is to notice the day + * the product moves rather than the day someone bumps a pin. Reality settled it + * the other way within the week. A change landed on `main` that rewrote this + * prompt for behaviour no release carries yet, and the two binaries were run + * side by side in the same fresh repository to be sure: the released one applies + * the registration and prints a notice naming `--fix-gitignore`, which is what + * the prompt here tells the agent to watch for, and the one built from `main` + * writes that ignore rule itself and prints no notice at all. + * + * The prompt's own first instruction is to download `releases/latest`. Copying a + * prompt written for an unreleased build would have told an agent to fetch one + * binary and then look for a report only a different one prints. So the prompt + * is read at the pinned tag, like everything else derived here, and the product + * moving is caught where it belongs — by the pin no longer being the newest + * release, which brings the new prompt along with the new flags and the new + * capture in one bump. + */ +const readmeAt = (tag) => + `https://raw.githubusercontent.com/heurema/goalrail/${tag}/README.md`; /** The heading in the README whose blockquote is the prompt. */ const HEADING = '### Or hand it to your agent'; @@ -122,6 +144,11 @@ function report(subject, problem) { console.error(problem); } +const pin = JSON.parse(await readFile(PIN, 'utf8')); +const surface = JSON.parse(await readFile(SURFACE, 'utf8')); +const commands = await readFile(COMMANDS, 'utf8'); +const README = readmeAt(pin.tag); + let source; try { const response = await fetch(README); @@ -151,7 +178,7 @@ if (quoted === null) { const expected = normalize(quoted); report( - 'the copied prompt matches the product README', + `the copied prompt matches the README at ${pin.tag}`, normalize(INSTALL_PROMPT) === expected ? null : ` src/content.ts INSTALL_PROMPT has drifted from ${README}\n` + @@ -233,9 +260,6 @@ report( * the act that pulls the site forward, and the assertions below are what make * ignoring a release impossible rather than merely unwise. */ -const pin = JSON.parse(await readFile(PIN, 'utf8')); -const surface = JSON.parse(await readFile(SURFACE, 'utf8')); -const commands = await readFile(COMMANDS, 'utf8'); if (latest !== null) { report(