Skip to content

Feature: make the pinned arcade reference the authority every build and review checks against - #112

Merged
StarshipSuperjam merged 12 commits into
mainfrom
claude/xevious-reference-fidelity
Sep 3, 2026
Merged

Feature: make the pinned arcade reference the authority every build and review checks against#112
StarshipSuperjam merged 12 commits into
mainfrom
claude/xevious-reference-fidelity

Conversation

@StarshipSuperjam

@StarshipSuperjam StarshipSuperjam commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Closes #111

A green mechanical check below shows this change conforms to the engine's rules — not that it is correct. What covers correctness is the behavioural steps in Review and your own read of the change; a green check is never a substitute for that. Your merge is the binding gate.

About those checks: only the one that runs when the change is proposed for merge can stop a risky merge — a check that ran while the change was still being written is early advice. Each check is itself proven against a deliberately broken example it must catch, so a passing check can't be one that quietly did nothing — but that proves the check works, not that this change is right. And a check that could not run leaves its area unverified.

Purpose

Make the pinned arcade source the authority every build and review checks the port against, instead of the docs/spec/ prose.

  • Motivated by the slice 8 (Feature: Toroid vertical slice — first live flying enemy (slice 8) #110) Toroid-swing regression: a settled spec sentence described the swing backwards, both conformance reviewers flagged the correct build against the prose, the session "fixed" the build to match, and three gameplay bugs reached the playtest. The session then had no local reference to check against and had to fetch the source over the web.

Impact: a citation that no longer points where it claims cannot pass unnoticed, and a reviewer reads the arcade source itself before a build is handed over for playtest.

Scope

Tools, a citation audit, a fidelity-gated handover, a CI backstop, two review personas, and a conduct rule — no gameplay code.

  • tools/reference_checkout.py — a verified throwaway clone of jotd666/xevious at the pin, so any session or CI can open the cited source (ensure/path/verify; offline once cached).
  • tools/reference_citations.py — resolves every citation in docs/spec/ and docs/mechanics/ to a real label and line range at the pin (102 citations, 0 unresolved).
  • tools/playtest_package.py — the handover gate: it runs the reference checks and refuses to build a playtest .sb3 while any citation is unresolved.
  • The one-time citation audit that corrects the citations the resolver could not resolve.
  • .github/workflows/reference-fidelity.yml — an advisory CI backstop.
  • Two reference-fidelity review personas at the plan and pre-submission gates, and a conduct-reference-outranks-spec code, plus an always-loaded section in CLAUDE.md/AGENTS.md.

Change profile — the shape of this pull request at a glance:

  • Size: 35 files changed, +1529 / −38 lines.
  • Kinds of thing touched: 2 tools, 1 operation, 32 other files (not in the engine's map).
  • Where: docs, tests, tools, .claude/agents, .codex/agents, .engine/tools, .engine, .engine/conduct, .engine/knowledge, .engine/operations, .github, .gitignore, AGENTS.md, CLAUDE.md, README.md.
  • Shape: 10 commits on this branch.

Impact: the reference becomes the authority mechanically (resolver + handover gate + CI) and by routed judgment (the personas), and every session loads the rule.

Behaviors

The falsifiable capabilities this change delivers, each with the test that exercises it:

  • A verified reference checkout is obtainable without leaving the repo or fetching over the web — tests/test_reference_checkout.py.
  • Every citation resolves to a real label and range at the pin; a nonexistent label, an approximate ~NNN, an over-wide range, or a missing file fails — tests/test_reference_citations.py, tests/test_reference_corpus.py.
  • The handover tool refuses to build while any citation is unresolved — tests/test_playtest_package.py.

Out of scope

Deliberately excluded, with reasons.

  • The bare-range → labelled-citation conversions the resolver only advises on. On inspection the "clean" set was an artifact of a line-based advisory over-reporting citations the paragraph-based resolver already checks (verified: update_solvalou_sprite_XY, find_idle_and_init_radiating_bullet, and others resolve today); the genuine remainder either maps awkwardly to a single label ("the four angle tables span 6290–6427") or exceeds the span cap. Forcing labels would reduce accuracy, so they stay as prose.
  • The aerial-enemies.md Toroid swing prose made explicit ("reverses away", not homing). That draft doc is being rewritten by in-flight PR Feature: Toroid vertical slice — first live flying enemy (slice 8) #110; the clarification belongs there.
  • Making the citation check a required merge gate. You chose in-session + advisory CI (backstop only); see Risk.

Impact: the settled-doc edits are held to the genuinely-broken citations, keeping the guardrail-ack surface tight.

Risk

No engine guardrail is weakened; the friction is deliberate consent, and two engine self-tests are hand-edited.

  • Settled specs + guarded policy → guardrail-ack. This edits four settled specs (core-game-systems, player-craft-and-weapons, difficulty-and-formations, area-progression-and-terrain) and the guarded docs/REFERENCE_POLICY.md. All are locator-only or a disclosed posture change; none change gameplay meaning. Apply guardrail-ack to clear the product-lock and guarded-path checks.
  • docs/REFERENCE_POLICY.md posture change, called out distinctly. The policy said "nothing here depends on [the reference clone]." After this, the fidelity tooling depends on the reference being obtainable at the pin (not on a stored copy). Your guardrail-ack covers this as a deliberate change, not a typo fix.
  • Two engine self-tests are hand-edited. test_conduct and test_lens_consumption hardcode the shipped defaults and fail when a project uses a documented feature (a conduct code, a project review persona). They are relaxed minimally. These files are overwritten on each engine update, so the edits must be re-applied then. Filed as Engine fault: required self-tests block documented per-deployment customization (project conduct code and review persona) #113; you noted such an issue won't reach the engine home, so this is a recurring per-update chore.
  • "Fails CI" is advisory, not blocking — an accepted deviation from the issue text. Issue Feature: Make the pinned arcade reference the authority every build and review checks against #111 said a bad citation should "fail CI"; you chose in-session enforcement + advisory CI. So a broken citation turns the advisory reference-fidelity check red but does not block merge. Recorded here as the deliberate choice it is.
  • Engine update drops the lens wiring. The reference-fidelity line in build-orchestration.md is overwritten on update; the hard lens-consumption check then goes red with a self-describing fix. A recurring one-line chore.

Impact: every friction here is a consent gate you clear or a disclosed recurring cost, not a silent weakening.

Validation

Full local suites green; the new CI job passes; two checks stay red pending your guardrail-ack.

  • 217 project tests pass; python tools/scratch_project.py verify and reference_extract.py --verify pass; reference_citations.py reports 102 checked, 0 unresolved against a real clone at the pin; actionlint clean.
  • The engine CI check-suite passes with no hard findings; the two previously-failing engine self-tests now pass; a full engine self-test run confirms no other breakage.
  • On the live PR, reference-fidelity, xevious-project, runtime-harness, secret-scan, CodeQL, actionlint, and engine-overlay-disclosure are green. engine-ci and engine-guard stay red until you apply guardrail-ack (the settled-doc / guarded-policy gate).

Impact: an approver can rely on the mechanical checks; the two red ones are your consent step, not defects.

Review

A thorough cold review ran at Standard depth: four plan-review passes before building and five pre-submission passes after, plus a blind reference-fidelity replay. Real findings were fixed; two blocking findings were corrected.

  • Plan-review (architecture, feasibility, risk-governance, product-intent) shaped the design before any code: block extent from the label index (not bytes_under_label), bare file-token normalization and paren-group scoping, a CI-safe git identity in tests, and the matrix-regen check were all adopted from these passes.
  • Pre-submission (spec-conformance, divergence-hunter, technical-integrity, security-governance, reference-fidelity) ran cold on the built change. Outcomes:
    • Blocking (technical-integrity), fixed: the resolver never scanned the file+range (no-label) citation shape, so four real citations were invisible and an impossible .inc citation would pass. Now scanned (98 → 102 citations), with a fixture test.
    • Blocking (spec-conformance), fixed: two required engine self-tests failed on this project's conduct code and persona lens. Relaxed minimally; both pass; engine fault Engine fault: required self-tests block documented per-deployment customization (project conduct code and review persona) #113 filed.
    • Serious, fixed: an over-wide range was accepted (now capped at 256 lines); the second ~514 survived and its guard no-oped (both fixed); unguarded git steps raised raw tracebacks (now clean CheckoutError); the playtest checklist routed around the handover gate (now through it); CI actions pinned to SHAs.
    • Confirmed not a defect: leaving reference_citations.py and playtest_package.py unguarded is your explicit choice (guard the pin-deciding files only); the citation audit fixes are genuine, verified against the source.
  • Between reviewed and submitted: the reviewed commit was 86f4bef; four fix commits followed (b6ad1d8, 41bb604, 75ea249, and this body). Net change since the reviewed version: about +190 / −40 lines across the resolver, checkout tool, tests, two engine self-tests, and three docs — a material post-review delta, re-verified by the full suite and a fresh resolver run (102/0).

Impact: the engine's own account of the review — your merge is the binding gate.

Demonstration

The slice-8 replay: the reference-fidelity persona, blind to the answer, reads the source and judges the prose.

  • Run the persona (or by hand) against docs/spec/aerial-enemies.md lines 41–44 with a checkout present. It opens src/xevious_main.68k toroid_toggle_dir 3289–3321 and the two swing arms, determines from the source alone that the swing reverses away from the craft (the two arms adjust _dY by ∓1, locked at the trigger), and reports whether the prose is faithful. In this PR's run it correctly found the swing is a reversal and the current wording faithful — the exact read that was missing when slice 8 trusted the prose.
  • Mechanically: python3 tools/reference_citations.py --checkout "$(python3 tools/reference_checkout.py path)" prints 102 citations checked, 0 unresolved; break any cited label or range and it prints that citation as unresolved and exits non-zero.

Impact: you can watch the gate read the source and catch a wrong citation, rather than trust the prose.

Files of interest

The core is three tools and the resolver logic.

  • tools/reference_citations.py — the resolver (grammar, scoping, resolution rule).
  • tools/reference_checkout.py, tools/playtest_package.py — the checkout and the handover gate.
  • .claude/agents/reference-fidelity*.md — the review personas.
  • The citation audit across docs/spec/*.md and docs/mechanics/{020,022,015}.

Impact: these most determine whether the reference stays the authority.

AI involvement

Built with Claude Code (Opus 4.8).

  • Claude designed and wrote the tools, tests, personas, and docs; ran four cold plan-review and five cold pre-submission passes (Opus and Sonnet review personas) and a blind reference-fidelity replay; dispositioned every finding and fixed the blocking ones. Load-bearing judgment: the citation grammar and resolution rule, the span cap value, and the disposition of the deferred conversions.
  • All citation fixes were verified against the pinned source, not authored from memory.

Impact: the resolver logic and the citation audit are where AI judgment is load-bearing; the source verification bounds it.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

This pull request touches more than one kind of engine file. Here is what the next update does with each — none of it blocks your merge:

Heads-up: this pull request changes engine files the next engine update is set to overwrite.

The engine keeps its own machinery current by replacing these files wholesale when you update — so a change you make to them here won't survive the next update; it will quietly revert:

  • .engine/operations/build-orchestration.md
  • .engine/tools/test_conduct.py
  • .engine/tools/test_lens_consumption.py

If that was a one-off, this is just so you know. If you want the change to last, the durable home for an edit to engine machinery is upstream in the engine project these files come from (StarshipSuperjam/engine-template) — a fix there travels to every update. And if what you actually want is to customize how the engine behaves, the settings that do survive an update are your tunable policy (via /engine-tune) and your operator notes — those are preserved; these files are not.

These engine-internal files are regenerated by the update.

The engine rebuilds them from your project's own current state on each update. A change here that reflects a real change in your project — you edited what the file is generated from and regenerated it — is reproduced, not lost. But content typed directly into one of these files, with no matching change to the source it comes from, will not survive: it is rebuilt away on the next update. So if you meant to change what one of these records, edit the source it is generated from (for the settled-criteria record, your docs/spec/), not the file itself:

  • .engine/knowledge/graph.json

This is a heads-up only — it does not block your merge, and your merge is the decision.

@StarshipSuperjam
StarshipSuperjam marked this pull request as ready for review September 2, 2026 22:34
StarshipSuperjam and others added 12 commits September 2, 2026 16:03
…ty (claim)

Claim commit for issue #111. Opens the draft pull request; no changes yet.
The work lands as an ordered commit series behind this claim.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tools/reference_checkout.py: ensure/path/verify a throwaway local clone of
jotd666/xevious at the pinned commit, so a build session or CI can open the
cited source lines and re-derive the spec data. Hash verification reuses
reference_extract.SourceFile (one implementation), and the clone lives in a
shared cache outside the repo by default so many worktrees reuse it.

Tests run fully offline: a fake five-file checkout with reference_extract
EXPECTED_SHA256 patched to the fakes' digests (shared module object, plain
import so the patch reaches SourceFile), a real git repo in tempfile for the
at-pin no-op and wrong-commit-offline paths, each with a committer identity set
inline so CI with no global git identity still passes.

.gitignore: ignore an in-repo /.xevious-reference/ when --dir points inside the
tree (default is outside); placed in the un-fenced project region.

Part of #111.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tools/reference_citations.py: read every citation in docs/spec and
docs/mechanics and check it points at a real label and line range at the pin.
A citation resolves when the file is one of the five pinned sources, the label
exists, and the range starts at or sits inside that label's block (block extent
computed from the label index, never bytes_under_label, which collapses on a
code routine). File tokens accept the bare or src/ spelling and are sticky
within a parenthetical group; a document declares its default file once. Labelled
citations are recognised regardless of surrounding parentheses so a correct
citation outside parens is never silently skipped. Mechanics records are scanned
only on their reference-provenance line, binding ranges to backticked labels so
coordinate bounds and supplementary prose numbers are not mistaken for citations.

Fixture tests only (the resolver is not run against the real corpus until the
audit lands): one positive per citation shape, one negative per hard failure,
against a fake five-file checkout with EXPECTED_SHA256 patched to its digests.

Part of #111.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Correct the citations the resolver cannot resolve at the pinned commit — locators
only, no gameplay-meaning change:

- Nonexistent labels -> their real names at the pin: sub_2_fn_8/_22 gain their
  __fire_freq_mask_* suffixes (difficulty-and-formations.md, mechanics 022),
  sub_2_fn_3 gains __inc_enemy_AI_and_flying_enemies (mechanics 020).
- Missing file in scope -> declared: player-craft-and-weapons.md and
  difficulty-and-formations.md each declare their default source file
  (main and sub respectively); core-game-systems.md and andor-genesis.md gain an
  explicit file token on the one citation that leaves the surrounding default.
- Approximate reference -> exact: area-progression-and-terrain.md's `~514`
  becomes `main_gameplay_loop` 514-521 (the area-advance / 16->7 loop).

Add tests/test_reference_corpus.py: three network-free guards (every reference
provenance line names the index pin; every spec citation has a file in scope; no
approximate line reference survives), which run in the ordinary test job.

The settled-criteria matrix is unchanged (edits fall outside acceptance-criteria
cells, verified by regenerating it). Deferred, deliberately: the bare-range ->
labelled-citation conversions, which cluster in the draft enemy docs that PR #110
and the upcoming settling slices rewrite; they land there under the new fidelity
review rather than conflicting with in-flight work.

Touches four settled specs (player-craft-and-weapons, difficulty-and-formations,
core-game-systems, area-progression-and-terrain) -> needs guardrail-ack.

Part of #111.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…4/6)

tools/playtest_package.py: the one command a build session runs to produce the
playtest .sb3. It ensures a verified checkout, re-derives the generated data,
resolves every citation, and only then builds — refusing to emit a build while
any citation is unresolved, so a build that adapted to a wrong spec cannot reach
the playtest (the slice-8 gap). The guarded builder is invoked unmodified as a
subprocess. Tests stub each step and assert the gate: a failing check stops the
build; the build runs only after all checks pass.

.github/workflows/reference-fidelity.yml: an advisory workflow (not in the
required ruleset) that clones at the pin, re-verifies, and resolves citations on
every pull request — a backstop, not the in-session fix, and loud if upstream is
unreachable.

.engine/operator-guarded-paths.json: guard the pin-deciding files
(reference_extract.py, reference_checkout.py) and the new workflow; the maturing
resolver and handover tool stay ungated while they settle.

Docs: docs/spec/index.md replaces the manual /tmp clone recipe with the tool,
drops "run by a person, not CI", and states the citation grammar; docs/mechanics
/README.md requires labelled line ranges; docs/principles.md records that
citation resolution is engine-checked while prose fidelity is the review's
judgment; README.md adds the reference and handover commands; docs/REFERENCE_
POLICY.md reconciles "nothing depends on it" — the reference must now be
obtainable at the pin for the fidelity tooling to ground a change (guarded file
-> guardrail-ack).

Part of #111.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… 5/6)

Add two project-owned review personas sharing the reference-fidelity lens:
reference-fidelity-plan (plan gate) and reference-fidelity (pre-submission gate).
They are the only reviewers that read the arcade source itself: they obtain a
verified checkout, resolve the change's citations, and open the cited source
lines to check the spec prose, the derived-behaviour, and the Scratch evidence
against what the source does — the source wins, and a "could not ground" is a
blocking finding, never a pass. The plan-gate persona additionally requires that
any alleged spec divergence be checked against the source before the plan adopts
it, the step missing when the Toroid swing was "fixed" to match wrong prose.

Wire reference-fidelity into the plan-review and pre-submission lines of the
consumed-review-lenses block so lens-consumption stays green, and commit the
Codex renders (codex_gen) and the regenerated knowledge graph. Both personas are
judgment/opus/high per the model bindings.

Part of #111.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…#111, 6/6)

.engine/conduct/operator.md: add the standing code conduct-reference-outranks-spec
— before accepting a factual claim about the game's behaviour (from the spec, a
review finding, or memory), open the cited source lines and read them; where the
reference and the spec disagree the reference wins and the spec is corrected,
never the build changed to match wrong prose. Worded as grounding posture only —
it leaves the review gates and approval untouched (no weakening).

CLAUDE.md and AGENTS.md: append an "Arcade reference fidelity" section after the
engine-managed fence (preserved across engine updates) so both runtimes load the
rule every session — the pin, the checkout and resolver commands, and that a
playtest build is produced only through tools/playtest_package.py while every
citation resolves.

Closes #111.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…#111)

Resolver (tools/reference_citations.py):
- Scan the file+range (no-label) citation shape the docstring promised but the
  scanner never emitted, so citations like `xevious.inc 61-81` and `xevious_ram.68k
  161-218` are now checked instead of silently invisible (4 real citations; the
  resolver total rises from 98 to 102, still 0 unresolved).
- Cap a citation range at 256 lines (widest real citation is ~74) so an over-wide
  range that merely contains the label is rejected rather than accepted as precise.
- Resolve --paths to absolute so a relative argument no longer crashes on
  relative_to(ROOT).

Corpus guard (tests/test_reference_corpus.py): strengthen the approximate-line
guard to catch a bare `~NNN` anywhere, not only next to a label token, while
leaving durations (`~56 frames`, `~12.4 s`) alone. This catches the second `~514`
in docs/mechanics/015-area-clock.md, now made exact (514).

Checkout (tools/reference_checkout.py): wrap the git init/remote/checkout steps in
clone_at_pin so a dirty cache surfaces a clean CheckoutError instead of a raw
traceback; add a clone_at_pin test that clones from a local file:// origin and one
that forces a prepare failure.

New fixture tests cover the file+range shape and the over-wide rejection.

Part of #111. Addresses technical-integrity and divergence-hunter findings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…code (issue #111)

Two required engine self-tests hardcode the shipped defaults and fail when a
deployment uses a documented customization: test_operator_override_ships_empty
asserts operator.md carries no conduct codes, and test_lens_consumption asserts an
exact nine consumed review lenses. This project adds both a conduct code and the
reference-fidelity review persona (both documented as supported), so the tests are
relaxed minimally — the conduct file must be well-formed rather than empty, and the
stock lenses must be a subset of the consumed set rather than an exact match.

These are engine-owned, overlay-overwritten files, so the edits are wiped on each
engine update and must be re-applied then. The operator confirmed an engine-fault
issue filed in this repo will not reach the engine home, so this is a recurring
per-update chore, recorded on issue #113 and in the test comments.

Part of #111.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…issue #111)

docs/PLAYTEST_CHECKLIST.md: the "before you start" build step now uses
tools/playtest_package.py, so the operator's own canonical procedure runs the
reference checks before producing the .sb3 instead of pointing straight at the raw
builder — closing the contradiction with the CLAUDE.md rule that a playtest build
goes only through the handover tool.

.github/workflows/reference-fidelity.yml: pin actions/checkout and
actions/setup-python to commit SHAs (matching the engine's stricter convention)
instead of mutable @v7 tags.

Part of #111. Addresses divergence-hunter, product-intent, and security-governance findings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e (issue #111)

The engine self-test relaxations reference the upstream bug where it can be
fixed (StarshipSuperjam/engine-template#1200) rather than the local record.

Part of #111.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…refine guard (issue #111)

Rebasing onto main (slice 8, PR #110) surfaced two things the fidelity tooling
now covers:

- docs/mechanics/023-aiming-and-slot-positions.md cited `cpy_dY_dX_to_obj` by its
  routine address ($3383) where the convention is a line number; the resolver
  flagged it. Corrected to the line (5135) so it resolves — the gate catching a
  real citation slip in freshly-merged work.
- The approximate-line guard mis-flagged value suffixes new in slice 8 records
  (`~128 half-px`, `~40%`). Refined: flag a bare `~NNN` only when it is not
  followed by a unit word or `%`, so a genuine `~line)` reference is still caught
  while counts, distances, and percentages are left alone.

Resolver: 107 citations checked, 0 unresolved. Full suite (229 tests) green.

Part of #111.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@StarshipSuperjam
StarshipSuperjam force-pushed the claude/xevious-reference-fidelity branch from 3cd1547 to 35be6ed Compare September 2, 2026 23:08
@StarshipSuperjam StarshipSuperjam added the guardrail-ack You add this to approve a change the engine flagged as weakening a built-in safety protection. label Sep 3, 2026
@StarshipSuperjam
StarshipSuperjam merged commit f3b24f8 into main Sep 3, 2026
14 of 16 checks passed
@StarshipSuperjam
StarshipSuperjam deleted the claude/xevious-reference-fidelity branch September 3, 2026 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

guardrail-ack You add this to approve a change the engine flagged as weakening a built-in safety protection.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Make the pinned arcade reference the authority every build and review checks against

1 participant