Skip to content

fix(coord): provenance on every rescue-ref writer, and an audit that reads all three namespaces (BACKLOG #1349) - #819

Merged
wshallwshall merged 8 commits into
mainfrom
fix/1349-rescue-ref-provenance-remainder
Sep 4, 2026
Merged

fix(coord): provenance on every rescue-ref writer, and an audit that reads all three namespaces (BACKLOG #1349)#819
wshallwshall merged 8 commits into
mainfrom
fix/1349-rescue-ref-provenance-remainder

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

What this lands

BACKLOG #1349's remainder. The write-time control already shipped and this PR does not touch it. What was left was the population the audit grades: three writer surfaces bypassed the mefor-rescue-v1 marker, and the audit read two of the three namespaces refs land in.

I re-measured before building rather than trusting the score. rescue.ps1 -Check in this worktree reported EXAMINED 1671 / UNVERIFIABLE 1671, and all three writer surfaces were as described. Nothing here was already built.

1. Provenance on the sh post-commit hook

scripts/hooks/durability_push.sh force-pushed a bare ref recording nothing. It now builds an annotated tag object carrying the same mefor-rescue-v1 message rescue.ps1 -Anchor writes, and pushes it by object id.

No local ref is created, and that is the design decision worth reviewing. Writing the same annotated tag under local refs/tags/ first would have been simpler and would have made every rescue tag reachable from a branch tip, so git push --follow-tags or git push --tags would carry them to whatever remote a hand reaches for. The default one here is public. A test asserts the local tag namespace stays empty.

How the never-fail-a-commit constraint was met, and how it was tested. Two new failure modes exist: git var GIT_COMMITTER_IDENT can fail, and git hash-object fsck-validates the tag object and refuses a malformed tagger line. Either leaves $TAGOBJ empty, the push falls back to the bare HEAD: form, one warning goes to stderr, and the hook still exits 0. Driven in a test through its most reachable trigger, an empty GIT_COMMITTER_NAME, which was measured to fail git var with fatal: empty ident name ... not allowed, exit 128. Two further arms cover an unreachable remote and an unarmed hook. The provenance work stays in the foreground rather than joining the background push, so a failure reaches the terminal that caused it instead of arriving after the prompt returns; measured cost is 0.19s of CPU per commit over five paired runs of each form. Wall clock is deliberately not quoted, because the box was running six concurrent test processes and the figure would be about load.

A detached HEAD omits was-tip rather than guessing it. Both True and False would be claims about a branch that does not exist, and omitting the line is what makes -Check report SELF-DESCRIBING.

2. The printed remedy

unbacked_check.ps1 printed git push --force <remote> <branch>:refs/tags/rescue/branch/<branch>, so following the tool's own advice wrote an unverifiable ref. It now runs -Anchor and pushes what that wrote, and carries the reason with the command so the extra step does not read as ceremony and get cut. The test asserts both halves: -Anchor is present and the bare form is gone.

3. Widening the audit

-Check now enumerates remote-tracking prefixes from git remote rather than hardcoding private, and grades refs/remotes/<remote>/rescuetags and refs/remotes/<remote>/rescue as a separate mechanism. Namespaces matching nothing are still listed at zero.

The two mechanisms are kept apart, which the item asks for by name. A snapshot behind its branch is behind it forever; a push-updated ref behind a live branch may merely be lagging. Mechanism changes the wording of BEHIND and DIVERGED and changes no verdict -- deliberately, because once the branch is gone nothing pushes again and the branch-gone arms are equally final either way. Counts are not deduplicated: the two names disagreeing is the finding.

The numbers I actually measured

before after
EXAMINED 1671 3202
UNVERIFIABLE 1671 3202

Read that honestly: widening the audit found more unverifiable refs, it repaired none. The two counts moving together is the correct result, not a regression. The split is 1671 snapshot and 1531 push-updated. The snapshot half can never be retrofitted, because the information was never captured. The push-updated half heals on the next commit to the branch it tracks, so that count falls with activity rather than with time.

The absolute figure drifts while you watch it: it read 3187, then 3202, minutes apart, because other sessions are committing. Treat it as a dated sample.

Assumptions I made where the brief left something open

  1. The tag object is pushed by id, with no local ref. The brief said "give it provenance" without saying how. This was the choice with the smallest new blast radius.
  2. refs/remotes/<remote>/rescue is read as well as rescuetags. It was not named in the brief; it exists in this checkout, holding one ref per remote.
  3. The banner stays open. backlog_status_check.py records that a builder does not flip a banner; for closing-act code the LANDER flips it on merge. I moved it to the in-progress banner and wrote what landed, what is not work, and why it is still open.
  4. The scoring table row near the top of docs/BACKLOG.md was left alone. It is a dated snapshot of the 2026-09-03 pass and editing one row would misrepresent that pass.
  5. A grep of scripts/, .github/ and ide/ finds no fourth writer surface. Stated as what the grep covered, not as a completeness claim about refs written by hand outside the repository.

Out of scope, and not touched

The ruling that a behind ref is a snapshot and not a defect (rescue.ps1 DESCRIPTION and the BEHIND arm), any repair or rewrite of the 3202 existing refs, and BACKLOG #1378.

The annotated-tag trap

git rev-parse on these returns the tag object. diff, merge-base and rev-list dereference silently, so ancestry looks fine while a published sha does not resolve. Every sha the code publishes goes through %(*objectname) or ^{commit}, and a test pins it. No sha is published in this PR body.

Checks

Run from the worktree root with the primary checkout's interpreter, since this worktree has no .venv. Import source verified first, because a run started from the wrong directory tests the primary checkout:

$ .venv/Scripts/python.exe -c "import messagefoundry; print(messagefoundry.__file__)"
C:\Users\Scott\Code\MessageFoundry\.claude\worktrees\agent-a39323af6516fac7b\messagefoundry\__init__.py
check result
ruff check . All checks passed
ruff format --check . 1221 files already formatted
pytest tests/test_durability_hook_provenance.py 8 passed
pytest tests/test_rescue_ref_provenance.py 15 passed (12 pre-existing, 3 new), re-run after the simplify pass
pytest tests/test_coord_unbacked_check.py 12 passed
pytest tests/test_installed_coord_hooks.py 16 passed, 2 failed and PRE-EXISTING -- see note
scripts/docs/backlog_status_check.py OK, 657 items
mypy messagefoundry SKIPPED, and it does not apply -- nothing under messagefoundry/ changed
full suite SKIPPED deliberately. Eleven sessions are live on this box and tests/test_connscale_smoke.py hard-codes a port block that is not parallel-safe across worktrees (BACKLOG #1014)

The two failures, stated rather than waved past. test_the_installed_coord_hook_matches_the_committed_source[claim_check.py] and [push_guard.py] compare the copies installed in the shared .git/hooks/ against the committed source. Both files are byte-identical to main on this branch -- git diff main...HEAD names neither -- so the drift is between this machine's installed hooks and the source, in a worktree that has not re-run install-git-hooks.ps1. The parametrized set comes from the installer's $payloads list, which holds the Python payloads; the sh post-commit hook this PR changes is not in it. I did not re-run the installer, because that writes into the shared git directory eleven live sessions are using.

Legs only a hosted runner can report, and they need reading: the full pytest matrix, and windows-service-smoke. My process exits when this PR opens.

What a reviewer should look at hardest

  1. The fallback path in durability_push.sh. Durability must not regress under any provenance failure, and the tests are ordered that way on purpose.
  2. Whether pushing a tag object by id has a failure mode I have not driven. It was measured working against a local bare remote, including the force-update case.
  3. The claim that no local ref is created. That is the only thing standing between this and an accidental publication path.

🤖 Generated with Claude Code

wshallwshall and others added 6 commits September 3, 2026 17:31
… #1349)

The post-commit hook force-pushed a bare ref recording nothing, so the ref could
only be graded against a branch that still existed -- the population a rescue ref
was never needed for. It now builds an annotated tag object carrying the same
mefor-rescue-v1 message rescue.ps1 -Anchor writes, and pushes it by object id.

No local ref is created. A local annotated tag reachable from a branch tip would
be swept up by git push --follow-tags to whatever remote a hand reaches for, and
the default one is public.

Every new failure mode degrades to a warning. git hash-object fsck-validates the
tag and git var GIT_COMMITTER_IDENT can fail; either way the push falls back to
the bare form, one warning goes to stderr, and the commit stands. Measured cost:
0.19s of CPU per commit over five paired runs.

Detached HEAD omits was-tip rather than guessing it, which is what makes -Check
report SELF-DESCRIBING instead of a claim about a branch that does not exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…or (BACKLOG #1349)

The remedy block printed a bare
    git push --force <remote> <branch>:refs/tags/rescue/branch/<branch>
so an operator following the tool's own advice wrote a ref that records nothing
about what it captured. Measured 2026-09-03 in this checkout: rescue.ps1 -Check
examined 1671 refs and returned UNVERIFIABLE for all 1671, every one of them
written by a bare push of that shape.

The block now runs -Anchor first and pushes the annotated tag it wrote, and says
why the extra step is the remedy rather than decoration.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…KLOG #1349)

-Check read refs/rescue and refs/tags/rescue only, so it graded 1671 refs while
1505 more sat under refs/remotes/private/rescuetags unexamined. That namespace
exists because remote.private.fetch remaps refs/tags/rescue/* into it, so one
server-side tag arrives under a second local name that git tag -l cannot see.

Remote prefixes are derived from git remote rather than hardcoded, since the
remote's name is an operator's choice. Namespaces matching nothing are still
listed at zero, because a silently dropped namespace is how a reader loses the
ability to tell what was examined.

The two mechanisms are kept apart. A snapshot behind its branch is behind it
forever; a push-updated ref behind a LIVE branch may merely be lagging, and the
next commit fixes it. Mechanism changes the wording of BEHIND and DIVERGED and
deliberately changes no verdict: once the branch is gone nothing pushes again,
so the branch-gone arms are equally final either way. Counts are not
deduplicated -- the two names disagreeing is the finding.

EXAMINED goes from 1671 to 3187 in this checkout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eight new arms over the durability hook in tests/test_durability_hook_provenance.py,
three over the widened audit, one over the printed remedy.

The hook arms are ordered durability first, because that is the guarantee the
provenance layer must not move: a broken committer identity degrades to a bare
push with a warning and exit 0, an unreachable remote does not fail the commit,
and an unarmed hook pushes nothing. The sabotage arm disarms the hook while its
fixture commit is made, so the verdict cannot be decided by which of two racing
pushes lands second.

The push is detached, so the remote is polled rather than read once. Nothing
asserts on elapsed time.

The audit arms drive ONE object under TWO names and require the details to
differ: a snapshot behind its branch is behind it forever, a push-updated mirror
may merely be lagging. An empty namespace is still required to appear in the
coverage, since a silently dropped one reads exactly like a clean bill.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… #1349)

Coverage counts are grouped in one pass instead of filtering the row set once
per namespace, which walked 3202 rows six times in the checkout measured.

The mechanism legend is padded by the format string rather than by hand-counted
spaces, so renaming a constant cannot shear the legend away from its table.

The push-updated test is hoisted to one named boolean shared by the BEHIND and
DIVERGED arms.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
All three writer surfaces the 2026-09-03 score named now carry provenance and
the audit reads the namespace it was missing. EXAMINED went from 1671 to 3202
and UNVERIFIABLE with it: widening the audit found more unverifiable refs, it
did not repair any, and the two counts moving together is the correct result.

Recorded as NOT work so nobody schedules it: the snapshot refs cannot be
retrofitted because the information was never captured, and the push-updated
ones heal on the next commit to the branch they track.

The banner stays open. A builder does not flip it -- the closing act is code,
and the LANDER flips the banner on merge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 3, 2026
@github-actions github-actions Bot added the ci-red A required check went red. Attribute it before retrying. label Sep 4, 2026
tests/test_tooling_partition.py::test_every_non_engine_test_is_classified reds
all three required test legs when a test that does not import the engine is
absent from tests/tooling_manifest.txt. This PR adds such a test, so CI could
not go green as it stood.

The manifest is read as a set, so this is a single inserted line at its
alphabetical slot; no existing line moves.
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

Lander: added the missing tests/tooling_manifest.txt line for the test this PR introduces.

tests/test_tooling_partition.py::test_every_non_engine_test_is_classified reds all three required
test legs when a test that does not import the engine is absent from the manifest, and it is not
marked tooling, so -m 'not tooling' does not deselect it. This PR could not go green as it stood.

Verified before pushing: the test fails on the previous head naming this exact file, and passes with
the line added. The manifest is read as a set(), so the change is one inserted line at its
alphabetical slot -- no existing line moves.

Re-applied the reviewed label, which the push stripped.

@github-actions github-actions Bot removed the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall
wshallwshall added this pull request to the merge queue Sep 4, 2026
@wshallwshall
wshallwshall removed this pull request from the merge queue due to a manual request Sep 4, 2026
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

Lander: dequeued temporarily -- this is my error, not a defect in this PR.

819 and 825 each merge CLEANLY against main, but they conflict WITH EACH OTHER in tests/tooling_manifest.txt. The merge queue stacks entries, so 819 built on top of 825 could not form a merge group at all, which showed as UNMERGEABLE with no merge-group runs.

The cause is my manifest repair earlier today: I inserted a line into both pull requests at the same alphabetical slot, between tests/test_claim_check.py and tests/test_gate_ci_mirror_parity.py. Each insert was correct on its own; together they are a textual conflict.

Nothing is wrong with this branch. I will re-arm it once 825 lands, at which point it conflicts with main in the ordinary way and the resolution is to keep both lines.

The conflict is mine, not this branch's. During the manifest repair I inserted a line
into this pull request and into 825 at the SAME alphabetical slot, between
tests/test_claim_check.py and tests/test_gate_ci_mirror_parity.py. Each insert was
correct alone; together they are a textual conflict, which only surfaced once 825 landed.

Both lines are needed, so both are kept, in alphabetical order:
  tests/test_coord_alloc_strand_sweep.py   (from 825, now on main)
  tests/test_durability_hook_provenance.py (this branch's own test)

Verified: tests/test_tooling_partition.py passes all 9, and each path appears exactly once.
@github-actions github-actions Bot removed the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall
wshallwshall added this pull request to the merge queue Sep 4, 2026
Merged via the queue into main with commit aa11f21 Sep 4, 2026
42 of 43 checks passed
@wshallwshall
wshallwshall deleted the fix/1349-rescue-ref-provenance-remainder branch September 4, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-red A required check went red. Attribute it before retrying. reviewed A reviewer has read this. Removed automatically when new commits arrive.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant