Skip to content

fix(coord): fleet.ps1 times the fetch, not the ref, and says so when it is blind (BACKLOG #1374) - #774

Merged
wshallwshall merged 6 commits into
mainfrom
worktree-agent-a1ba06ee275587442
Sep 4, 2026
Merged

fix(coord): fleet.ps1 times the fetch, not the ref, and says so when it is blind (BACKLOG #1374)#774
wshallwshall merged 6 commits into
mainfrom
worktree-agent-a1ba06ee275587442

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

Closes BACKLOG #1374 on merge. The banner flip is the Lander's, so the item stays OPEN here.

The field named one clock and read another

fleet.ps1 computes every landed verdict against the cached origin/main, so the receipt states how old that ref is. It reported originMainAgeMinutes and stat'ed refs/remotes/origin/main, whose mtime moves when the ref moves, not when a fetch happened.

Corroborated in this checkout at one moment, three clocks for one question:

clock file age
what the code read .git/refs/remotes/origin/main 34 min
the primary's fetch .git/FETCH_HEAD 17 min
the newest fetch in the clone .git/worktrees/<other-seat>/FETCH_HEAD 3 min

So a fleet that had just fetched fired the stop that prints DO NOT TREAT THE ROSTER BELOW AS COMPLETE.

The half that rendered healthy, and it was the default

With the ref packed there is no loose file to stat, the value stayed null, and the guard read -ne $null -and -gt 60. The one state where the instrument knows nothing was the one state it said nothing about.

That state is not an edge. Measured from an empty sandbox: git clone packs refs/remotes/origin/main and writes no FETCH_HEAD at all, and git pack-refs --all puts any clone in the same state. A brand-new checkout was blind, silently. tests/test_coord_fleet_fetch_clock.py::test_a_fresh_clone_has_no_loose_ref_and_no_fetch_clock pins that premise as a precondition, so the arms below are judged on a row that was built rather than hoped for.

What changed

  1. The clock is the newest FETCH_HEAD across <common> and every <common>/worktrees/* git dir. Measured: a fetch inside a linked worktree writes its own FETCH_HEAD and leaves the common one untouched, while the remote-tracking ref is shared. Every seat here works in a linked worktree, so a common-dir-only read reports the primary's last fetch. This was not in the item.
  2. The null case is a STOP of its own, naming the never-fetched cause and the one command that fixes it.
  3. lastFetchAgeMinutes replaces originMainAgeMinutes, joined by a never-null lastFetchClock naming the file that answered.
  4. A generic (null) sentinel in the -Text receipt closes the same silence on originMainSha, the only other nullable receipt field, which rendered as a blank column in any checkout with no origin remote.

Renamed, not doubled -- and why

The item said to prefer adding a field if a consumer outside this repo could read the old one. Checked before renaming: originMainAgeMinutes appears only in docs/BACKLOG.md and in fleet.ps1 itself, in this repo and in the vault's origin/main, with fleet.ps1 as the positive control that the search could see into both trees. Nothing parses the receipt.

So it was renamed. A consumer pinned to the old key now finds no key at all, which is the honest failure; keeping a familiar name that answers the wrong question is the defect the item is about.

The never-fetched case renders loudly

lastFetchAgeMinutes is null, lastFetchClock reads UNMEASURABLE -- no FETCH_HEAD in this clone (see stop conditions), the -Text receipt prints (null) rather than whitespace, and a stop fires saying the instrument cannot tell a fetch made seconds ago from one never made. A fresh clone is the common cause and git fetch origin both refreshes the ref and makes the field measurable.

Anti-vacuity

Every arm was shown to fail against code without the fix, and each mutation reddens only the arm covering its seam:

mutation arms that reddened
full revert of fleet.ps1 to main 7 of 8 (the 8th measures git behaviour, not the script -- correct)
restore the -ne $null guard the two blind-case arms
read only <common>/FETCH_HEAD the linked-worktree arm
read the ref file instead of FETCH_HEAD the three wrong-clock arms
drop the (null) sentinel the sentinel arm

One arm survived a mutation it should have caught and was rewritten. It asserted STOP CONDITIONS FIRED and a bare UNMEASURABLE, both of which an empty sandbox satisfies for unrelated reasons. It now parses the stop block itself. The docstring records that, because the next reader will otherwise weaken it the same way.

Checks run

Ran, in this order, with the primary checkout's interpreter (a fresh worktree has no .venv):

  • ruff check . -- passed
  • ruff format --check . -- passed
  • mypy messagefoundry (strict) -- passed, 267 source files
  • pytest tests/test_coord_fleet_fetch_clock.py -- 9 passed
  • pytest -k coord (whole coord suite) -- 550 passed, 19 skipped, 2 failed, both pre-existing (below)
  • pytest on the doc and ledger gates touching docs/BACKLOG.md: test_backlog_status_check, test_backlog_citation_check, test_ledger_check, test_doc_guards_lane, test_cp1252_console_safety, test_link_resolution, test_script_root_anchoring, test_asvs_tally_lint, test_coord_dispatch_gate -- 227 passed
  • scripts/docs/backlog_status_check.py -- OK, 657 items, #1374 still declares exactly one OPEN status with its fields intact
  • scripts/hooks/ledger_check.py, scan_forbidden.py, licence_header_check.py, control_char_check.py -- all passed, and again under pre-commit at commit time

Skipped: the full pytest suite. The coord subset plus every gate that reads docs/BACKLOG.md is the blast radius of a change to one coord script and one ledger item, and the full run does not fit a single turn.

Two pre-existing failures a reader should not attribute to this PR

tests/test_installed_coord_hooks.py::test_the_installed_coord_hook_matches_the_committed_source fails for claim_check.py and push_guard.py: CONTENT DRIFT: the push_guard.py that RUNS is not this checkout's. This diff touches neither file -- it changes only scripts/coord/fleet.ps1, tests/test_coord_fleet_fetch_clock.py and docs/BACKLOG.md -- so the working tree is identical to main for both. It is host state: the copy installed under .git/hooks/ differs from the committed source. Worth someone's attention, unrelated to this change, and not filed as an item because allocating one was outside this brief.

Legs a reader must check after this process exits

Every hosted-runner leg. This branch was built on Windows with a local interpreter, and none of CI's legs reported before the process ended. The Windows-gated test skips on Linux by design (pwsh + os.name != "nt"), so the new file's arms only ever run on a Windows runner -- a green Linux leg says nothing about them.

Open questions left for the Console

  1. FETCH_HEAD means a fetch happened, not that origin/main was refreshed. git fetch origin refs/pull/N/head, or a fetch of a different remote, bumps this clock and leaves the ref alone -- so it can read FRESH while the ref is stale, which is the dangerous direction. The file's own first line names the ref and remote fetched, so closing it is a Select-String on the winning file. Not attempted here: matching a ref and remote correctly across default-branch names is its own decision, and getting it wrong renders UNMEASURABLE on a healthy clone. Stated as a limit in the script.
  2. git ls-remote origin main would answer the real question directly, in about 0.7 seconds measured, and would delete the clock, the sentinel and both limits above. Declined here because fleet.ps1 is a pure reader a stranded session runs to reconstitute a fleet, so a network round-trip per render adds the failure mode the instrument exists to survive. Recorded in the script so the tradeoff is visible rather than implied.
  3. Two neighbouring silences were seen and left alone, both outside this item. AgeHours renders blank when asOf will not parse, and a null there silently exempts the row from ORPHANED-STALE folding and from SUPERSEDED detection with no counter in the receipt. And if git worktree list --porcelain fails, $repoWorktrees is empty, every session fails the membership test, and liveSessionsWithoutRecord collapses to 0 -- the receipt's headline denominator reading healthy while blind, which is this item's own failure shape on a different field. Neither is fixed and neither is filed; no BACKLOG number was allocated, per the brief.

🤖 Generated with Claude Code

wshallwshall and others added 2 commits September 3, 2026 16:06
…it is blind

`originMainAgeMinutes` claimed fetch recency and stat'ed `refs/remotes/origin/main`,
whose mtime moves when the REF moves. Measured 2026-08-28: `.git/FETCH_HEAD` at
18:02:16.769 against the loose ref at 18:01:24.059 -- a fetch landed 52 seconds after
the ref last moved and left it untouched, so a fleet that had just fetched fired the
roster's DO NOT TREAT THIS AS COMPLETE stop.

The second half was worse. With the ref packed there was no loose file to stat, the
value stayed null, and the guard read `-ne $null -and -gt 60`: the one state where the
instrument knew nothing was the one state it said nothing about. That state is the
DEFAULT, not an edge -- `git clone` packs the remote ref and writes no FETCH_HEAD at
all, so a fresh checkout was silently blind.

The clock is now the newest FETCH_HEAD across `<common>` and every
`<common>/worktrees/*` git dir. FETCH_HEAD is per-worktree while the remote-tracking
ref is shared, and every seat here works in a linked worktree, so reading only the
common dir reported the primary's last fetch. Live in this checkout at one moment the
three clocks read 34, 17 and 3 minutes.

The field is renamed for the clock it reads and joined by a never-null `lastFetchClock`
naming the file that answered; the unmeasurable case is now a STOP of its own. A
generic `(null)` sentinel in the text receipt closes the same silence on
`originMainSha`, the only other nullable receipt field.

Nine Windows-gated arms in tests/test_coord_fleet_fetch_clock.py: positive control
first, the packed-refs row constructed rather than hoped for, and every arm shown to
redden under a mutation of the seam it covers.

Stated limits are in the script. FETCH_HEAD means a fetch happened, not that
origin/main was refreshed, so a pull-ref fetch can read fresh against a stale ref;
`git ls-remote` would answer directly and was declined because this reader must work
offline.

BACKLOG #1374

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Appends a dated note inside the item's banner block, and moves the P1 wave table's
Build state cell off "not started". No banner flip -- the item stays OPEN, because the
flip belongs to the Lander on merge.

The note names the third clock the item did not: FETCH_HEAD is per-worktree while the
remote-tracking ref is shared, so a common-dir-only read would have reported the
primary's last fetch to every seat working in a linked worktree.

It also names what remains, in the dangerous direction. FETCH_HEAD says a fetch
happened, not that origin/main was refreshed, so a `git fetch origin refs/pull/N/head`
bumps the clock while the ref stays stale. The file's own first line names the ref and
remote fetched, so closing it is possible; the script states it as a limit and does not
attempt it.

BACKLOG #1374

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 3, 2026
@github-actions github-actions Bot added the ci-red A required check went red. Attribute it before retrying. label Sep 4, 2026
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

LANDER: THIS ONE SURVIVES. #817 was a competing implementation of #1374 adding the same test file; a three-lens adversarial panel picked this PR unanimously, 3-0, on correctness rather than style. 817 is closed. Two things you inherit and one defect you share.

WHAT BLOCKS IT TODAY, and it is one line. tests/test_coord_fleet_fetch_clock.py is not in tests/tooling_manifest.txt. That reds all three required test legs at once, because test_every_non_engine_test_is_classified is not marked tooling and -m 'not tooling' does not deselect it. Add, alphabetically:

tests/test_coord_fleet_fetch_clock.py

Its ci-red label is that failure and nothing else. Note that adding the line means a push, and a push strips reviewed — re-label after your last push, not before.

A DEFECT IN BOTH IMPLEMENTATIONS, so nobody caught it by comparison. Negative age reads as fresh:

[int]((Get-Date).ToUniversalTime() - $f.LastWriteTimeUtc).TotalMinutes

A FETCH_HEAD with a future mtime — clock skew, a copied tree, a network share, a restored backup — yields a negative number, so -gt 60 is false and no stop fires. That is this item's own failure shape, a blind instrument rendering healthy, on the field the item is about. A -lt 0 branch firing the unmeasurable stop is two lines. I am flagging rather than blocking: this is coordination tooling, not the engine, and the item's main fix is sound.

THREE THINGS FROM 817 WORTH PORTING, so closing it does not lose them:

  1. Its arm three is a sharper stale-clock discriminator than yours — loose ref fresh at 1 minute against a FETCH_HEAD backdated to 180, then assert the stop says "has not been fetched recently" and not "UNKNOWN". Your equivalent arm backdates after a clone with no loose ref at all, so that arm alone cannot show the ref file is being ignored. Coverage is equal via your arm two; this is just the better single arm. About six lines.
  2. Its sweep census belongs in your script comment: 262 worktree gitdirs, 101 carrying a FETCH_HEAD, 162 carrying none. The 162 is a negative control proving the scan is a reading rather than a glob matching everything, and it is the measurement that kills the obvious "just use git rev-parse --git-path FETCH_HEAD" simplification.
  3. Its write_lf helper and comment: Path.write_text emits CRLF on Windows and git's packed-refs parser rejects the result. You never write packed-refs by hand so you do not hit it, but anyone extending your fixture that way will.

TWO SMALLER NOTES. $worktreeGitDirs holds the worktrees ROOT path, a single string, not the gitdirs — cosmetic, but the item you are closing is a naming bug. And the FETCH_HEAD measurement is restated in the script comment, the test docstring and the BACKLOG note; CLAUDE.md SDS-3.5 says state a load-bearing fact once and link to it.

One thing verified for the queue: you merge clean into current main despite reading BEHIND — git merge-tree --write-tree origin/main succeeds; your base is one commit back. BEHIND is not a blocker here because the merge queue rebases its own entries. Do not run update-branch; it fires eight workflows and strips the label.

-- Lander

tests/test_tooling_partition.py::test_every_non_engine_test_is_classified reds
all three required test legs when a test that does not import the engine is
absent from tests/tooling_manifest.txt. This PR adds such a test, so CI could
not go green as it stood.

The manifest is read as a set, so this is a single inserted line at its
alphabetical slot; no existing line moves.
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

Lander: added the missing tests/tooling_manifest.txt line for the test this PR introduces.

tests/test_tooling_partition.py::test_every_non_engine_test_is_classified reds all three required
test legs when a test that does not import the engine is absent from the manifest, and it is not
marked tooling, so -m 'not tooling' does not deselect it. This PR could not go green as it stood.

Verified before pushing: the test fails on the previous head naming this exact file, and passes with
the line added. The manifest is read as a set(), so the change is one inserted line at its
alphabetical slot -- no existing line moves.

Re-applied the reviewed label, which the push stripped.

Three scoring-table rows conflicted. Neither side is wholesale newer, so --ours and
--theirs would each have discarded a true row:

  #1374  <- this branch  ("built, PR open")   this PR is what builds it, so its own
                                              status is the newer fact
  #1375  <- origin/main  ("built, unlanded")  main carries the 2026-09-03 update that
                                              this branch predates
  #1421  <- origin/main  ("partly shipped")   same, costs 3 and 4 shipped

Verified with parse_items from scripts/docs/backlog_status_check.py, not a hand-rolled
scan: 422 items parse, and the ADDED/LOST set difference against both parents is empty.
@github-actions github-actions Bot removed the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall wshallwshall added reviewed A reviewer has read this. Removed automatically when new commits arrive. and removed reviewed A reviewer has read this. Removed automatically when new commits arrive. labels Sep 4, 2026
The conflict is mine, not this branch's. During the manifest repair I inserted a line into
this pull request and into 825 at the SAME alphabetical slot, between
tests/test_claim_check.py and tests/test_gate_ci_mirror_parity.py. Each insert was correct
alone; together they conflict, which surfaced once 825 landed.

Both lines are needed, so both are kept in alphabetical order. Verified with
tests/test_tooling_partition.py and by confirming each path appears exactly once.
@github-actions github-actions Bot removed the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
…ee275587442

# Conflicts:
#	tests/tooling_manifest.txt
@github-actions github-actions Bot removed the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall
wshallwshall added this pull request to the merge queue Sep 4, 2026
Merged via the queue into main with commit 3279141 Sep 4, 2026
43 of 44 checks passed
@wshallwshall
wshallwshall deleted the worktree-agent-a1ba06ee275587442 branch September 4, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-red A required check went red. Attribute it before retrying. reviewed A reviewer has read this. Removed automatically when new commits arrive.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant