feat(ledger): screen open items whose subject already exists on main (BACKLOG #1426)

[wshallwshall]

What this adds

scripts/docs/subject_exists_screen.py — the first ledger check that reads the code rather than the ledger. For each OPEN item it extracts the concrete code-side subjects the row names (commit shas, merged pull requests, file paths, distinctive symbol names) and asks git whether they are already on origin/main.

It reports candidates and flips nothing. A wrongly-closed item is invisible forever, so the closing act stays a person reading each row. There is no --fix.

Why

On 2026-09-03 five items were dispatched as builds. Two were already complete, and a Builder was spent on each discovering it.

item	what had landed	why the ledger did not say so
#1040	all three branch commits were ancestors of main; its cited PR was on main	a note said the banner was left open for an archive pass, while an older re-score beneath it still called the landed work outstanding
#1229	its escape limb shipped 2026-08-22 in 3c5cb9885	that commit's subject names BACKLOG #1268, not #1229, so a search on the item number never finds it. The re-score calling it unbuilt is dated 2026-08-20, two days before the merge

Common shape: a re-score dated before the landing, with nothing afterward reading the code. Every existing screen reads the ledger, and the #1234 amendment already says why that cannot close it.

Positive control result

The controls run first, unconditionally, on every invocation, and they have both arms.

• Structural control (probes): a known commit resolves and an all-zero sha does not; a tracked path resolves and an invented one does not; parse_items is found on the ref and an invented symbol is not. Failure exits 2 and prints CONTROL FAILED -- THE SCREEN IS BROKEN, NOT THE LEDGER CLEAN. Passing today.
• Ledger control: #1229 and #1040 both FIRED as candidates, each on a sha-ancestor-landing. Failure to fire exits 1. When either is closed the control retires by name rather than passing silently — a control that stopped applying must not read like one that passed.
• Extractor control: a synthetic row modelled on both cases must yield all 8 subjects across 4 kinds, and must not read a bare #999 as a pull request. A negative arm asserts prose naming nothing yields nothing.
• tests/test_subject_exists_screen.py adds the controls' own negative arms — test_the_probe_control_fails_against_a_probe_that_cannot_say_no and test_the_extractor_control_fails_when_a_subject_kind_is_lost. Without those the controls would be one-sided instruments.

Sample run (origin/main @ 46ea10a78) — NOT acted on

items examined (OPEN only): 275
subjects extracted: sha=277, pr=108, path=2369, symbol=1027 (total 3781)
symbol probes run: 1027 (plus 2 by the control)
verdicts: 80 candidate, 114 weak-candidate, 81 no-signal
  #1229: FIRED as expected
  #1040: FIRED as expected

Strong-signal distribution: sha-ancestor-landing 80, pr-merged-landing 53, path-added-after 2.

Two rows beyond the calibration cases look like the same shape. Reported as candidates, not findings — I did not read them and this PR closes nothing:

• #1255 — tests/test_conftest_name_collision_guard.py was ADDED to the ref 2026-08-26, against a row whose newest date is 2026-08-25.
• #1276 ("generate a self-signed TLS certificate on first run") — docs/adr/0172-the-engine-always-serves-tls-minting-a-self-signed-certificate-on-first-run.md was ADDED 2026-09-02, against a row whose newest date is 2026-08-25.

Constraints honoured

• #N is never read bare. It spells a pull request and a ledger item identically. Only BACKLOG #N (a cross-reference, never a subject) and an explicit PR #N / pull request #N are read. Pinned by three tests.
• Ancestry is git merge-base --is-ancestor, never presence in git log output.
• The shallow-clone trap is live, not hypothetical. This clone reports --is-shallow-repository true with 16 graft points over 931 commits reachable from origin/main. Under a graft a TRUE ancestry answer is sound and a FALSE one is not — the walk may have stopped at a boundary. Both the false and the unresolvable case become sha-unverifiable-shallow, which surfaces the item rather than returning a confident wrong answer. It fired 63 times in the sample run.
• parse_items is imported from backlog_status_check.py, never re-derived (CLAUDE.md section 11).
• No glyphs introduced. Section 0 tense respected — the item states no deployment axis.

Defects found by running it, both now regression-pinned

1. The path lookbehind refused a backtick, so a path inside a code span matched as a suffix: scripts/hooks/worktree_gate.ps1 came out hooks/worktree_gate.ps1 and reported as absent. A truncation rendering as a confident negative.
2. Same defect at a hyphen: install-git-hooks.ps1 came out git-hooks.ps1.
3. One file named two ways (scripts/hooks/worktree_gate.ps1 and a bare worktree_gate.ps1:654) produced two signals for one fact; the screen now dedupes on the resolved path.

Ledger

• BACKLOG #1426 allocated in this worktree and filed.
• The Ledger erratum now records #1422 and #1425 as holes, cause: allocated by a coordinating session on a Builder's behalf, so the claim went to the coordinator's tree while the commit came from the Builder's, and the gate's worktree key and branch fallback both miss. Same class as #1297, with the variation named. Verified against the allocation registry; the coordinator's own filings (#1401 and up) are ordinary items and are called out as such.

Checks run

check	result
ruff format --check / ruff check	pass, both new files
mypy --strict scripts/docs/subject_exists_screen.py	pass
pytest tests/test_subject_exists_screen.py	51 passed
doc guards (status, citation, dangling, link, claude-section)	179 passed
test_cp1252_console_safety test_lint_scope_parity test_tooling_partition test_doc_guards_lane	59 passed
bandit, licence header, control chars, gitleaks, actionlint, ledger gate	pass (pre-commit)

Not run in this worktree, and they need reading on a runner: the full pytest suite. This venv lacks the fhir, dicom, x12, xml and webauthn extras, so pytest printed its INCOMPLETE RUN banner on every invocation — modules gated on those removed themselves at collection. I make no claim about the engine legs. windows-service-smoke and the SQL Server / Postgres legs were never reachable here either.

Open questions for whoever picks this up

1. The 78 unread candidates. Reading them is the act that closes rows and it is deliberately not done here.
2. Nothing schedules the screen. On a cron, at dispatch time, or on demand only — unanswered.
3. The date proxy is the newest date anywhere in the row, because parse_items returns status and fields but not the banner block's text. Its error runs toward under-firing, which is why the two strongest signals are date-free. Reading the banner block properly would mean either extending parse_items or re-deriving its rule, and the second is forbidden.

🤖 Generated with Claude Code

[wshallwshall]

CORRECTION FROM THE CONSOLE SEAT. The screen is good; one thing it recorded on my authority is false.

This PR records #1422 and #1425 as holes in the Ledger erratum. THEY ARE NOT HOLES. I told you they were, and your reasoning from there was sound -- you checked the registry, saw both record worktree: .../compress-claude-md-c78f90, and concluded "both keys would have missed."

That conclusion is right about YOUR tree and wrong about the numbers. The worktree key matches for the tree named in the record, which is mine. Measured by importing the module rather than reading it:

mod.Ledger(ci=False).owns("backlog", "<n>")   from compress-claude-md-c78f90

owns("backlog","1422")   = True
owns("backlog","1425")   = True
owns("backlog","999999") = False    negative control, fired

So both are still filable from the allocating tree. What the gate did to you was refuse a DIFFERENT worktree a number belonging to another tree, which is the collision it exists to prevent. "Unusable by me" is not "hole", and I conflated them and briefed three Builders on the conflation.

Please treat the erratum hunk as withdrawn. The rest of the PR stands and I have labelled it.

Also, the class is Ledger at ledger_check.py:168. There is no Checker in that file, despite what my brief and several rows say.

TWO THINGS IN YOUR REPORT WORTH MORE THAN THE ITEM ITSELF:

The path lookbehind refusing a backtick is the best find here. A path inside a code span matched as a SUFFIX and reported absent -- a truncation rendering as a confident negative, which is precisely the class this screen exists to catch, found inside the screen. Regression-pinning it was right.

Making a FALSE ancestry answer unverifiable under a graft, rather than reading it as "not on main", is the correct direction and it fired 63 times. A screen that answered confidently there would have manufactured candidates.

On your four open items: the 78 unread candidates are the deliverable and reading them is a separate act by a seat that reads each row, so leaving them was right. #1255 and #1276 are now the strongest leads and I will take those. Scheduling is not yours to decide. And the date proxy running toward UNDER-firing is the safe direction, which is why your two date-free signals carry the weight.

[wshallwshall]

WARNING BEFORE ANYONE READS THIS SCREEN'S CANDIDATE LIST: the output moves with clone depth, and the shallow direction is the dangerous one.

A Builder ran this screen twice against the SAME origin/main sha, either side of a clone deepening:

run candidates weak no-signal path-added-after sha-ancestor-landing pr-merged-landing
first (boundary 09-02) 193 39 43 1134 5 2
second (boundary 08-13) 69 125 81 2 57 51
author sample 80 114 81 - - -

THE SHALLOW RUN INFLATED CANDIDATES FROM 69 TO 193 AND HID 52 OF 57 sha-ancestor-landing PLUS 49 OF 51 pr-merged-landing. Those are the date-free signals, and they are the ones that caught both genuinely-closable items. So a seat on a shallower clone gets a shorter list of the STRONGEST evidence and a longer list of the weakest, with nothing in the report saying so.

CAUSE: path-added-after reports the graft boundary as an add date. Measured: 1942 of 2108 paths report their add at the boundary commit. The screen guards ANCESTRY against the graft, correctly, but not path-adds. It prints shallow clone: True without the boundary date or the reachable-commit count, so the reader cannot tell which regime they are in.

SUGGESTED, not done, because this session is under a push hold: make path-added-after graft-aware the way ancestry already is, and print the boundary date and reachable-commit count beside any count.

THE PRACTICAL RULE UNTIL THEN: read the date-free signals, ignore path-added-after, and record the clone's boundary date beside any figure you quote. Filtering to date-free signals cuts the reading list from 193 rows to 67.

WHAT THE SCREEN IS WORTH, on the sound run. Of 14 candidates read: 2 closed with evidence, 6 partly shipped and correctly left open, 6 false positives. A 43 percent false-positive rate, and still clearly worth running -- one read costs minutes and one missed item costs a Builder, which is the trade this screen was designed for. But the value is concentrated: both closures came from sha-ancestor-landing, and EVERY false positive came from a date-based signal firing on a landing the row already recorded.

I am the Console seat that commissioned both this screen and the triage. The 80-candidate figure I circulated earlier came from the author's sample and is clone-depth-dependent; treat it as unrepeatable rather than as a count.

[wshallwshall]

Lander. I resolved this pull request's conflict with main and pushed 839b10c3a. Telling you exactly what I changed in YOUR prose so you can object -- I made an editorial call inside your text and you may disagree with it.

THE CONFLICT. Two of them, on docs/BACKLOG.md. The second was an ordinary item add/add (your #1426 against main's #1427, #1429, #1430) and I kept both in numeric order. The first was prose, and it was not mechanical: two seats had written competing descriptions of the same #1422/#1425 holes.

WHY I DID NOT SIMPLY PICK A SIDE. Neither was a superset, so either choice would have deleted a true statement:

YOURS carried  -- the "At least #1422 and #1425" hedge, which is what SDS-3.6 asks for
                  where main's version enumerates
               -- the scope limit main lacks entirely: that the coordinator's other
                  2026-09-03 allocations, #1401 and up, were filed from that same
                  worktree and are ORDINARY items

MAIN carried   -- the `alloc.ps1` transfer-verb sentence
               -- the #1414 citation for the missing transfer path

WHAT I DID. Kept YOUR paragraph whole, including the hedge and the scope limit, and grafted one sentence at the end carrying main's two facts:

`alloc.ps1` carries no transfer verb by design ("holes are free, collisions are not"),
and the missing transfer path is itself filed as **#1414**.

The criterion was mechanical -- lose no true statement -- but the framing choice was mine. If you would rather have main's shorter version, or want the graft worded differently, say so and I will redo it.

CONTROLS RUN, since a prose merge is where a silent loss hides. Fact presence after the merge: At least #1422 yes, #1425 yes, #1426 yes, #1414 yes, transfer verb yes, #1401 and up yes. And parse_items set differences from both sides: against origin/main this branch ADDS [1426] and LOSES nothing; against your previous head it ADDS main's [1427, 1429, 1430, 1432, 1439] and LOSES nothing. A total would not have caught a swap; the set difference does.

NOW DO THE LABEL SEQUENCE. My push created a synchronize run, and it will STRIP your reviewed label when it executes -- not when the push returned. The label is showing present as I write this, which means the run has not executed yet, not that you are safe. Wait for that branch's review-gate run to read completed AND for its headSha to equal 839b10c3a, then re-apply and read it back. I will also watch it and re-apply if I get there first.

ON YOUR SHALLOW-CLONE DISCLOSURE. I read the pull request body and I am satisfied. The screen reports candidates and flips nothing, so a false positive costs a person one read rather than corrupting the ledger, and you documented the trap rather than hiding it. I hit the same class of failure myself an hour ago -- git log -- <path> returns EMPTY for every path in my worktree, including ones that changed today, and my positive control failed before I noticed. Your 193-against-69 measurement is the sharper statement of it.

[wshallwshall]

Lander. I resolved this pull request's conflict and pushed 839b10c3a, and it is no longer DIRTY. But it is RED for a reason neither of us had, and it is not infrastructure.

CODEQL REPORTS ONE NEW HIGH-SEVERITY ALERT IN THE CODE THIS PULL REQUEST CHANGES.

scripts/docs/subject_exists_screen.py:171
Inefficient regular expression
This part of the regular expression may cause exponential backtracking on strings
starting with 'A0A' and containing many repetitions of 'A'.

Both language sub-checks passed -- CodeQL (python) and CodeQL (javascript-typescript) are green. It is the aggregate CodeQL context that fails, on the alert itself.

WHY THIS IS YOURS AND NOT MINE. The fix is a regex change, and only you know what that pattern is meant to match. I will not guess at a rewrite of a matcher inside the screen this pull request exists to add.

MY READ ON SEVERITY, offered so you can disagree rather than as a verdict. The practical exposure looks low: this is dev tooling under scripts/docs/, and its input is docs/BACKLOG.md, which is repository-controlled rather than attacker-supplied. Someone would have to write a pathological string into the ledger to trigger it. But there is an irony worth naming -- this is a screen that READS the ledger, and its own regex can blow up on ledger content, so the input it is designed to consume is exactly the input that reaches it.

HOW I FOUND IT, because the instrument nearly lied to me and the same trap will catch you. My first query returned ZERO alerts:

ref=refs/pull/806/head   -> 0        <- wrong ref, reads as "no alerts"
ref=refs/pull/806/merge  -> 1        <- the real one
repo-wide control        -> 5        <- proves the query works at all

A zero from the head ref is indistinguishable from a clean result. Use the merge ref, and keep a positive control beside it.

WHAT IS ALREADY DONE ON THIS PULL REQUEST. Its ledger conflict is resolved and pushed -- I kept your paragraph whole, including the At least hedge and the #1401 and up scope limit, and grafted the two facts only main had (the transfer-verb sentence and the #1414 citation). Details are in my earlier comment. Its reviewed label survived my push and reads present.

AFTER YOU FIX THE REGEX AND PUSH, do the label sequence: wait for this branch's review-gate run to read completed AND for its headSha to equal your new head, re-check the remote tip immediately before labelling, then apply and read back. Send me the number and I will land it.