Skip to content

docs(testing): mark the vaulted tree so absence stops reading as evidence (BACKLOG #1435) - #831

Merged
wshallwshall merged 4 commits into
mainfrom
builder/1435-vaulted-path-marker
Sep 4, 2026
Merged

docs(testing): mark the vaulted tree so absence stops reading as evidence (BACKLOG #1435)#831
wshallwshall merged 4 commits into
mainfrom
builder/1435-vaulted-path-marker

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

Closes nothing. BACKLOG #1435 stays open — the gate half is deliberately not built, and why is the substance of this PR.

The problem

An item body cites a path under docs/testing/ or docs/security/ as though it were local. Those trees are gitignored — /docs/security/ is a blanket rule and docs/testing/ was tracked at exactly one file — so the real documents live in the separate vault checkout. From an engine checkout, work that is UNREADABLE is indistinguishable from work that is UNSTARTED. A reader greps, finds nothing, and concludes it was never done.

Measured three times in one day across two independent fleets: BACKLOG #1152 was about to claim it had satisfied a master-test-plan exit criterion it could never read; a peer fleet's #1193 found a sub-limb unreachable rather than undone; a peer fleet's #1191 found half an availability strategy vaulted.

What shipped

  • docs/testing/README.md, tracked via a second negation under /docs/testing/*. It says the material moved and that absence is not evidence, and tells a grader to record such a limb as unverified — never as satisfied, never as missing.
  • docs/README.md carries the same warning at the front door.
  • tests/test_private_paths_stay_ignored.py pins the tracked pair as an exact set, so the marker cannot silently stop reaching fresh clones. Positive control: the pin failed before the marker was staged and passed after, so it genuinely reads git ls-files.
  • A correction at docs/BACKLOG.md 4763/4771. #1011 step 6 called a vault-only file "the only place in the repo that records the true state" — false from any checkout. Both citations now name the unreadability in the same breath as the path, which is the remedy shape rather than a deletion.

The gate is not built, and that is the finding

Granularity Cite Unflagged
Per line 55 23
Per item body 27 2
Per item + master-test-plan 32 5

Three reasons, each sufficient:

  1. Unscoped it reds about 22 times on day one, against a corpus nobody fixes in one PR.
  2. The quiet variant measures the wrong thing. At 2 unflagged, item-granularity would ship comfortably — and it misses #1152, the motivating case, because that body flags the scorecard while leaving its master-test-plan clause unflagged. A gate quiet because it is aimed slightly wrong is worse than none: it converts an open problem into a false all-clear.
  3. "Cites a vaulted path" overfires on prose about the arrangement and on count vocabulary, both of which CLAUDE.md permits. The real predicate is a citation a reader must follow in order to act — a judgement about intent that no pattern tried here expressed.

Both controls are written into #1435 so a future builder does not re-derive them: the fires side at 4763/4771 (recoverable from 024de5b68), and the quiet side in PRs 804 and 812, whose bodies carry the remedy shape and are not yet on main.

Disclosure

Nothing enumerates the vault. The marker names no vaulted document; the item names only what the public ledger already carries. A gate's failure message would be a disclosure surface too — naming the offending path would publish the map into CI logs and the workflow file, both public — and that constraint is recorded in the item.

Checks

backlog_status_check OK (658 items). link_check OK (5000 links). 182 passed across seven doc and ledger guard files. ruff check and ruff format --check pass; all pre-commit hooks green on all three commits. Interpreter provenance verified to read this worktree. Full suite and hosted-runner legs left to CI — no engine code changed, the only Python edit is one test file.

🤖 Generated with Claude Code

wshallwshall and others added 4 commits September 3, 2026 18:22
…ence (BACKLOG #1435)

From an engine checkout `git ls-files docs/testing` returns one file, so a path
cited under that tree is indistinguishable from work that was never done. A
reader greps a cited path, finds nothing, and concludes the document does not
exist -- when it does, in the vault clone.

Measured 2026-09-03 against docs/BACKLOG.md: #1011 step 6 tells a reader to mark
a disposition discharged in a path under docs/testing/ and calls that file "the
only place in the repo that records the true state". That sentence is false from
any engine checkout, because the file is not in the repo at all.

Adds docs/testing/README.md as the checkout-side marker. It is a POINTER and
never an index -- it names no vaulted document, because a path-to-document map
over a closed set discloses what is NOT covered by subtraction. The
docs/security/ half already has its rule in SECURITY-DOCS-POLICY.md, so the
marker links there rather than restating it.

The marker needs a second negation under /docs/testing/*, so _TRACKED_EXCEPTIONS
moves from one file to an exact set of two. Its negation test is parametrized
over both, and a reverse-direction assertion keeps a third file from travelling
without a written decision. While there: that test's docstring claimed
WIN2025-TEST-PLAN.md "carries the host and its service identity"; .gitignore
already records that correction, so the docstring now matches it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ct on a file the repo does not contain

#1011 step 6 instructed a reader to mark SEC-71 discharged in a path under
docs/testing/ and called that file "the only place in the repo that records the
true state". The file is not in the repo at all -- docs/testing/ is gitignored
except VERIFY.md -- so the sentence is false from any engine checkout, and a
builder acting on it would have claimed a criterion it could never read. This is
the cleanest measured instance of the class BACKLOG #1435 describes: a citation a
reader must FOLLOW in order to act, naming a document that is unreadable here.

Both citations in that item now name the unreadability in the same breath as the
path, which is the remedy shape rather than a deletion -- the citation is real
and carries provenance, so removing it would lose more than it fixes.

docs/README.md gains the same warning at the front door: a path you cannot open
is not proof the work was never done. It points at the two rules rather than
restating either, and it enumerates nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…t stopped short (BACKLOG #1435)

Files the item for the defect the marker addresses, and records the gate
research so nobody re-derives it.

The gate was the first choice and is deliberately NOT built. Three findings, each
enough on its own to stop it, all measured against 46ea10a:

  1. An unscoped gate reds about 22 times on day one, against a corpus nobody
     repairs in one pull request.
  2. The quiet item-granularity variant measures the wrong thing. At 2 unflagged
     it would ship comfortably, and it misses #1152 -- the case that motivated
     the work -- because that body flags a different vaulted document while
     leaving its master-test-plan clause unflagged.
  3. "Cites a vaulted path" overfires on prose about the arrangement and on count
     vocabulary. The real predicate is a citation a reader must FOLLOW in order
     to act, and no pattern tried here expresses that judgement.

The item names both controls a future builder needs, because a gate proven on one
side proves nothing. Fires side: BACKLOG lines 4763 and 4771 before this branch
corrected them. Quiet side: the remedy bodies on PR 804 and PR 812, which were
unmerged on 2026-09-03, so calibrating against main alone shows the unflagged
versions and teaches the wrong lesson.

It also records that a gate's failure message is itself a disclosure surface: a
message naming which paths are vaulted publishes the map into public CI logs.

Severity is stated for a not-deployed beta. This is a documentation and process
defect, and no claim in the item is present-tense about a running site.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both sides only APPEND items to docs/BACKLOG.md, so nothing was dropped
and nothing was rewritten. Main added #1424, #1427, #1429, #1430, #1431,
#1432 and #1439; this branch added #1435. The resolution keeps every one
and orders them numerically, placing #1435 between #1432 and #1439.

Verified with the ledger's own parser rather than by reading: zero
conflict markers remain, #1435 appears exactly once and still declares
OPEN, and backlog_status_check reports 665 items each declaring exactly
one status. That check matters because a conflicted ledger parses
WITHOUT error and yields a census counting both sides.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall
wshallwshall added this pull request to the merge queue Sep 4, 2026
@github-actions github-actions Bot added the ci-red A required check went red. Attribute it before retrying. label Sep 4, 2026
Merged via the queue into main with commit b9f4f38 Sep 4, 2026
44 of 45 checks passed
@wshallwshall
wshallwshall deleted the builder/1435-vaulted-path-marker branch September 4, 2026 23:03
wshallwshall pushed a commit that referenced this pull request Sep 4, 2026
A clean fast merge with no conflicts: #831 touches .gitignore, docs/README.md,
docs/testing/README.md, tests/test_private_paths_stay_ignored.py and a
different region of docs/BACKLOG.md, none of which this branch edits. It
landed while the conflict resolution below was being measured. Merged rather
than left behind so the pull request hands over in one state instead of two.
wshallwshall pushed a commit that referenced this pull request Sep 4, 2026
Clean auto-merge, no resolution decisions. origin/main advanced under this
worktree while the first merge was being verified -- the fleet shares one
repository, so another worktree's fetch moved the ref.

Nothing here touches #1317. The only ledger overlap is #1435's own new row,
which git merged textually and which sits nowhere near #1317. The row
reconciliation stays isolated in c8f9569, which is what a scripted row-merge
needs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-red A required check went red. Attribute it before retrying. reviewed A reviewer has read this. Removed automatically when new commits arrive.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant