Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,25 @@ scripts/security/scan-tokens.local.txt
# directory itself readable so the negation binds. Do not "simplify" it back.
/docs/testing/*
!/docs/testing/VERIFY.md
# docs/testing/README.md is the SECOND negation, and it is a marker rather than QA material.
#
# The rule above works, and its success is the problem: from an engine checkout the directory holds
# one file, so a path cited under it is INDISTINGUISHABLE from work that was never done. A reader
# greps a cited path, finds nothing, and concludes the document does not exist -- when it does, in
# the vault. Measured 2026-09-03: BACKLOG #1011 step 6 instructed a reader to mark a disposition
# discharged in a path under this tree and called that file "the only place in the repo that records
# the true state", a sentence that is false from any engine checkout because the file is not in the
# repo at all. A builder acting on it would have claimed a criterion it could never read.
#
# The marker is the checkout-side answer: standing in the directory, or browsing to it on GitHub,
# you are told the material moved and told not to read absence as evidence. It is a POINTER, never
# an index -- it names no vaulted document, because a path-to-document map over a closed set
# discloses what is NOT covered by subtraction. Keep it that way if you edit it.
#
# This negation and the VERIFY.md one are the ONLY two permitted here, and
# tests/test_private_paths_stay_ignored.py pins the pair as an exact set. A third fails there until
# someone writes down why. See BACKLOG #1435.
!/docs/testing/README.md
# DO NOT ADD docs/benchmarks/ OR docs/quality-gates/ HERE. Both are DELIBERATELY ABSENT. ADR 0160 D6.2
# considered both and left them: the 21 bench handoffs carry the measurement narrative for the ~133
# data files beside them, so untracking either half strips the rationale from records that stay.
Expand Down
63 changes: 61 additions & 2 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4876,15 +4876,15 @@ Retiring the tree costs the engine nothing operationally: **`tests/test_ech_egre
3. **Rewrite the `SECURITY.md` 12.1.5 paragraph** so the residual rests on the true ground: not "infeasible", but *buildable off-stdlib and deliberately not owned, because no partner endpoint publishes an ECHConfig and the engine's own TLS stack cannot originate ECH until OpenSSL 4.0*. Same accepted residual; a premise that survives inspection.
4. **Re-point `rest.py`'s docstring** — `ech_sidecar_url_from_settings` names `tools/ech-sidecar/` by path and asserts it is "proven to hide the SNI". After deletion that path resolves to nothing; the reference becomes the historical commit, and the "proven" claim needs whatever evidence actually backs it or should go.
5. **Decide `samples/ech-sidecar/README.md`** — the operator recipe (the only file under that sample dir) describes running a sidecar the repo no longer contains. Re-aim it at the generic contract (any loopback ECH-terminating proxy) or retire it with the tree.
6. **Mark SEC-71 discharged** in `docs/testing/master-test-plan/16-security-phi-and-supply-chain.md`, which specifies this exact disposition and is currently the only place in the repo that records the true state.
6. **Mark SEC-71 discharged** in `docs/testing/master-test-plan/16-security-phi-and-supply-chain.md`, which specifies this exact disposition. **Corrected 2026-09-03 (BACKLOG #1435): that path is NOT in this repository and this step is not reachable from an engine checkout.** `docs/testing/` is gitignored except `VERIFY.md`, so the document lives in the vault clone and whoever holds it performs this step. The clause this replaces said the file "is currently the only place in the repo that records the true state" — false from any checkout, since the file is not in the repo at all, and a builder acting on it would have claimed a criterion it could never read. See [`testing/README.md`](testing/README.md).

**Migration cost: none.** MessageFoundry is a not-yet-deployed beta with zero production instances, and `tools/` has never been in an sdist or a wheel — so no consumer of any published artifact is affected by deleting it. There is no deprecation window to run and nothing to sequence.

**Trigger:** already fired — not the ECH build trigger (ADR 0139's is *a partner endpoint begins publishing ECH configs*, and it has not), but this item's own: **an unowned tree is in the repository and two security documents contradict it today**. The ruling is owed now and does not wait on ECH deployment.

**Related:** #1010 (split from the same draft — the licence-header gate; `main.go` is 1 of the 196 headerless sources and gets a header only if this rules *keep*), #272 (ADR 0139's owning item), #353 (an ungated compliance artifact — same "nothing compares it to the record" shape), #1000 (a gate that has never been watched fail; here the failure is a *language* nothing gates at all).

**Source:** master test plan **SEC-71** (`docs/testing/master-test-plan/16-security-phi-and-supply-chain.md`), which specifies this disposition; escalated 2026-08-04 when the SPDX half of the original draft was found to rest on a false claim and was split out as **#1010**. Every fact above was re-executed against `origin/main` at `df9c4d54`.
**Source:** master test plan **SEC-71** (`docs/testing/master-test-plan/16-security-phi-and-supply-chain.md` — vault-only, unreadable from an engine checkout; see [`testing/README.md`](testing/README.md)), which specifies this disposition; escalated 2026-08-04 when the SPDX half of the original draft was found to rest on a false claim and was split out as **#1010**. Every fact above was re-executed against `origin/main` at `df9c4d54`.

## 1002. AG-rig validation: prove the multi-subnet failover reconnect

Expand Down Expand Up @@ -21928,6 +21928,65 @@ The other four are posture, inherited from the tripwire as it already stood:

---

## 1435. docs/testing has no checkout-side marker, so a vaulted path reads as unstarted work

> 🚧 **Filed 2026-09-03. The marker is built; the gate is researched and deliberately NOT built.** Value **5/10** · Difficulty **2/10** for what shipped, **6/10** for what is left. A documentation and process defect, not a product exposure: MessageFoundry has zero deployments, nothing here reaches an operator, and no impact claim below is present-tense about a running site.

**The defect, in one sentence: from an engine checkout, work that is UNREADABLE is indistinguishable from work that is UNSTARTED.**

`git ls-files docs/testing` returns one file, and `/docs/security/` is blanket-ignored at `.gitignore:177`. The real documents live in the vault clone. So a reader who greps a cited path, finds nothing, and concludes the work was never done is reasoning correctly from everything the tree shows them, and reaching a false conclusion.

**The measured instance, and it carries its own proof.** #1011 step 6 told a reader to mark SEC-71 discharged in a path under `docs/testing/`, and called that file *"currently the only place in the repo that records the true state"*. That sentence is false from any engine checkout, because the file is not in the repo at all. A builder acting on it would have claimed a criterion it could never read. Corrected in the same change that filed this item; both citations in #1011 now name the unreadability in the same breath as the path.

Two peer fleets hit the same wall independently on 2026-09-03, on #1152 and on #1193, and a third case put half an availability strategy in a vaulted document under #1191. Three sessions worked it out unaided. The next one may not, and a better brief does not fix it, because the brief author hits the same blind spot.

### What shipped here

`docs/testing/README.md`, tracked through a second negation under `/docs/testing/*`. A reader standing in the directory, or browsing to it on GitHub, is told the material moved and told not to read absence as evidence. `docs/README.md` carries the same warning at the front door.

**It is a POINTER and never an index.** It names no vaulted document. A path-to-document map over a closed set discloses what is NOT covered by subtraction, which is the disclosure the gitignore exists to prevent. Keep it that way. The `docs/security/` half already has its rule in [`SECURITY-DOCS-POLICY.md`](SECURITY-DOCS-POLICY.md) and the marker links there rather than restating it.

`tests/test_private_paths_stay_ignored.py` pins the pair as an exact set, so a third negation fails until someone writes down why, and the marker cannot be silently dropped from fresh clones.

### The gate: researched, measured, and NOT built. This is the finding, not a shortfall.

A test that reds when a ledger body cites a vaulted path without flagging it is the most durable shape and was the first choice. It does not survive contact with the corpus. All figures measured 2026-09-03 against `46ea10a78`, and they are measurements rather than targets.

| Granularity | Detection | Result on the corpus |
|---|---|---|
| Per line | path pattern | 55 lines cite, 23 carry no flag on the same line |
| Per line, about three lines of context | path pattern plus `THREAT-MODEL` and `asvs-scorecard` | 75 lines cite, 22 carry no signal (measured independently, two readers agreeing to the line) |
| Per item body | path pattern | 27 items cite, 2 unflagged |
| Per item body | path pattern plus `master-test-plan` | 32 items cite, 5 unflagged |
| Per citation, plus or minus 300 to 1200 characters | path plus name | 15 to 21 items carry at least one unflagged citation |

**Three findings, each independently enough to stop the build.**

1. **An unscoped gate reds about 22 times on day one**, against a corpus nobody repairs in one pull request. It needs a grandfather baseline or a changed-lines-only scope before it is shippable at all.
2. **The item-granularity gate is quiet and measures the wrong thing.** At 2 unflagged it would ship comfortably, and it MISSES #1152 — the case that motivated this item. #1152's body flags a different vaulted document (the scorecard) while leaving its master-test-plan clause unflagged, so the item passes while the defect stands. A gate that is quiet because it is aimed slightly wrong is worse than no gate.
3. **"Cites a vaulted path" overfires, and the real predicate may not be expressible as a pattern.** Line 22 is prose ABOUT the arrangement, not a citation anyone must follow. A table row giving a cell count is permitted vocabulary. Neither is a defect. The actual predicate is **a citation a reader must FOLLOW in order to act**, which is a judgement about intent, and no regex tried here expresses it.

**The window parameter is undefended.** Three lines is arbitrary; nobody has a reason for it, and a single-block rule already missed a stale assertion in `docs/SERVICE.md` on a peer's item. Treat 53 against 22 as an order of magnitude, not a threshold.

**IF SOMEONE BUILDS THIS, THE FAILURE MESSAGE IS ALSO A DISCLOSURE SURFACE.** A message naming WHICH paths are vaulted publishes the map into CI logs and into the workflow file, both public. That is the enumerating-marker defect wearing different clothes, and the helpful-error-message instinct walks straight into it. A message of the shape *"this item body cites a maintainer-internal path; state in the body that it is unreadable from an engine checkout"* tells the author exactly what to do and discloses nothing. Do not echo the offending path into the message, the summary, or the annotation.

### The two-sided controls, named, so nobody re-derives them

A gate proven on one side proves nothing: one that fires on everything and one that fires on nothing both look green against a single-sided test.

**Fires side (must red), and it is on `main` today:** `docs/BACKLOG.md` lines 4763 and 4771 before the correction in this change. Both named a `docs/testing/master-test-plan/` chapter, and 4763 instructed the reader to act in it. The cleanest true positive in the corpus, and independent of #1152. Recover the pre-correction text from this item's own commit.

**Quiet side (must pass):** the remedy bodies on PR 804 (#1191) and PR 812 (#1193). **Both were still unmerged on 2026-09-03**, so a builder calibrating against `main` alone sees the UNFLAGGED versions and draws the wrong conclusion about what the remedy looks like. Read them with `gh pr diff 804` and `gh pr diff 812`. #1191 is the sharper of the two: it records that the availability inventory is vaulted AND records that its builder deliberately did not add a read-side anchor to the public `_RESOURCE_ANCHORS` registry, because the registry is public and the document is not, so the anchor would red a tree nobody in an engine checkout can see or repair.

Both are the shape a gate must PERMIT. An unflagged citation is the defect; a flagged one is the remedy the gate exists to produce.

### What is left

- Decide whether the follow-to-act predicate is expressible. If it is not, that is a finding and this item closes on the marker.
- If it is, scope the gate to changed lines or grandfather the day-one population, and prove it on both controls above.
- The `docs/security/` half has no marker of its own. `/docs/security/` is a blanket rule, so a placeholder there needs a deliberate gitignore negation. Judged not worth it here: [`SECURITY-DOCS-POLICY.md`](SECURITY-DOCS-POLICY.md) already states that rule in a tracked file and says how to request the material, and a second marker would restate a load-bearing fact instead of linking to it.

**Related:** #1011 (the corrected instance), #1152, #1191 and #1193 (the three peer cases), #1244 (an engine change breaking a vault anchor produces no attribution — the same public-repo-cannot-see-the-vault shape, from the other direction).
## 1439. webconsole_seam_snapshot.py resolves messagefoundry from sys.path, not from its own repo

> 🔢 **Filed 2026-09-03 - BUILT IN THIS COMMIT, not yet landed.** Found while building #1139, by a session that spent its debugging on the gate instead of the generator. The census in LIMB 3 is the reason this is filed as one defect and not a class.
Expand Down
6 changes: 6 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ different repositories for opposite audiences. `docs/ASVS-L2-PHASE0-CHANGES.md`
`docs/testing/VERIFY.md` is an operator tool, not a test plan. Where a name misleads, this index says so
rather than repeating it.

**A path you cannot open is not proof the work was never done.** Parts of `docs/` are kept in a separate
maintainer repository, so a citation elsewhere in the tree can be real and still not resolve here.
[`testing/README.md`](testing/README.md) covers `docs/testing/`, and
[`SECURITY-DOCS-POLICY.md`](SECURITY-DOCS-POLICY.md) states the separate rule for `docs/security/` and
`docs/reviews/`, including how to request the withheld material.

> **Reporting a security vulnerability?** → **[`.github/SECURITY.md`](../.github/SECURITY.md)**, the
> disclosure policy. **Not** `docs/SECURITY.md`, which despite the identical filename is the
> authentication and RBAC reference. Two files, same name, different jobs.
Expand Down
63 changes: 63 additions & 0 deletions docs/testing/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Most of this directory is not in this repository

Only [`VERIFY.md`](VERIFY.md) is tracked here. Everything else under `docs/testing/` is
maintainer QA planning, and it is kept in a separate maintainer repository.

**So a path under `docs/testing/` that you cannot open says nothing about whether the work
exists.** Unreadable and unstarted look identical from an engine checkout, and that is the
mistake this page exists to stop.

## Why you are probably reading this

You followed a citation. `docs/BACKLOG.md`, an ADR, a test docstring or a source comment names a
path under `docs/testing/`, you went looking for it, and it is not on disk. That citation is real.
It names a document that exists, and it is written for provenance: it records where a decision or
a test row came from, not a file you can open here.

`git ls-files docs/testing` returns one file. That is the rule working, not a gap.

## What this means when you are grading work

Do not treat an unreadable document as an absent one.

If an item asks you to satisfy a criterion, discharge a row, or check a claim in a document under
`docs/testing/`, you cannot verify that half from an engine checkout. Say so in those words. Record
the limb as unverified, not as satisfied and not as missing. Claiming you met a criterion you could
not read is the failure mode; it happened, and it is why this file is here.

The same caution applies to the reverse move. A limb that looks undone may simply be graded in a
document you do not have.

## Why the split

[ADR 0160](../adr/0160-public-repo-content-policy-operator-and-security-review-material-only.md)
sets the test: a tracked file must be something an operator running MessageFoundry needs, or
something a security reviewer assessing it needs. QA planning for a maintainer build box is
neither, so it moved. `VERIFY.md` stayed because it documents `messagefoundry verify`, the on-box
acceptance check a real deployment runs.

None of this is a confidentiality control and it must never be described as one. This material was
never secret. It is process noise, and moving it was subject-matter tidying.

The `.gitignore` block that implements the split carries the full reasoning. Search it for
`/docs/testing/*`.

## The neighbouring case, which is a different rule

`docs/security/` and `docs/reviews/` are also absent, and they are held back for their own reasons.
[`SECURITY-DOCS-POLICY.md`](../SECURITY-DOCS-POLICY.md) states that rule, says what is public, and
tells adopters, evaluators and security reviewers how to request the withheld material. Read it
there. Do not assume the reasoning on this page carries over.

## What this page will not do

It will not list what is in the other repository. A path-to-document map over a closed set hands out
what is not covered by subtraction, so this is a pointer and never an index. If you need something
named here, ask a maintainer through the route `SECURITY-DOCS-POLICY.md` documents.

## If you are writing a citation

Name the unreadability in the same breath as the path. One clause is enough, for example that the
path is vault-only, gitignored, or unreachable from an engine checkout. A reader who meets your
citation months later gets the same warning you would have wanted, without having to find this page
first.
Loading
Loading