Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
4383fb8
Acceptance process: the test proves the clarity of the change, with k…
LTSCommerce Oct 12, 2026
36d87d1
Acceptance process: review 1 findings applied
LTSCommerce Oct 12, 2026
8a63448
Acceptance process: review 2 findings applied
LTSCommerce Oct 12, 2026
4cc5688
Acceptance process: review 3 findings applied
LTSCommerce Oct 12, 2026
c31d49a
Acceptance process: CHANGES.md names pull request 22
LTSCommerce Oct 12, 2026
972cbb7
Merge branch 'editor/acceptance-coherent' into editor/detector-route-…
LTSCommerce Oct 12, 2026
e6bdf40
Keys and routed findings: open points from runs 53 to 59
LTSCommerce Oct 12, 2026
04aa914
Merge branch 'editor/acceptance-coherent' into editor/detector-route-…
LTSCommerce Oct 12, 2026
ed30e69
Detector: the detectable route, the gap duty, the remainder and the m…
LTSCommerce Oct 12, 2026
4081aa4
Toolchain key and routed findings: open points from runs 61 and 62
LTSCommerce Oct 12, 2026
a44ddf8
Merge branch 'editor/acceptance-coherent' into editor/detector-route-…
LTSCommerce Oct 12, 2026
c1c986e
Run 060: detector route rule cohort, failed on two findings
LTSCommerce Oct 12, 2026
4cbecad
Detector: a Message row in section 2, and who may grade a detector
LTSCommerce Oct 12, 2026
9907492
Routed findings from runs 63 and 64
LTSCommerce Oct 12, 2026
890d8e3
Run 063: detector route rule cohort, failed on one finding
LTSCommerce Oct 12, 2026
d157143
Detector 5.3: a fuller report elsewhere is not a remainder
LTSCommerce Oct 12, 2026
44fdd9e
Merge branch 'editor/acceptance-coherent' into editor/detector-route-…
LTSCommerce Oct 12, 2026
5852a85
Routed findings: four detector rows closed by run 66, two toolchain r…
LTSCommerce Oct 12, 2026
da3e431
Merge branch 'editor/acceptance-coherent' into editor/detector-route-…
LTSCommerce Oct 12, 2026
c78b76c
Run 066: detector route rule cohort passed
LTSCommerce Oct 12, 2026
f3ec69e
Routed findings: rows from runs 19 to 30, two closed by defined-terms…
LTSCommerce Oct 12, 2026
ae5f8da
Runs 060, 063 and 066: pull request 23 and the posted answer sheets
LTSCommerce Oct 12, 2026
8defdae
Detector 5.3 and 8.1: the location owed only where the summary leaves…
LTSCommerce Oct 12, 2026
bc6ae14
Method key: clause 3.3's narrowing sentence; routed findings from run…
LTSCommerce Oct 12, 2026
d67174c
Merge branch 'editor/acceptance-coherent' into editor/detector-route-…
LTSCommerce Oct 12, 2026
dbdf841
Detector 5.3 and 7.1: a usable summary defined, detectability in one …
LTSCommerce Oct 12, 2026
13b497b
Toolchain key: the 4.3 row says which case the fixture presents; ROUT…
LTSCommerce Oct 12, 2026
f32e542
Run 073: detector route rule cohort, failed on clauses 5.3 and 7.1
LTSCommerce Oct 12, 2026
cca75a7
Run 077: detector route rule cohort, failed on clause 5.3's remainder…
LTSCommerce Oct 12, 2026
c5f17b5
Detector 5.3: the remainder in three plain sentences
LTSCommerce Oct 12, 2026
0c21e98
Merge branch 'editor/acceptance-coherent' into editor/detector-route-…
LTSCommerce Oct 12, 2026
ac14996
Changelog: clause links point at the headings as they are
LTSCommerce Oct 12, 2026
08b9e56
Run 078: detector route rule cohort passed
LTSCommerce Oct 12, 2026
389ef3f
Acceptance: a missing-document remark is a note however it quotes; a …
LTSCommerce Oct 12, 2026
1f0c4f8
Run records: links to the answer sheets posted on the pull request
LTSCommerce Oct 12, 2026
3565069
Merge branch 'editor/acceptance-coherent' into editor/detector-route-…
LTSCommerce Oct 12, 2026
0da928d
ROUTED: run 81 names the toolchain 6.1 Project record sentence again;…
LTSCommerce Oct 12, 2026
ae6faf2
ROUTED: run 82 names the toolchain 6.2 last paragraph again
LTSCommerce Oct 12, 2026
a095fa7
ROUTED: toolchain 6.2's first paragraph against clause 6.4's defaults…
LTSCommerce Oct 12, 2026
852dcd8
ROUTED: run 84 names the toolchain Project record entry against claus…
LTSCommerce Oct 12, 2026
568e323
ROUTED: run 85 names the toolchain 6.2 first paragraph against clause…
LTSCommerce Oct 12, 2026
40f7895
ROUTED: run 86's findings in untouched toolchain text, clause 6.4 aga…
LTSCommerce Oct 12, 2026
df07036
Merge branch 'editor/acceptance-coherent' into editor/detector-route-…
LTSCommerce Oct 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
287 changes: 222 additions & 65 deletions ACCEPTANCE.md

Large diffs are not rendered by default.

41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,47 @@ pass it.

## Detector specification (DETECTOR-SPEC.md)

### Unreleased

- Clarity, no obligation changed: four passages that cold readers of an unrelated change read as
the document pulling against itself or could not follow, each listed in `acceptance/ROUTED.md`.
Clause [7.1](DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable)'s detectable route is now a bespoke rule the project writes for the detector under
clause [4.1](DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it), after six runs read "a Rule the project can write in the Detector itself" as the
project editing the detector's own source, and a new paragraph says what detectable means and
what each of the two ways is. Clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps)'s duty to record a known gap falls only on a detector
that declares a version, and says so beside "The declaration is optional", with the method's
clause [3.2](SPEC.md#32-build-the-net) record applying either way. Clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) says what the remainder is, and that
there is none where the output prints every finding. Clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) says it asks for the
mechanism and not for every page, a missing page failing clause [6.3](DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), and that a
documentation path printed in the message is not enough on its own, after three readers of
run 54 set that sentence against the mechanism. Prompted by runs 32 to 54 on the defined-terms
change, in which two or more readers quoted each passage.
[Run 60](acceptance/runs/060-detector-route-rule-run-1.md) failed on two findings in the
revised text, every keyed item within threshold: two readers could not find Message defined,
so section [2](DETECTOR-SPEC.md#2-terminology)'s table gains a Message row; and two readers stopped at clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps)'s
"anyone who exercises the clauses above", which now says anyone may grade a detector by
checking it against each clause of sections [4](DETECTOR-SPEC.md#4-authoring-rules) to [7](DETECTOR-SPEC.md#7-suppression).
[Run 63](acceptance/runs/063-detector-route-rule-run-2.md) failed on one finding in the revised
text, every keyed item five of five: two readers needed two readings of clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s
sentence on a report that only adds detail, which is now a sentence of its own with its subject
stated; three readers named clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation)'s paragraph on a project's own rule, which this
change does not touch and `acceptance/ROUTED.md` now lists.
[Run 66](acceptance/runs/066-detector-route-rule-run-3.md) passed, every keyed item five of
five, its one finding being that same clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) paragraph, named again by three readers,
which stays in `acceptance/ROUTED.md` for a change of its own. Run 69, a cohort of the
defined-terms change on this document with this change merged in, then had two readers set
clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s "MUST carry both a usable summary and the location of the remainder" against its
"need not name it", and two readers stop at clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps)'s gap sentence and its MAY: clause
[5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) now owes the location only where the summary leaves a finding out, and clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) says
where a practitioner's record of a gap lives and that a detector without a declaration can
still be found to conform. [Run 73](acceptance/runs/073-detector-route-rule-run-4.md) failed
on clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s "usable summary" and on clause [7.1](DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable)'s second paragraph; clause
[5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) now says what a usable summary is and what the remainder is, and clause [7.1](DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) states
the two ways a route is detectable in one paragraph. [Run
77](acceptance/runs/077-detector-route-rule-run-5.md) failed on the remainder sentence alone,
four readers unable to tell what "is not one" referred to; it is now three plain sentences.
Accepted by [run 78](acceptance/runs/078-detector-route-rule-run-6.md).

### 1.0.0, 2026-09-08

First version, split out of toolchain specification 0.1.0 on the same day. States what a detector,
Expand Down
32 changes: 20 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ review. It is a patch version when released.
SHOULD or MAY in a numbered section or appendix. It needs everything a clarity edit needs, plus
the five tests in SCOPE.md answered in the pull request, and it is at least a minor version when
released. `tools/changes.py` tells the two apart mechanically and CI runs it on every pull
request.
request. A change to the acceptance process, as ACCEPTANCE.md defines one under "Changes to
this process", is neither: ACCEPTANCE.md says how it is made, and it is recorded in
`acceptance/CHANGES.md`, not in CHANGELOG.md.

## What a pull request contains

Expand All @@ -55,20 +57,26 @@ survives, the change is yours.
the editor sets the pre-release header.
2. **Scope.** For an obligation, the editor confirms each SCOPE.md test. A failed test closes the
pull request with a row in `DECLINED.md`.
3. **Review and rewrite.** An independent review of the diff against the document, then the
editor's rewrite, pushed to the pull request branch where the contributor allows it or to an
editor's branch that supersedes it.
3. **Review and rewrite.** An independent review of the diff against the whole document, by a
reviewer who did not draft the change, then the editor's rewrite, pushed to the pull request
branch where the contributor allows it or to an editor's branch that supersedes it. The pull
request names the reviewer and links or contains the review.
4. **Keys.** Where the change touches a sentence an acceptance key quotes, or adds a load-bearing
obligation, the editor updates the key before the cohort runs, never after.
5. **The cohort.** The acceptance test in ACCEPTANCE.md runs on the final text. Its record is
committed under `acceptance/runs/`, the `Unreleased` changelog line names it, and CI refuses
a specification change that lacks one. For an obligation, the editor sets the document's
header to `<next version>-dev, unpublished` in the same commit if it still names a published
version. A wording change after the run means a fresh run.
obligation, the editor updates the key before the cohort runs, never after. The key's
required elements stay what its question asks for; `acceptance/README.md` says what a key
update may and may not do.
5. **The cohort.** The acceptance test in ACCEPTANCE.md runs on the final text, one cohort per
changed document. Its record is committed under `acceptance/runs/`, the `Unreleased`
changelog line names it, and CI refuses a specification change that lacks one. For an
obligation, the editor sets the document's header to `<next version>-dev, unpublished` in the
same commit if it still names a published version. A change to the document's text after the
run means a fresh run, as ACCEPTANCE.md says under "When it runs". A finding the
run routes to `acceptance/ROUTED.md` is not fixed in this pull request.
6. **Merge** into `next` by merge commit, and the branch is deleted.

Steps 2 to 5 are done by a person with a Claude Code session, not by CI, and they happen in
batches. A quiet pull request is not a declined one; a declined one says so.
Steps 2 to 5 are the editor's, not CI's, and they happen in batches. The editor may have an
agent draft, review or mark, as ACCEPTANCE.md allows for marking; every decision in these steps
stays the editor's. A quiet pull request is not a declined one; a declined one says so.

## Versions and releases

Expand Down
51 changes: 34 additions & 17 deletions DETECTOR-SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ definition; the ones this document leans on most are, in short:
| -------------- | ------------------------------------------------------------------------------ |
| [Detector] | A tool that reads code without executing it and reports what it finds |
| [Rule] | One check a [Detector] evaluates |
| [Message] | The text a [Rule] prints when it fires |
| [Identifier] | The stable name printed with a finding that leads to its documentation |
| [Practitioner] | Whoever is doing the work, a person or an [Agent] |
| [Toolchain] | Everything a project assembles to run its checks through, [Detectors] included |
Expand Down Expand Up @@ -157,7 +158,11 @@ any.
### 5.3 The result MUST reach the practitioner in the output of the command they ran

Where the [Detector] writes fuller detail elsewhere, the invoked command's own output MUST
carry both a usable summary and the location of the remainder.
carry a usable summary and, where the summary leaves any finding out, the location of the
remainder. A summary is usable when each finding in it carries where it is and its
[Identifier]. The remainder is the findings the summary leaves out. Where the output prints
every finding, there is no remainder. A fuller report written elsewhere, such as a page with a
code snippet for each finding, is not a remainder, and the output need not name it.

**Why**: method specification clause 8.2. A report the [Practitioner] has to go and find is a
report that arrives after the decision it was meant to inform.
Expand All @@ -183,13 +188,16 @@ Two cases, by who wrote the [Rule]:
| The project's own | The project | Print the [Identifier] unaltered; the lookup is the [Toolchain]'s to give |

In both, the mechanism is keyed on **the [Identifier] exactly as printed**. A command, an index
file or a URL are all acceptable forms. Each case in turn:
file or a URL are all acceptable forms. This clause asks for the mechanism, not for every page:
where the mechanism works but a [Bundled rule]'s [Identifier] has no documentation to resolve
to, that fails clause 6.3, not this clause. Each case in turn:

**For a [Bundled rule]**, the [Detector] supplies the documentation and the
lookup, and clauses 6.2 and 6.3 say where. The mechanism MUST resolve an [Identifier]
presented alone, because the reader who needs it most has the [Identifier] from a log, a
ticket or a colleague and not the [Message]. A [Message] that carries its own
documentation path resolves that [Message], not the [Identifier].
ticket or a colleague and not the [Message]. A documentation path printed in the [Message] is
therefore not enough on its own: it helps only a reader who has the [Message], and the
[Identifier] alone still leads nowhere.

**For a project's own [Rule]**, the [Detector] cannot know the documentation, so what
it owes is the half it can give: the [Identifier] printed unaltered under clause 4.3, and no
Expand Down Expand Up @@ -236,7 +244,7 @@ An automated check over every [Identifier] a [Bundled rule] can print, applying
family list or pattern of clause 6.5 where one is used, that blocks the release when any of them
lands on no page.

**Why**: clause 6.1 is the clause most easily believed to be satisfied whilst being broken, because
**Why**: clause 6.3 is the clause most easily believed to be satisfied whilst being broken, because
the documentation is written by the same person who wrote the [Rule] and its absence is
invisible from the inside. Where the [Detector] is shipped inside a [Toolchain],
the [toolchain specification](TOOLING-SPEC.md)'s self-audit makes this check a MUST for the [Toolchain]; a
Expand Down Expand Up @@ -269,9 +277,16 @@ nothing to run.
An inline ignore comment, a per-line directive and a generated [Baseline] are all such
routes. The [Detector] MAY ship them. For each one it MUST do at least one of two things, and either
alone satisfies this clause: make the route disableable by configuration; or make the route
detectable, by a [Rule] the project can write in the [Detector] itself or by a mechanical check
the [Detector] documents. Either lets a project which decides to forbid the route enforce that
decision. A route that can be neither switched off nor seen does not [Conform].
detectable, by a bespoke [Rule] the project writes for the [Detector] under clause 4.1 or by a
mechanical check the [Detector] documents. Either lets a project which decides to forbid the
route enforce that decision. A route that can be neither switched off nor seen does not [Conform].

A route is detectable when the project can find every use of it, in either of two ways: by a
bespoke [Rule] the project registers under clause 4.1 that matches the route wherever it is
used, which the [Detector] runs like any other bespoke [Rule] and need not ship itself; or by a
mechanical check the [Detector] documents, such as a text search for the route's spelling. The
first way needs the [Detector]'s [Rules] to be able to match the route's text, so a [Detector]
whose [Rules] never see comments offers only the second for an ignore comment.

**Why**: the method specification's position is that [Suppression] is an [Owner]
decision under its clause 3.4 and section 4, and an [Owner] cannot decide something they are
Expand Down Expand Up @@ -310,17 +325,19 @@ for a [Detector].

The declaration is machine-readable, in whatever form the [Detector]'s ecosystem uses to
record dependencies. The same declaration is where a gap against this document is recorded once it is
known: a [Detector] that has learnt, from its own checks or from a [Practitioner]'s
report under the method's clause 3.2, that it fails a MUST in sections 4 to 7 MUST record that gap
alongside the version it declares, in the same file or one it names. A declaration with a non-empty
gap record is a statement of where the [Detector] stands and is not a claim of
known: a [Detector] that declares a version and has learnt, from its own checks or from a
[Practitioner]'s report under the method's clause 3.2, that it fails a MUST in sections 4 to 7
MUST record that gap alongside that version, in the same file or one it names. A declaration
with a non-empty gap record is a statement of where the [Detector] stands and is not a claim of
[Conformance].

The declaration is optional. It is how a maintainer claims [Conformance]; it is not a
condition of it. A [Detector] that predates this document, or whose maintainer has never read
it, MAY be graded [Conforming] on evidence by anyone who exercises the clauses above against
it, and a verdict on any [Detector], declared or not, rests on that exercise and not on the
claim. A declaration tells the reader what the maintainer believes and what they know to be missing;
The declaration is optional, and so the duty above falls only on a [Detector] that has one; a
gap a [Practitioner] finds is recorded in their own project's record under the method's clause
3.2 in any case. The declaration is how a maintainer claims [Conformance]; it is not a condition
of it, and a [Detector] without one can still be found to [Conform]. A [Detector] that predates
this document, or whose maintainer has never read it, MAY be graded [Conforming] on evidence:
anyone may grade it, by checking it against each clause of sections 4 to 7, and a verdict on
any [Detector], declared or not, rests on that check and not on the claim. A declaration tells the reader what the maintainer believes and what they know to be missing;
the reader still runs the [Harness].

The shape is illustrative rather than prescribed. In a Composer manifest:
Expand Down
38 changes: 38 additions & 0 deletions acceptance/CHANGES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Changes to the acceptance process

Changes to the acceptance process, as [ACCEPTANCE.md](../ACCEPTANCE.md) defines one under
"Changes to this process", recorded here as it describes: a line is added at the top for each
change, naming its pull request. The process is not a specification and is not versioned, and a
change to it is not tested by a cohort. Changes to the three specifications are recorded in
[CHANGELOG.md](../CHANGELOG.md).

- [Pull request #22](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/22):
the test is made to prove what it exists to prove, the clarity of the
document to a cold reader, after thirty-three runs on two changes (19 to 30, 31 to 33 and 35
to 52) in which keyed answers reached five of five and the runs still failed. Five defects in
the process were found. Keys required elements the question never asked for (detector 2b's
"need not remove or forbid", detector 2c's "at a version tracked together", and on pull
request 14 method 2d's order of the kinds), so readers who answered the question as put were
marked wrong: now each key lists, per item, the elements an answer must state, one for
each part of the question, and the questions are reworded to ask for exactly those; anything
else in the key only catches a wrong assertion. A passage two readers found confusing stayed
open whilst any single reader of a later cohort named any part of its paragraph, so cleared
passages kept coming back: now a finding is settled run by run, raised by two readers of one
run and clear when the next names it by fewer than two. A finding landed on text the change
never touched, so a change's cohort acted as a review of the whole document and could not
converge: now a finding in text the change wrote or altered blocks the change, and a finding
elsewhere is routed to `acceptance/ROUTED.md` and to a change of its own, whilst the keyed
questions still prove the whole document on every run. Markers invented judgements the rules
left open and later runs cited them as precedent: now an open point is marked for the reader
and settled in this file or the key before the next run, and a run record states no judgement
the rules do not. The rule that any miss whose reasoning quoted the document was a finding
whatever the count is replaced by the key naming the wrong readings the text invites, which
count against the document on one reader, every other single miss being tolerated by the
all-but-one threshold. Question 2's instruction asks for the sentence or sentences relied on,
since three keys rest on two. The cohort, the packet, set-aside readers, the posted sheets,
the fuller record and the route for a process change are kept from the earlier proposal in
pull request 13, which this change supersedes. Relative to the rules on `next`, the
traceable-miss row is narrower and the confusion row is applied to the change rather than
the document; both are recorded here as the reason the earlier rules could not converge, and
neither changes a mark already given. Prompted by runs 19 to 30 on pull request 14 and runs
31 to 33 and 35 to 52 on the defined-terms change.
Loading
Loading