From 4383fb8a45812f01f141dc695445ce74f8bead6d Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 11:34:10 +0100 Subject: [PATCH 01/34] Acceptance process: the test proves the clarity of the change, with keys that ask what the question asks Keys list the required elements per part of each question, questions ask for exactly those, a confusion finding is settled run by run and blocks only in text the change wrote or altered (others go to acceptance/ROUTED.md), single misses count against the document only where the key names the reading, and open marking points are settled in the rules before the next run rather than invented per run. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 252 ++++++++++++++++++++++-------- CONTRIBUTING.md | 32 ++-- acceptance/CHANGES.md | 36 +++++ acceptance/README.md | 51 ++++-- acceptance/ROUTED.md | 32 ++++ acceptance/detector/KEY.md | 129 +++++++++------ acceptance/detector/QUESTIONS.md | 9 +- acceptance/method/KEY.md | 107 +++++++------ acceptance/method/QUESTIONS.md | 3 +- acceptance/runs/README.md | 25 +-- acceptance/toolchain/KEY.md | 116 ++++++++------ acceptance/toolchain/QUESTIONS.md | 3 +- 12 files changed, 546 insertions(+), 249 deletions(-) create mode 100644 acceptance/CHANGES.md create mode 100644 acceptance/ROUTED.md diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index 31c8df0..84ab39b 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -6,49 +6,76 @@ document that only a strong reader can follow is not a specification, and the re matter most, coding agents meeting the method in a failing pipeline, are often the weakest. The test is run with a low-strength model so that clarity is measured at the floor, not the ceiling. +The test proves one thing: that a capable reader, reading the document cold and once, takes from +it what the document exists to say, and is not confused by the text the change put there. The +keyed questions prove the document; the confusion question proves the change. Everything below +serves that, and a rule that stops it proving that is a defect in this file, fixed as +[Changes to this process](#changes-to-this-process) describes. + ## The cohort -- **At least five independent readers**, each a fresh instance of a low-strength model (the - smallest current model in its family, for example Haiku), with no shared context and no - access to the change history, the critique findings or each other. -- Each reader receives **only the changed document or documents** and the fixed question set - below. Nothing else is read first. +- **An odd number of independent readers, at least five**, each a fresh instance of a + low-strength model (the smallest current model in its family, for example Haiku), with no + shared context and no access to the change history, the critique findings or each other. + +- **Each reader is a separate process with no tools and no project context, only its packet**: + started from an empty directory outside any repository, with no user, project or local settings + files, no `CLAUDE.md`, skills, plugins, hooks or MCP servers, and no saved session, so that no + project instruction reaches it. The model tool's own fixed system prompt is not project + context. For example, with the packet on standard input: + + ``` + claude -p --safe-mode --model haiku --tools "" --setting-sources "" --strict-mcp-config --no-session-persistence --output-format json < packet.md + ``` + + A reader is set aside, and its answers do not count, when its answer text is empty, when its + process reports an error, or when the model version it reports is not the cohort's model. A run + with a reader set aside is void: it takes a run number and has a record, and a fresh cohort + runs. No reader is set aside for any other cause. + +- Each reader receives **only its packet**: the document's question set, the changed document + and the fixture the questions name, each in full in the reader's prompt, the question set + first. Each file is preceded by exactly one line, `===== FILE: =====`, where `` is + the name the question set gives the file (so `fixtures/remediation-ledger.md`, and + `QUESTIONS.md` for the question set itself). The question set carries the instruction to the + reader, and nothing else is added. Every reader in a run receives the same packet. + - Readers are run **after** every stronger review has been applied. They are the last gate, not the first draft. ## The question set -Written before the run, with an **answer key** for every keyed question drawn verbatim from the -document, so that marking is mechanical. Each reader answers all of them. +Written before the run, with an **answer key** for every keyed question, so that marking is +mechanical. The key states, for each keyed item, exactly what an answer must contain and what +makes it wrong; the question asks for exactly that and nothing the key does not expect. A key +asks for nothing the question does not. Each reader answers all of them. 1. **Restatement.** In their own words, what the document asks of its subject, in at most six bullets. Keyed against the list of MUST clauses: a bullet is correct when it names an obligation the document actually states. -2. **Load-bearing statements.** Three questions, fixed per document, each asking for one - sentence and a quotation: +2. **Load-bearing statements.** Three or more questions, fixed per document, each asking for a + short answer and the sentence or sentences of the document it rests on, quoted. Each has one + or more parts, and the key lists the element an answer must state for each part: - `SPEC.md`: what a practitioner does before fixing a defect, in order; who may baseline or - suppress; what "conforms" means for a remediation. + suppress, and whether an agent may; what "conforms" means for a remediation and what a + verdict rests on. - `DETECTOR-SPEC.md`: what "conforms" means and whether a tool that predates the document can - conform; what a detector must do about an inline suppression route; where a bundled rule's - documentation must live. - - `TOOLING-SPEC.md`: where conformance is measured; how a grade is assigned when parts come - from different sources; how many grades a project that ships tooling has. + conform; what a detector must do about an inline suppression route and whether one of those + things is enough; where a bundled rule's documentation must live, at what version, and + whether it must be reachable offline. + - `TOOLING-SPEC.md`: where conformance is measured and what the toolchain is; how a grade is + assigned when parts come from different sources; how many grades a project that ships + tooling has and whether one implies the other. 3. **Application.** A fixture case supplied with the questions, graded clause by clause with - Yes, Partial or No and one line of reasoning. Keyed against the intended verdicts; a divergence - counts against the document only when the reader's reasoning shows the clause was read - differently, not when the fixture was misread. + Yes, Partial or No and one line of reasoning. Keyed against the intended verdicts, with the + readings the key names as ones the text invites counting against the document. 4. **Confusion.** Any passage the reader found confusing on its one reading, quoted with its clause number, and nothing else: not what would make it clearer, not a proposed change. -The cohort settles one thing: is the document clear, coherent and comprehensive. A reader getting -everything right after one reading proves it; a reader getting confused fails it. The document -is designed by its editor to resolve the comprehension failures the cohort shows; the cohort is -the acceptance layer, not the designer. - ## What the cohort is not asked The cohort may point at what it found confusing. It is never asked to rate the document, to say @@ -58,61 +85,162 @@ not there to design the text, and a document shaped by what its weakest reader s a document designed by that reader. The signals this test takes from the cohort are a keyed answer missed, a wrong verdict reasoned from the document, and a passage pointed at; what to do about any of them is the editor's decision. An earlier version of this test asked the reader -what would make a clause clearer and for a clarity rating; both were removed on the author's +what would make a clause clearer and for a clarity rating; both were removed on the editor's instruction for that reason, and neither is reinstated. +## Marking + +The editor marks each answer sheet against the document's key, or delegates the marking to a +separate marker that has seen neither the drafting nor the review. A delegated marker receives +the document, its question set, fixture and key at the commit the readers read, this file, the +answer sheets, and the facts the record needs that the sheets cannot show. Its marks are final +unless the editor shows that a rule here was misapplied, and a re-mark says which rule and why. + +Every mark is given by a rule in this file or in the key, as follows. Where neither settles a +mark, the marker gives the reader the benefit of the doubt, records the point in the run record, +and the editor settles it by a change to this file or to the key before the next run. A run +record states no judgement that is not written here or in the key; a judgement an earlier run +recorded is not a rule. + +- **Keyed items.** Question 1 as a whole is one keyed item, and so is each item of question 2 + and each answer of question 3 for which the key gives an intended answer. +- **Reading an answer.** An answer is read for its sense, not its wording: a paraphrase, a + synonym, or a defined term used in its defined sense states what it means. A reader quotes the + document when it gives consecutive words of the document, in the document's order and of any + length, in quotation marks or a block quote, verbatim apart from letter case, quotation-mark or + apostrophe style, and markdown link or emphasis markup. +- **Question 1** is correct when at least four bullets each name an obligation the key lists and + no bullet asserts one the key marks wrong. +- **Question 2.** An answer is the reader's sentences and its quotation read together. It is + correct when it states every element the key lists as required for that item, in the + document's words or the reader's own, asserts nothing the key marks wrong, and quotes the + document. An element stated only in the quotation is stated. Anything in the key beyond the + required elements and the wrong assertions is explanation for the marker, and an answer is + never marked down for leaving it out. +- **Question 3.** Each item is correct when its grade is one the key accepts and its reasoning + mentions what the key says it must mention for that grade. A grade the key accepts without a + condition is correct whatever the reasoning. +- **A miss that counts against the document.** The key names, for question 1 and for each item + of question 3, the wrong readings the text invites. A miss of that kind blocks whatever the + count, since one reader in five taking that reading means one agent in five acting on it. Every + other single miss under questions 1 and 3 is the reader's and is tolerated by the threshold + below; a second reader making the same miss is a failed threshold, not a judgement. +- **An unanswered question.** Each keyed item left unanswered is a miss. Question 4 left + unanswered counts as no passage named. +- **Question 4.** A reader names a passage when it quotes words of the document or gives a clause + number with a description that identifies a sentence or paragraph. A remark that is not about + a passage of the document, such as that another document was not supplied, is recorded as a + note and is not a passage. Two readers name the same passage when the words or sentences they + name lie in the same paragraph; a heading, a list item, a table row and a terminology entry are + each a paragraph of their own. A passage named by two or more readers of the run is a finding. +- **Where a finding lies.** The record says, for each finding, whether it lies in text the change + wrote or altered: a sentence or paragraph the pull request adds, rewords or moves, read against + the branch it targets. A finding there blocks the change. A finding anywhere else is a finding + against the document, not against the change: it is recorded in + [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not + block. The change is judged on what it changed; the document is proved by the keyed questions, + which every run answers over the whole of it. + ## Pass criteria -The change passes when all of the following hold on a single run: +The change passes when all of the following hold on a single run. The thresholds apply to the +readers whose answers count; with five readers, "every reader" is five of five and "all but one" +is four of five. -| Criterion | Threshold | -| ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- | -| Load-bearing statements (question 2) | **5 of 5** readers correct on every one | -| Restatement and application (questions 1 and 3) | **At least 4 of 5** readers correct on each keyed item | -| A wrong answer traceable to the text | **None.** A miss whose reasoning quotes the document is a finding, whatever the count | -| A passage found confusing (question 4), by two or more | **None outstanding.** It is a finding and blocks until the text is changed and a fresh cohort no longer names it | -| Anything from one reader only | Recorded; applied at the editor's discretion; does not block | +| Criterion | Threshold | +| ------------------------------------------------------------------------------------- | -------------------------------------------------------------- | +| Load-bearing statements (question 2) | **Every** reader correct on every item | +| Restatement and application (questions 1 and 3) | **All but one** reader correct on each keyed item | +| A miss the key says counts against the document | **None**, whatever the count | +| A passage found confusing by two or more readers, in text the change wrote or altered | **None.** It is a finding and blocks until the text is changed | +| A passage found confusing by two or more readers, elsewhere | Recorded in `acceptance/ROUTED.md` and routed; does not block | +| Anything from one reader only | Recorded; applied at the editor's discretion; does not block | The asymmetry is deliberate. A load-bearing statement is the sentence the whole document exists to make, and one reader in five getting it wrong means one agent in five will act on the wrong reading; unanimity is the only acceptable result there. Everywhere else a single divergent reader -is more likely to be the reader than the text, so four of five is tolerated, but a miss that the -reader can justify from the document is always the document's fault. - -## The bar only moves up - -No criterion in this document is ever relaxed to let a change through. If the cohort is confused, -the test is working and the text is wrong; the fix is to the text, never to the threshold, the -question, the fixture or the key. A threshold may be raised, a keyed question may be added, and a -key may be corrected where it misquoted the document; none of those makes a failing run pass. The -one time a threshold was relaxed, after three runs, the relaxation was reversed on the author's -instruction and is recorded here so that it is not tried again. Removing the clarity rating and -the request for suggestions was not a relaxation: it changed what the cohort is asked, not how -strictly an answer is marked. +is more likely to be the reader than the text, so one such reader is tolerated, unless the key +says the text invites that reading. A finding is settled run by run: a passage two readers named +in one run has no standing in the next, which is judged on its own readers, and a passage the +fresh cohort names by fewer than two is clear. + +## Changing the test + +Nothing in this test is changed to let a run through. A threshold, a question, a key or a +fixture is changed only before a run's readers start, never changes a mark already given, and +is recorded with its reason: a threshold, a question or the rules here by a change to this +process, a key by step 4 of `CONTRIBUTING.md` when the text it quotes is reworded or when a +load-bearing obligation is added, and each in [acceptance/CHANGES.md](acceptance/CHANGES.md). A +question is reworded only so that it asks for what its key requires, and a key's required +elements are never more than its question asks for. A run made after such a change is a fresh +run under the new rules, not a re-mark of an old one. The one time a threshold was relaxed to +let a change through, after three runs, the relaxation was reversed on the editor's instruction +and is recorded here so that it is not tried again. Removing the clarity rating and the request +for suggestions was not that: it changed what the cohort is asked, not how strictly an answer is +marked. ## After a failure -Fix the text, never the questions, and run a fresh cohort. Readers are not reused across runs. -The number of runs it took is recorded; a document that needed three runs is telling its editor -something. +Fix the text the findings name and run a fresh cohort; readers are never reused. A keyed miss is +fixed in the text that carries the obligation, wherever the change left it. A finding on text the +change wrote or altered is fixed there. A finding elsewhere goes to `acceptance/ROUTED.md` and +is not fixed in this change. The number of runs it took is recorded; a document that needed three +runs is telling its editor something. ## When it runs -Once per pull request, on the final text, after the editor's rewrite and any key update. It is -the last gate before the merge to `next`, so that every change on `next` is one the cohort has -accepted, and the release branch needs no cohort of its own unless it changes specification text. -A wording change after the run means a fresh run; readers are never reused. +On the final text of each pull request, after the editor's rewrite and any key update, one cohort +per changed document. It is the last gate before the merge to `next`, so that every change on +`next` is one the cohort has accepted, and the release branch needs no cohort of its own unless it +changes specification text. A fresh cohort runs after every change to the pull request's own +text that follows a run, and after every void run; a text that failed is never run again +unchanged. A version number or date that `PUBLISHING.md` or step 5 of `CONTRIBUTING.md` has the +editor set is not such a change, and neither is a merge of `next` into the branch that brings in +text another cohort accepted and alters no sentence this change wrote. ## The record -Each run has a file under `acceptance/runs/`, named `NNN-.md` with `NNN` the next number in -the one sequence shared by the three documents, stating: the document, the pull request and the -commit read, the cohort size and model, the run number in that pull request, the marks per -reader per question, the findings applied, and whether it passed. The changelog line for the -change names the file. CI refuses a specification change whose pull request adds no run file; -it checks for at least one, and a pull request that changes two documents adds a run for each. - -Each published version's changelog entry carries the lines of the changes it contains, so a -version's cohort record is the set of runs those lines name. A version with no such line has not -been accepted and MUST NOT be described as published. The question sets and answer keys live -under `acceptance/` in this repository so that a run can be repeated and compared. +Each run has a file under `acceptance/runs/`, named `NNN--run-.md` with `NNN` the next +number in the one sequence shared by the three documents and `` the run number within the pull +request, stating: the document; the pull request and the commit read; the cohort size and model, +with the model version each reader reported; how the readers were isolated and the command they +were started with; any reader set aside and why; who marked and, for a delegated marker, how it +was started; the marks per reader per keyed item, with each miss and the reader's reasoning in +brief; every passage named under question 4, with the words quoted or the description given, its +clause and the readers naming it; each finding and whether it lies in text the change wrote or +altered; any point the rules did not settle; whether the run was made under a process change +before it merged and, if so, at which commit; the findings applied afterwards; the verdict; and +a link to the answer sheets. + +The answer sheets are not kept in this repository. Every reader's whole output as its process +returned it, the exact prompt every reader received, the command the readers were started with, +and the instruction a delegated marker was given are posted in one or more comments on the pull +request, linked from the record and never edited, hidden or deleted after posting. + +The changelog line for the change names the record. CI refuses a specification change whose pull +request adds no run file; it checks for at least one, and a pull request that changes two +documents adds a run for each. Each published version's changelog entry carries the lines of the +changes it contains, so a version's cohort record is the set of runs those lines name. A version +with no such line has not been accepted and MUST NOT be described as published. The question sets +and answer keys live under `acceptance/` in this repository so that a run can be repeated and +compared. + +`acceptance/ROUTED.md` lists every finding routed from a run, open until a change that fixes +its passage passes a cohort, or until the editor closes it with a recorded reason. The editor +reads that list before cutting a release. + +## Changes to this process + +The process files are this file, `acceptance/README.md`, `acceptance/runs/README.md`, +`acceptance/ROUTED.md`, `CONTRIBUTING.md` and the checks that enforce them: `tools/changes.py`, +the key-quotation checks in `spec-qa.py`, the CI workflow and the pull request template. A change +to any of them, or to a question set, is a change to this process. They are not specifications, +and a change to them is not tested by a cohort: the cohort measures whether a low-strength reader +can follow a specification, and the process is followed by the editor. Such a change names the +runs or the case that showed the need for it, and goes through steps 1, 3 and 6 of +`CONTRIBUTING.md`, with no scope, key or cohort step; its review reads every process file it +changes. It applies to every run whose readers start after it merges; a run made before then may +be made under it, following all of it at a commit named in the record before the readers start. +It is recorded by a line added at the top of [acceptance/CHANGES.md](acceptance/CHANGES.md), +naming its pull request, not in the changelog, which is for the three specifications. A change to +a key or a fixture follows `acceptance/README.md`. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fa1743e..013bb9d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -29,7 +29,9 @@ review. It is a patch version when released. SHOULD or MAY in a numbered section or appendix. It needs everything a clarity edit needs, plus the five tests in SCOPE.md answered in the pull request, and it is at least a minor version when released. `tools/changes.py` tells the two apart mechanically and CI runs it on every pull -request. +request. A change to the process files ACCEPTANCE.md names, this file among them, is neither: +ACCEPTANCE.md says how it is made, and it is recorded in `acceptance/CHANGES.md`, not in +CHANGELOG.md. ## What a pull request contains @@ -55,20 +57,26 @@ survives, the change is yours. the editor sets the pre-release header. 2. **Scope.** For an obligation, the editor confirms each SCOPE.md test. A failed test closes the pull request with a row in `DECLINED.md`. -3. **Review and rewrite.** An independent review of the diff against the document, then the - editor's rewrite, pushed to the pull request branch where the contributor allows it or to an - editor's branch that supersedes it. +3. **Review and rewrite.** An independent review of the diff against the whole document, by a + reviewer who did not draft the change, then the editor's rewrite, pushed to the pull request + branch where the contributor allows it or to an editor's branch that supersedes it. The pull + request names the reviewer and links or contains the review. 4. **Keys.** Where the change touches a sentence an acceptance key quotes, or adds a load-bearing - obligation, the editor updates the key before the cohort runs, never after. -5. **The cohort.** The acceptance test in ACCEPTANCE.md runs on the final text. Its record is - committed under `acceptance/runs/`, the `Unreleased` changelog line names it, and CI refuses - a specification change that lacks one. For an obligation, the editor sets the document's - header to `-dev, unpublished` in the same commit if it still names a published - version. A wording change after the run means a fresh run. + obligation, the editor updates the key before the cohort runs, never after. The key's + required elements stay what its question asks for; `acceptance/README.md` says what a key + update may and may not do. +5. **The cohort.** The acceptance test in ACCEPTANCE.md runs on the final text, one cohort per + changed document. Its record is committed under `acceptance/runs/`, the `Unreleased` + changelog line names it, and CI refuses a specification change that lacks one. For an + obligation, the editor sets the document's header to `-dev, unpublished` in the + same commit if it still names a published version. A change to the pull request's own text + after the run means a fresh run, as ACCEPTANCE.md says under "When it runs". A finding the + run routes to `acceptance/ROUTED.md` is not fixed in this pull request. 6. **Merge** into `next` by merge commit, and the branch is deleted. -Steps 2 to 5 are done by a person with a Claude Code session, not by CI, and they happen in -batches. A quiet pull request is not a declined one; a declined one says so. +Steps 2 to 5 are the editor's, not CI's, and they happen in batches. The editor may have an +agent draft, review or mark, as ACCEPTANCE.md allows for marking; every decision in these steps +stays the editor's. A quiet pull request is not a declined one; a declined one says so. ## Versions and releases diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md new file mode 100644 index 0000000..a076ff1 --- /dev/null +++ b/acceptance/CHANGES.md @@ -0,0 +1,36 @@ +# Changes to the acceptance process + +Changes to the process files that [ACCEPTANCE.md](../ACCEPTANCE.md) names, recorded here as it +describes: a line is added at the top for each change, naming its pull request. The process is +not a specification and is not versioned, and a change to it is not tested by a cohort. Changes +to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). + +- Pull request NNN: the test is made to prove what it exists to prove, the clarity of the + document to a cold reader, after thirty-four runs on two changes (19 to 30, 31 to 33 and 35 + to 52) in which keyed answers reached five of five and the runs still failed. Four defects in + the process were found. Keys required elements the question never asked for (detector 2b's + "need not remove or forbid", detector 2c's "at a version tracked together", and on pull + request 14 method 2d's order of the kinds), so readers who answered the question as put were + marked wrong: now each key lists, per item, the elements an answer must state, one for + each part of the question, and the questions are reworded to ask for exactly those; anything + else in the key only catches a wrong assertion. A passage two readers found confusing stayed + open whilst any single reader of a later cohort named any part of its paragraph, so cleared + passages kept coming back: now a finding is settled run by run, raised by two readers of one + run and clear when the next names it by fewer than two. A finding landed on text the change + never touched, so a change's cohort acted as a review of the whole document and could not + converge: now a finding in text the change wrote or altered blocks the change, and a finding + elsewhere is routed to `acceptance/ROUTED.md` and to a change of its own, whilst the keyed + questions still prove the whole document on every run. Markers invented judgements the rules + left open and later runs cited them as precedent: now an open point is marked for the reader + and settled in this file or the key before the next run, and a run record states no judgement + the rules do not. The rule that any miss whose reasoning quoted the document was a finding + whatever the count is replaced by the key naming the wrong readings the text invites, which + count against the document on one reader, every other single miss being tolerated by the + all-but-one threshold. Question 2's instruction asks for the sentence or sentences relied on, + since three keys rest on two. The cohort, the packet, set-aside readers, the posted sheets, + the fuller record and the route for a process change are kept from the earlier proposal in + pull request 13, which this change supersedes. Relative to the rules on `next`, the + traceable-miss row is narrower and the confusion row is applied to the change rather than + the document; both are recorded here as the reason the earlier rules could not converge, and + neither changes a mark already given. Prompted by runs 19 to 30 on pull request 14 and runs + 31 to 52 on the defined-terms change. diff --git a/acceptance/README.md b/acceptance/README.md index 4cfb6ec..49a097b 100644 --- a/acceptance/README.md +++ b/acceptance/README.md @@ -2,8 +2,10 @@ The question sets and answer keys that [ACCEPTANCE.md](../ACCEPTANCE.md) requires. One directory per specification document, each holding the fixed questions a cold reader answers and the key an -editor marks against. The fixture cases the application question uses live in `fixtures/`, and -the record of every run lives in `runs/`. +editor marks against. The fixture cases the application question uses live in `fixtures/`, the +record of every run lives in `runs/`, findings routed out of a run to a change of their own are +listed in [ROUTED.md](ROUTED.md), and changes to the acceptance process itself are recorded +in [CHANGES.md](CHANGES.md). | Document | Questions | Key | | ------------------ | ------------------------------------------------ | ------------------------------------ | @@ -14,24 +16,47 @@ the record of every run lives in `runs/`. ## How a run works 1. The editor identifies which documents the change touched. Only those are tested. -2. For each, five or more fresh low-strength readers each receive the document's current text, - its `QUESTIONS.md`, and the fixture the questions name. Nothing else: no key, no changelog, no - critique findings, no other reader's answers. -3. The editor marks each answer sheet against `KEY.md` and applies the pass criteria in - `ACCEPTANCE.md`. -4. The result is recorded as a file under [runs/](runs/), with cohort size, model, run number, - marks and the findings applied, and the change's `Unreleased` changelog line names that file. - A run that fails is followed by a change to the text, never to the questions, and a fresh - cohort. +2. For each, an odd number of fresh low-strength readers, at least five, each isolated as + `ACCEPTANCE.md` describes, receive the document's current text, its `QUESTIONS.md`, and the + fixture the questions name. Nothing else: no key, no changelog, no critique findings, no other + reader's answers. +3. The editor, or a separate marker the editor delegates to, marks each answer sheet against + `KEY.md` under the marking rules in `ACCEPTANCE.md` and applies its pass criteria. +4. The result is recorded as a file under [runs/](runs/), stating what `ACCEPTANCE.md` lists, + with the answer sheets posted on the pull request and linked from it, and the change's + `Unreleased` changelog line names that file. A run that fails is followed by a change to the + text the findings name and a fresh cohort; a finding on text the change did not write or + alter is added to [ROUTED.md](ROUTED.md) instead. + +## How a key is written + +A key says, for each keyed item, exactly what is marked and nothing a marker has to guess: + +- **Question 1**: the obligations a bullet may name, and the assertions that make a bullet wrong. +- **Question 2**: for each item, the elements an answer is required to state, numbered, one for + each part of the question; the assertions that make an answer wrong; and the sentence or + sentences of the document the answer rests on, quoted verbatim. The required elements are + what the question asks for, no more: an element the question does not ask for is not + required, however load-bearing, until the question is reworded to ask for it. +- **Question 3**: for each item, the grade or grades accepted, any condition on the reasoning, + and the wrong readings the text invites, which count against the document when one reader + takes them. + +Everything else in a key is explanation for the marker and never marks an answer down. ## Keeping the keys honest A key quotes the document. When a clause is reworded, the key line that quotes it is updated in the same commit, and the change is one the acceptance run itself then tests. A key that no longer -matches the document is a defect in this directory, not a reason to mark a reader wrong. +matches the document is a defect in this directory, not a reason to mark a reader wrong. A key +update under step 4 of `CONTRIBUTING.md` changes a quotation to follow the text, or adds a keyed +item for a new load-bearing obligation; it does not change what an existing item requires. A +change to what an item requires, or to a question, is a change to the process and is recorded in +[CHANGES.md](CHANGES.md) with its reason, before the readers of the next run start. Questions are fixed so that runs can be compared across versions. Adding a keyed question is -allowed when a new clause is load-bearing; removing or softening one to make a run pass is not. +allowed when a new clause is load-bearing; removing or softening one to make a run pass is not; +rewording one is allowed so that it asks for what its key requires, and for no other reason. The reader may quote a passage it found confusing; it is never asked to rate the document, say what would make a passage clearer, or propose a change: the cohort confirms comprehension and does not design the text. diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md new file mode 100644 index 0000000..bb82718 --- /dev/null +++ b/acceptance/ROUTED.md @@ -0,0 +1,32 @@ +# Passages routed out of a cohort + +A finding is a passage two or more readers of one cohort named as confusing, under question 4, +that lies outside the text the change being tested wrote or altered. `ACCEPTANCE.md` routes such +a finding here rather than blocking the change whose cohort raised it, so that each change is +judged on what it changed and the document is still made clearer, one change at a time. The +editor reads this list before cutting a release. + +A finding is open until a change that reworks its passage passes a cohort, when the line names +that run and moves below the rule, or until the editor closes it with a reason recorded here. A +finding raised again by a later run is noted on its line, not listed twice. + +## Open + +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | -------------------------------------- | +| `TOOLING-SPEC.md` | 9 and 9.1 | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 2, 4.2, 8.1 | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 9.2 | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 4.3 | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | run 33 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 4.1 | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 5.3 | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | open | +| `DETECTOR-SPEC.md` | 7.1 | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | 8.1 | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | 5.3 | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | +| `SPEC.md` | 1 (Owner) | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | open | +| `SPEC.md` | 4 | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 (one reader, 29) | open; a method question for the editor | +| `SPEC.md` | 2 and 3.1 | "technique" used for a way of writing the Rule in section 2 and for a way of searching in clause 3.1 | runs 29, 38 | pull request 14 rewords both | + +## Closed + +None yet. diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index 132e6cb..b0a3f97 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -1,8 +1,9 @@ # Answer key for DETECTOR-SPEC.md -Marked against the document at the version named in the changelog entry that records the run. -Quotations below are from the document; when a clause is reworded, update the quotation here in -the same commit. +Marked against the document at the commit the readers read, with this key, its question set and +fixture at that same commit. Quotations below are from the document; when a clause is reworded, +update the quotation here in the same commit. `ACCEPTANCE.md` says how each item is marked; +`acceptance/README.md` says what each part of this key is for. ## 1. Restatement @@ -11,67 +12,93 @@ bullet asserting an obligation the document does not state, is a correct restate - Support bespoke rules written by the project ([4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it)). - Provide a harness that runs a single rule against supplied code ([4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code)). -- Allow a stable identifier per rule and print it, unaltered, with every finding ([4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding)). +- Allow a stable identifier per rule and print it, unaltered, with every finding ([4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding)); a + bullet that says each rule carries a stable identifier names this obligation. - Be invocable locally by the practitioner, over a subset down to one file, with the result in - the command's own output and never only through a hosted service or CI-only mode ([5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) to [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally)). + the command's own output and never only through a hosted service or CI-only mode ([5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) to [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally)); + a bullet that says every finding, or each finding, must appear in that output names [5.3](../../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran), + whose summary may leave findings out, and is not marked wrong for the plural. - Resolve a printed identifier to documentation; for bundled rules from the installed copy, offline, with the documentation shipped and versioned with the rule ([6.1](../../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) to [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together)). - Make any inline suppression route detectable or disableable ([7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable)). Wrong if it asserts: the detector must enforce a project's suppression policy; the detector must declare a version to conform; the detector must list a project's active defences or produce an -agent summary (those belong to the toolchain specification). +agent summary (those belong to the toolchain specification). A bullet that states what +conformance means, or restates a SHOULD as advice, asserts no obligation and is neither correct +nor wrong. + +Wrong readings the text invites, which count against the document on one reader: none named. ## 2. Load-bearing statements -- **2a.** Conformance is satisfying every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression), judged on evidence; a detector - that predates the document can conform, and the declaration is optional. Key quotation, section - 8: "A Detector Conforms if it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression)." Supporting, [8.1](../../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps): "A - maintainer MAY declare the version of this document the detector conforms to". An answer that - says a declaration is required, or that partial satisfaction is conformance, is wrong. -- **2b.** It must make the route either disableable by configuration or detectable by a rule the - project can write or a mechanical check it documents; it is not required to remove or forbid - it. Key quotation, [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable): "For each one it MUST do at least one of two things, and either alone - satisfies this clause: make the route disableable by configuration; or make the route - detectable, by a Rule the project can write in the Detector itself or by a mechanical check the - Detector documents". An answer that says the detector must forbid the route, or must require a - reason ([7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason) is SHOULD), is wrong. -- **2c.** With the rule, in the installed copy, at a version tracked together, and yes, offline. - Key quotations, [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access): "Resolution of a bundled rule's identifier MUST work from the installed - copy, without network access"; [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together): "A bundled rule's documentation MUST ship with the rule, at - a version tracked together". An answer naming the detector's website as an acceptable location - is wrong. +- **2a.** Required: (1) a detector conforms when it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression); + (2) yes, a detector that predates the document can conform, because conformance is judged on + the evidence and a declaration is not a condition of it. Wrong if it says a declaration is + required, or that partial satisfaction is conformance. Key quotation, section [8](../../DETECTOR-SPEC.md#8-conformance): "A Detector + Conforms if it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression)." Supporting, [8.1](../../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps): "A maintainer + MAY declare the version of this document the detector conforms to". "Judged on evidence", "the + declaration is optional" or "graded on evidence by anyone" each state the reason in element + (2); one of them is enough. +- **2b.** Required: (1) it must let the project switch the route off in its configuration, or + make the route detectable (by a rule the project can write, or by a mechanical check the + detector documents; naming either means of detection, or just "detectable", states this); + (2) yes, either one alone is enough. Wrong if it says the detector must remove or forbid the + route, or must require a reason ([7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason) is SHOULD). Key quotation, [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable): "For each one it + MUST do at least one of two things, and either alone satisfies this clause: make the route + disableable by configuration; or make the route detectable, by a Rule the project can write in + the Detector itself or by a mechanical check the Detector documents". An answer need not say + that the detector is not required to remove or forbid the route; that follows from element (2) + and the question does not ask it. +- **2c.** Required: (1) with the rule, in the installed copy; (2) at a version tracked together + with the rule; (3) yes, it must be reachable without network access. Wrong if it names the + detector's website, or any location reached over the network, as acceptable. Key quotations, + [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access): "Resolution of a bundled rule's identifier MUST work from the installed copy, without + network access"; [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together): "A bundled rule's documentation MUST ship with the rule, at a version + tracked together". "Shipped with the rule", "in the installed package" or "on disk with the + detector" each state element (1); "versioned with the rule" or "at the same version" states + element (2). ## 3. Application: Lintomatic 4.2 -| Clause | Intended verdict | Reasoning the fixture supports | -| ------ | ---------------- | -------------------------------------------------------------------------------------------------------------------- | -| [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it) | Yes | Project rules via the `Rule` interface, first class in every report | -| [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, with a distinguishing exit code | -| [4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding) | Yes | `LM-NNNN` fixed at first release; project names taken as given, not rewritten; printed on every finding line | -| [5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) | Yes | Local, no account, no server, no network | -| [5.2](../../DETECTOR-SPEC.md#52-the-detector-must-support-invocation-over-a-subset-at-minimum-a-single-file) | Yes | `check ` accepts a single file | -| [5.3](../../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | Yes | One line per finding on standard output whether or not the HTML report is enabled | -| [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally) | Yes | The hosted dashboard shows nothing the command line does not print | -| [6.1](../../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | Yes | `explain ` resolves a printed identifier; project rules resolve to their declared `doc` string | -| [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access) | Yes | `explain` reads the installed package; there is no website | -| [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) | No | Two of forty bundled rules print `No documentation declared`; every identifier a bundled rule can print MUST resolve | -| [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | Yes | The ignore comment is disableable with `ignore_comments = false` | - -Section [8](../../DETECTOR-SPEC.md#8-conformance) verdict: **does not conform**, because [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) fails and partial conformance MUST NOT be -described as conformance. A reader who grades [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) Partial on the grounds that thirty-eight resolve -has read the clause differently and that counts against the document; a reader who grades [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) Yes -because they missed the last paragraph of the fixture has misread the fixture, not the document. - -Clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation): **does not change the verdict**. It is a SHOULD, and Lintomatic's failure to check its -own identifiers in CI is a missed SHOULD, not a further MUST failure; the MUST that fails is [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together). - -A reader who marks [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) No because the comment takes no reason has confused [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) with [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason), which is -SHOULD; that counts against the document if the reasoning quotes [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) or [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason). +| Clause | Accepted grade | Condition on the reasoning, and what the fixture supports | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it) | Yes | Project rules via the `Rule` interface, first class in every report | +| [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, with a distinguishing exit code | +| [4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding) | Yes | `LM-NNNN` fixed at first release; project names taken as given, not rewritten; printed on every finding line | +| [5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) | Yes | Local, no account, no server, no network | +| [5.2](../../DETECTOR-SPEC.md#52-the-detector-must-support-invocation-over-a-subset-at-minimum-a-single-file) | Yes | `check ` accepts a single file | +| [5.3](../../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | Yes | One line per finding on standard output whether or not the HTML report is enabled; a reader who notes that the HTML report need not be named for that reason is right | +| [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally) | Yes | The hosted dashboard shows nothing the command line does not print | +| [6.1](../../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | Yes | `explain ` resolves a printed identifier; project rules resolve to their declared `doc` string; the two undocumented bundled rules are [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together)'s failure | +| [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access) | Yes | `explain` reads the installed package; there is no website | +| [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) | No | Two of forty bundled rules print `No documentation declared`; every identifier a bundled rule can print MUST resolve | +| [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | Yes | The ignore comment is disableable with `ignore_comments = false` | + +No grade above carries a condition on the reasoning: the accepted grade is correct whatever the +reasoning, and a reader who adds a caveat about what the fixture does not say and still gives +the accepted grade is correct. + +Section [8](../../DETECTOR-SPEC.md#8-conformance) verdict: **does not conform**; a reader is correct who says so and names [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), the +two undocumented bundled rules, or partial conformance not counting, as the reason. + +Clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation): **does not change the verdict**; a reader is correct who says so and gives as the +reason that it is a SHOULD, or that the MUST which fails is [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together). + +Wrong readings the text invites, which count against the document on one reader: + +- [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) graded Partial on the grounds that thirty-eight of forty resolve, which has read the + clause as admitting a proportion. +- [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) graded No because the comment takes no reason, where the reasoning quotes [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) or [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason), + which has read [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason)'s SHOULD into [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable). + +A reader who grades [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) Yes because they missed the last paragraph of the fixture has misread the +fixture, which is the reader's miss. ## 4. Confusion -No keyed answer. Record every passage quoted. Two or more readers quoting the same passage is a -finding that blocks until the text is changed and a fresh cohort no longer quotes it. What to -change is the editor's decision; the reader is not asked and any suggestion it volunteers is -discarded unread. +No keyed answer. Record every passage named, as `ACCEPTANCE.md` defines naming and the same +passage. A passage named by two or more readers of the run is a finding: it blocks the change +where it lies in text the change wrote or altered, and is routed to `acceptance/ROUTED.md` +otherwise. What to change is the editor's decision; the reader is not asked and any suggestion +it volunteers is discarded unread. diff --git a/acceptance/detector/QUESTIONS.md b/acceptance/detector/QUESTIONS.md index 58d06d1..2666b2f 100644 --- a/acceptance/detector/QUESTIONS.md +++ b/acceptance/detector/QUESTIONS.md @@ -10,14 +10,15 @@ In your own words and in at most six bullets, what does this document require of ## 2. Load-bearing statements -Answer each in one sentence, followed by the sentence of the document you are relying on, quoted. +Answer each part of each question in a sentence or two, then quote the sentence or sentences of +the document you are relying on. - 2a. What does it mean for a detector to conform, and can a detector that was written before this document existed, and has never heard of it, conform? - 2b. A detector ships an inline ignore comment. What must the detector do about that route for - it to conform? -- 2c. Where must the documentation for a bundled rule's identifier live, and must it be - reachable without network access? + it to conform, and is doing one of those things enough? +- 2c. Where must the documentation for a bundled rule's identifier live, at what version, and + must it be reachable without network access? ## 3. Application diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index 3ca1a80..182d6b2 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -1,8 +1,9 @@ # Answer key for SPEC.md -Marked against the document at the version named in the changelog entry that records the run. -Quotations below are from the document; when a clause is reworded, update the quotation here in -the same commit. +Marked against the document at the commit the readers read, with this key, its question set and +fixture at that same commit. Quotations below are from the document; when a clause is reworded, +update the quotation here in the same commit. `ACCEPTANCE.md` says how each item is marked; +`acceptance/README.md` says what each part of this key is for. ## 1. Restatement @@ -18,62 +19,80 @@ bullet asserting an obligation the document does not state, is a correct restate with the rule ([3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation)). - Record a defect whose fix is deferred where the project's other decisions are enumerable, rather than only mentioning it (section [2](../../SPEC.md#2-when-the-method-applies)). +- Take a baseline, a suppression or an instance left unfixed to the owner, never on the + practitioner's own authority (section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom)). Wrong if it asserts: the practitioner fixes the reported instance first and defends afterwards; a test may serve as the detector; an agent may baseline the remaining instances. +Wrong readings the text invites, which count against the document on one reader: none named. + ## 2. Load-bearing statements -- **2a.** Attribute the defect to a class, express it as a detector rule, prove the rule fires, - then sweep and fix every instance, in that order. Key quotations, section [3](../../SPEC.md#3-the-method): "Six clauses, in +- **2a.** Required: (1) the four steps before the fix: attribute the defect to a class, express + the class as a rule in a detector, prove the rule fires, and sweep the codebase; (2) in that + order, with the fix after the sweep. Wrong if it has the fix before the rule, or the sweep + before the proof, or a test as the detector. Key quotations, section [3](../../SPEC.md#3-the-method): "Six clauses, in order."; [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance): "The Practitioner MUST run the Rule across the entire codebase and record the total Instance count before fixing anything, and MUST then fix every Instance found rather - than only the one". An answer that has the fix before the rule, or the sweep before the proof, - is wrong. -- **2b.** The owner of the codebase, always a human; an agent may not. Key quotation, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): - "An Agent MUST NOT Baseline, suppress or knowingly leave an Instance unfixed on its own - authority, whatever the Instance count turns out to be." An answer that lets the practitioner - decide because the count was small, or because the project recorded no rule, is wrong (the - default owner paragraph in the terminology entry for Owner still requires a recorded decision). -- **2c.** All six clauses of section [3](../../SPEC.md#3-the-method) were followed for that defect, and the verdict rests on the - reviewer reproducing the red and green runs, not on the report. Key quotations, section [7](../../SPEC.md#7-conformance): "A - remediation Conforms if all six clauses of section [3](../../SPEC.md#3-the-method) were followed for that Defect, and the - verdict that they were rests on reproduction"; "A verdict on a remediation or a Defence MUST - rest on reproduction, not on the report". An answer that accepts the pull request description - as the verdict is wrong, and so is one that states the six clauses and says nothing of - reproduction. + than only the one". An answer that names the steps by their clause numbers in order states + both elements. +- **2b.** Required: (1) the owner of the codebase, the human who holds that authority; (2) no, + an agent may not do so on its own authority. Wrong if it lets the practitioner or an agent + decide because the count was small, because the project recorded no owner, or at all. Key + quotation, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): "An Agent MUST NOT Baseline, suppress or knowingly leave an Instance + unfixed on its own authority, whatever the Instance count turns out to be." "The owner" names + the defined term and states that the decision is a human's; an answer need not add "a human" + to be correct. The default owner paragraph in the terminology entry for Owner still requires a + recorded decision, so an answer naming whoever instructed the practitioner, for a project that + has recorded nobody, names the owner and is correct. +- **2c.** Required: (1) all six clauses of section [3](../../SPEC.md#3-the-method) were followed for that defect; (2) the verdict + rests on reproduction, the reviewer rerunning the defence, not on the report. Wrong if it + accepts the pull request description or the practitioner's report as the verdict, or says + that most clauses suffice. Key quotations, section [7](../../SPEC.md#7-conformance): "A remediation Conforms if all six clauses + of section [3](../../SPEC.md#3-the-method) were followed for that Defect, and the verdict that they were rests on + reproduction"; "A verdict on a remediation or a Defence MUST rest on reproduction, not on the + report". "Reproduction" or "reproducing the runs" states element (2); the red and green runs + need not be named. ## 3. Application: the ledger remediation -| Clause | Intended verdict | Reasoning the fixture supports | -| ----------------------------------------------------------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | Class, hazard and the next wider rule are named; Partial or No is accepted where the reader notes that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | -| [3.2](../../SPEC.md#32-build-the-net) | Yes | A rule in the project's static analyser, reading code, not a test | -| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | Commit B is red on the fixture and the originating instance and survives as its own commit | -| [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | Five findings recorded, but one instance was left unfixed behind an inline ignore on the agent's own authority; Partial accepted where the reasoning names that suppression | -| [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | The rule is permanent and blocking, but the suppression was not a recorded decision; [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | -| [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | The message is terse and carries an identifier, but the documentation lives on a wiki, not versioned with the rule; Partial accepted where the reasoning quotes the versioning sentence | +| Clause | Accepted grades | Condition on the reasoning, and what the fixture supports | +| ----------------------------------------------------------------------------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | No condition. Class, hazard and the next wider rule are named; Partial or No is the reading that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | +| [3.2](../../SPEC.md#32-build-the-net) | Yes | A rule in the project's static analyser, reading code, not a test | +| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | Commit B is red on the fixture and the originating instance and survives as its own commit | +| [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | The reasoning names the instance left unfixed behind an inline ignore on the agent's own authority; five findings were recorded | +| [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | The reasoning names the suppression no recorded decision covers; the rule is otherwise permanent and blocking. [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | +| [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | The reasoning names the documentation on a wiki, not versioned with the rule; the message is terse and carries an identifier | -Section [7](../../SPEC.md#7-conformance): the **remediation does not conform**, because [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) and [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) were not followed. The -**defence does not conform** either, because [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) fails; a reader who says the defence conforms -because the sweep is what failed has missed [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) and that counts against the document only if the -reasoning quotes the section [7](../../SPEC.md#7-conformance) definition of a conforming defence. +Section [7](../../SPEC.md#7-conformance): the **remediation does not conform**; a reader is correct who says so and names +[3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed. The **defence does not conform** either; a reader is +correct who says so and names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the reasons, or [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which +section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent. Section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): the **inline ignore on `Report::render()`** was the owner's decision, not the -practitioner's. An answer naming instead the decision not to build the wider rule is half right: -that is also listed in section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) as the owner's to take, and an answer naming both is correct. -An answer naming neither is wrong. +practitioner's. A reader who names it is correct, with or without the decision to leave the +wider rule unbuilt, which section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) also gives to the owner. A reader who names only the wider +rule, or neither, is wrong. + +Wrong readings the text invites, which count against the document on one reader: + +- [3.5](../../SPEC.md#35-enforce-permanently-and-block) graded Yes because the rule fails the build, which has not applied the clause's opening + sentence on unrecorded suppression. +- [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) graded Yes because the URL resolves, which has read the versioning sentence of [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) + differently. +- The defence graded as conforming because the sweep is what failed, where the reasoning quotes + section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence. -A reader who grades [3.5](../../SPEC.md#35-enforce-permanently-and-block) Yes because the rule fails the build has not applied the clause's opening -sentence on unrecorded suppression, and that counts against the document. A reader who grades -[3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) Yes because the URL resolves has read the versioning sentence of [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) -differently, which counts against the document; a reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the -fixture. A reader who attributes the suppression to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) rather than [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) has read the narrowing -passage of [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) as covering it, which is a fair reading and is not marked wrong. +A reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the fixture, which is the reader's miss. A reader who +attributes the suppression to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) rather than [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) has read the narrowing passage of [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) as +covering it, which is a fair reading and is not marked wrong. ## 4. Confusion -No keyed answer. Record every passage quoted. Two or more readers quoting the same passage is a -finding that blocks until the text is changed and a fresh cohort no longer quotes it. What to -change is the editor's decision; the reader is not asked and any suggestion it volunteers is -discarded unread. +No keyed answer. Record every passage named, as `ACCEPTANCE.md` defines naming and the same +passage. A passage named by two or more readers of the run is a finding: it blocks the change +where it lies in text the change wrote or altered, and is routed to `acceptance/ROUTED.md` +otherwise. What to change is the editor's decision; the reader is not asked and any suggestion +it volunteers is discarded unread. diff --git a/acceptance/method/QUESTIONS.md b/acceptance/method/QUESTIONS.md index 072a3c0..34fa82d 100644 --- a/acceptance/method/QUESTIONS.md +++ b/acceptance/method/QUESTIONS.md @@ -11,7 +11,8 @@ do when they are handed a defect? ## 2. Load-bearing statements -Answer each in one sentence, followed by the sentence of the document you are relying on, quoted. +Answer each part of each question in a sentence or two, then quote the sentence or sentences of +the document you are relying on. - 2a. What does a practitioner do before fixing the defect they were handed, and in what order? - 2b. Who may adopt a baseline or suppress an instance, and may an agent do so on its own diff --git a/acceptance/runs/README.md b/acceptance/runs/README.md index 4994f74..3f7ad64 100644 --- a/acceptance/runs/README.md +++ b/acceptance/runs/README.md @@ -1,15 +1,22 @@ # Acceptance run records -One file per cohort run, `NNN-.md`, numbered in the one sequence the three documents -share. Runs 1 to 15 predate this directory and are summarised in the changelog entries for -1.0.0 and 1.0.1 of the method specification, 1.0.0 of the detector specification and 0.2.0 of -the toolchain specification; numbering here continues from 16. +One file per cohort run, `NNN--run-.md`, numbered in the one sequence the three +documents share, with `` the run number within its pull request. Runs 1 to 15 predate this +directory and are summarised in the changelog entries for 1.0.0 and 1.0.1 of the method +specification, 1.0.0 of the detector specification and 0.2.0 of the toolchain specification; +numbering here continues from 16. -A record states, in this order: the document; the pull request and the commit the readers were -given; the cohort size and model; the run number within that pull request; the marks per reader -per question against the key; the passages quoted as confusing and by how many readers; the -findings applied to the text as a result; and the verdict against the criteria in -`ACCEPTANCE.md`. Answer sheets are not pasted in full; the marks are the record. +A record states what `ACCEPTANCE.md` lists under "The record", in this order: the document; the +pull request and the commit the readers were given; the cohort size and model, with the model +version each reader reported; how the readers were isolated and the command they were started +with; any reader set aside and why; who marked and, for a delegated marker, how it was started; +the marks per reader per keyed item, with each miss and the reader's reasoning in brief; every +passage named under question 4, with the words quoted or the description given, its clause and +the readers naming it; each finding and whether it lies in text the change wrote or altered; any +point the rules did not settle; whether the run was made under a process change before it merged +and, if so, at which commit; the findings applied afterwards; the verdict; and a link to the +answer sheets posted on the pull request. Earlier records show the style only; where one differs +from this order, this file wins. Answer sheets are not kept here; the marks are the record. The `Unreleased` changelog line for the change names the record, and `tools/changes.py` refuses a specification change whose pull request adds no file here. diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index 5a2602b..c11ae32 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -1,8 +1,9 @@ # Answer key for TOOLING-SPEC.md -Marked against the document at the version named in the changelog entry that records the run. -Quotations below are from the document; when a clause is reworded, update the quotation here in -the same commit. +Marked against the document at the commit the readers read, with this key, its question set and +fixture at that same commit. Quotations below are from the document; when a clause is reworded, +update the quotation here in the same commit. `ACCEPTANCE.md` says how each item is marked; +`acceptance/README.md` says what each part of this key is for. ## 1. Restatement @@ -25,61 +26,72 @@ bullet asserting an obligation the document does not state, is a correct restate defences on its own source ([8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation), [8.2](../../TOOLING-SPEC.md#82-a-shipped-toolchain-must-run-its-own-bundled-defences-on-its-own-source)). Wrong if it asserts: the toolchain must be a single product; a declaration is required to -conform; the agent summary (section [7](../../TOOLING-SPEC.md#7-agent-context)) is a MUST. +conform; the agent summary (section [7](../../TOOLING-SPEC.md#7-agent-context)) is a MUST. A bullet that calls agent support or the +self-audit a further condition "where it applies", or says a declaration is not a condition, +asserts no obligation and is neither correct nor wrong. + +Wrong readings the text invites, which count against the document on one reader: none named. ## 2. Load-bearing statements -- **2a.** At the project level; the toolchain is whatever the project has assembled from - third-party, first-party and project-level parts. Key quotation, section [9](../../TOOLING-SPEC.md#9-conformance): "A project's - Toolchain Conforms if every MUST in sections [4](../../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) to [6](../../TOOLING-SPEC.md#6-the-project-record) holds across the assembled parts, wherever - each part came from". An answer that says conformance is graded per tool, or that only a - packaged product can conform, is wrong. -- **2b.** One grade; which part satisfied each clause is evidence, not a grade of its own. Key - quotation, section [9](../../TOOLING-SPEC.md#9-conformance): "The project-level verdict is a single grade; which part of the Toolchain - satisfied each clause is evidence for that grade, not a second grade." -- **2c.** Two, graded and declared separately, and neither implies the other. Key quotation, - [9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately): "The two verdicts MUST be graded and declared separately, and neither implies the other." - An answer giving one grade, or saying the artefact grade follows from the project grade, is - wrong. +- **2a.** Required: (1) at the project level; (2) the toolchain is whatever the project has + assembled to run its checks through, from third-party, first-party and project-level parts. + Wrong if it says conformance is graded per tool, or that only a packaged product can conform. + Key quotation, section [9](../../TOOLING-SPEC.md#9-conformance): "A project's Toolchain Conforms if every MUST in sections [4](../../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) to [6](../../TOOLING-SPEC.md#6-the-project-record) + holds across the assembled parts, wherever each part came from". An answer that says everything the + project runs its checks through, or the assembled parts from wherever they came, or that lists + the three kinds of part, states element (2). +- **2b.** Required: (1) one grade; (2) which part satisfied each clause is evidence for that + grade, not a grade of its own. Wrong if it gives a grade per part. Key quotation, section [9](../../TOOLING-SPEC.md#9-conformance): + "The project-level verdict is a single grade; which part of the Toolchain satisfied each + clause is evidence for that grade, not a second grade." +- **2c.** Required: (1) two, graded and declared separately; (2) neither implies the other. + Wrong if it gives one grade, or says the artefact grade follows from the project grade or the + reverse. Key quotation, [9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately): "The two verdicts MUST be graded and declared separately, and + neither implies the other." ## 3. Application: the Zigpipe project -| Clause | Intended verdict | Reasoning the fixture supports | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap | -| [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)` | -| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | -| [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | -| [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | Lintomatic, then the tests, stopping on a Lintomatic failure | -| [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is accepted where the reasoning counts the two `(none)` routes, since 5.1 says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | -| [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration) | Yes | Derived from `lintomatic.toml`, not hand-maintained | -| [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | Yes | The `proj.*` rules appear in the same listing | -| [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) | Yes | `quality-exceptions.toml` is a defined path that `zig-qa run` loads | -| [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | A justification is required and an empty one rejected, but nothing rejects a justification that could be pasted onto any entry | -| [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | `zig-qa list --exceptions` uses the same format and operation as the rule listing | - -Either Partial or No is accepted for [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one); the reasoning must mention the generic-justification -requirement. A Yes on [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) with reasoning that quotes only the first sentence of the clause is a -finding against the document. - -Section [9](../../TOOLING-SPEC.md#9-conformance) verdict: **does not conform**, because [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) and [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) fail; partial conformance MUST NOT be -described as conformance. **Not with agent support**, because section [7](../../TOOLING-SPEC.md#7-agent-context) is not met (no summary). -**Section [8](../../TOOLING-SPEC.md#8-self-audit) does not apply**, because Zigpipe ships nothing to other projects; a reader who applies -[8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) to Zigpipe on the basis that Zigpipe's tooling includes a shipped package (zig-qa) has read -"where the Toolchain is one the project ships" (section [9](../../TOOLING-SPEC.md#9-conformance)) as reaching the consuming project, and that counts against the document. The -correct reading is that section [8](../../TOOLING-SPEC.md#8-self-audit) bears on zig-qa's own artefact grade, not on Zigpipe. - -Declaration: **does not affect the verdict**. Key quotation, [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance): "The declaration is optional." -A reader who says Zigpipe cannot be graded without one has read [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) wrongly. - -A reader who grades [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) Yes on the grounds that Lintomatic "mostly" conforms has read either [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) -or the detector document's section [8](../../DETECTOR-SPEC.md#8-conformance) differently; it counts against this document only if the -reasoning quotes [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification). A reader who grades [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) Yes because they missed the two undocumented rules -has misread the fixture. +| Clause | Accepted grades | Condition on the reasoning, and what the fixture supports | +| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap; a reader who notes that the detector specification was not supplied and grades from this document's summary of it is correct | +| [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)`. The two are blocking, since their firing stops `zig-qa run`; a reader who grades No on that condition is correct | +| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | +| [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | +| [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | Lintomatic, then the tests, stopping on a Lintomatic failure | +| [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | No condition. `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is the reading that counts the two `(none)` routes, since 5.1 says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | +| [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration) | Yes | Derived from `lintomatic.toml`, not hand-maintained | +| [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | Yes | The `proj.*` rules appear in the same listing | +| [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) | Yes or Partial | `quality-exceptions.toml` is a defined path that `zig-qa run` loads. Partial is accepted only where the reasoning says the fixture names a place for exceptions and none for calibrations or conventions, which the project record entry also covers | +| [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | The reasoning mentions that nothing rejects a justification that could be pasted onto any entry; a justification is required and an empty one rejected | +| [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | `zig-qa list --exceptions` uses the same format and operation as the rule listing | + +Section [9](../../TOOLING-SPEC.md#9-conformance) verdict: **does not conform**; a reader is correct who says so and names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or +[4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses. **Not with agent support**; a reader is correct who says so because +section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, there being no summary. **Section [8](../../TOOLING-SPEC.md#8-self-audit) does not apply**; a reader is correct +who says so because Zigpipe ships nothing to other projects, or because section [8](../../TOOLING-SPEC.md#8-self-audit) bears on +zig-qa's own artefact grade and not on Zigpipe. **The absence of a declaration does not affect +the verdict**; a reader is correct who says so. Key quotation, [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance): "The declaration is +optional." + +Wrong readings the text invites, which count against the document on one reader: + +- [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) graded Yes with reasoning that quotes only the first sentence of the clause, which has + not read the sentence on a generic justification. +- [8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) applied to Zigpipe on the basis that its tooling includes a shipped package, which has + read "where the Toolchain is one the project ships" (section [9](../../TOOLING-SPEC.md#9-conformance)) as reaching the consuming + project. +- [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) graded Yes on the grounds that Lintomatic "mostly" conforms, where the reasoning quotes + [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), which has read it as admitting a proportion. +- Zigpipe held ungradeable for want of a declaration, which has read [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) as a condition. + +A reader who grades [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) Yes because they missed the two undocumented rules has misread the +fixture, which is the reader's miss. ## 4. Confusion -No keyed answer. Record every passage quoted. Two or more readers quoting the same passage is a -finding that blocks until the text is changed and a fresh cohort no longer quotes it. What to -change is the editor's decision; the reader is not asked and any suggestion it volunteers is -discarded unread. +No keyed answer. Record every passage named, as `ACCEPTANCE.md` defines naming and the same +passage. A passage named by two or more readers of the run is a finding: it blocks the change +where it lies in text the change wrote or altered, and is routed to `acceptance/ROUTED.md` +otherwise. What to change is the editor's decision; the reader is not asked and any suggestion +it volunteers is discarded unread. diff --git a/acceptance/toolchain/QUESTIONS.md b/acceptance/toolchain/QUESTIONS.md index a4da266..35a2b91 100644 --- a/acceptance/toolchain/QUESTIONS.md +++ b/acceptance/toolchain/QUESTIONS.md @@ -11,7 +11,8 @@ toolchain? ## 2. Load-bearing statements -Answer each in one sentence, followed by the sentence of the document you are relying on, quoted. +Answer each part of each question in a sentence or two, then quote the sentence or sentences of +the document you are relying on. - 2a. At what level is toolchain conformance measured, and what is the toolchain? - 2b. A project's tooling is assembled from a third-party detector, a first-party package and From 36d87d18fe3b2aa586472a45b5888812a6831aae Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 11:43:12 +0100 Subject: [PATCH 02/34] Acceptance process: review 1 findings applied One route for each kind of change to the test (quotation and new item in the specification pull request, everything else a process change); ROUTED.md and runs/ are records, not process files; question 1 defined once; open points marked in a stated direction; quoting needs a six-word run; a finding lies in the change's text when any sentence of its paragraph changed; wrong readings stated as grade plus reasoning; keys ask only what their questions ask (detector 2a, toolchain 2c); the question 3 tables separate the condition from the explanation. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 123 ++++++++++++++++++------------ acceptance/CHANGES.md | 4 +- acceptance/README.md | 17 +++-- acceptance/ROUTED.md | 7 +- acceptance/detector/KEY.md | 17 ++--- acceptance/method/KEY.md | 37 ++++----- acceptance/toolchain/KEY.md | 49 ++++++------ acceptance/toolchain/QUESTIONS.md | 2 +- 8 files changed, 144 insertions(+), 112 deletions(-) diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index 84ab39b..c5bd8cd 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -51,8 +51,8 @@ makes it wrong; the question asks for exactly that and nothing the key does not asks for nothing the question does not. Each reader answers all of them. 1. **Restatement.** In their own words, what the document asks of its subject, in at most six - bullets. Keyed against the list of MUST clauses: a bullet is correct when it names an - obligation the document actually states. + bullets. Keyed against the key's list of the document's obligations: a bullet is correct when + it names one of them. 2. **Load-bearing statements.** Three or more questions, fixed per document, each asking for a short answer and the sentence or sentences of the document it rests on, quoted. Each has one @@ -97,34 +97,41 @@ answer sheets, and the facts the record needs that the sheets cannot show. Its m unless the editor shows that a rule here was misapplied, and a re-mark says which rule and why. Every mark is given by a rule in this file or in the key, as follows. Where neither settles a -mark, the marker gives the reader the benefit of the doubt, records the point in the run record, -and the editor settles it by a change to this file or to the key before the next run. A run -record states no judgement that is not written here or in the key; a judgement an earlier run -recorded is not a rule. +mark, the marker records the point in the run record and marks it as the open-point rule below +says, and the editor settles it in this file or in the key, as [Changing the test](#changing-the-test) +allows, before the next run. A run record states no judgement that is not written here or in +the key; a judgement an earlier run recorded is not a rule. - **Keyed items.** Question 1 as a whole is one keyed item, and so is each item of question 2 - and each answer of question 3 for which the key gives an intended answer. + and each answer of question 3 for which the key gives an accepted grade or answer. - **Reading an answer.** An answer is read for its sense, not its wording: a paraphrase, a synonym, or a defined term used in its defined sense states what it means. A reader quotes the - document when it gives consecutive words of the document, in the document's order and of any - length, in quotation marks or a block quote, verbatim apart from letter case, quotation-mark or - apostrophe style, and markdown link or emphasis markup. + document when it gives, in quotation marks or a block quote, a run of six or more consecutive + words of the document in the document's order, verbatim apart from letter case, quotation-mark + or apostrophe style, and markdown link or emphasis markup; a quotation that drops or changes a + word quotes the document by its longest such run. - **Question 1** is correct when at least four bullets each name an obligation the key lists and - no bullet asserts one the key marks wrong. + no bullet asserts one the key marks wrong. Several bullets that name different clauses of one + line of the key's list each count. A bullet asserting an obligation that is in neither list is + recorded as an open point for the key and is not counted either way. - **Question 2.** An answer is the reader's sentences and its quotation read together. It is correct when it states every element the key lists as required for that item, in the document's words or the reader's own, asserts nothing the key marks wrong, and quotes the document. An element stated only in the quotation is stated. Anything in the key beyond the required elements and the wrong assertions is explanation for the marker, and an answer is never marked down for leaving it out. -- **Question 3.** Each item is correct when its grade is one the key accepts and its reasoning - mentions what the key says it must mention for that grade. A grade the key accepts without a - condition is correct whatever the reasoning. +- **Question 3.** Each item is correct when its grade is one the key accepts and, where the key + sets a condition on the reasoning for that grade, the reasoning meets it. A grade the key + accepts with no condition is correct whatever the reasoning. A grade given on a stated + condition is read as the grade for the case the fixture presents, where the key says which + case that is; otherwise the item is unanswered. - **A miss that counts against the document.** The key names, for question 1 and for each item - of question 3, the wrong readings the text invites. A miss of that kind blocks whatever the - count, since one reader in five taking that reading means one agent in five acting on it. Every + of question 3, the wrong readings the text invites, each as a grade or answer together with + what the reasoning states or leaves out. A miss that matches one blocks whatever the count, + since one reader in five taking that reading means one agent in five acting on it, and it + blocks wherever the text lies, because the keyed questions are what prove the document. Every other single miss under questions 1 and 3 is the reader's and is tolerated by the threshold - below; a second reader making the same miss is a failed threshold, not a judgement. + below; a second miss on the same item fails the threshold, and no judgement is needed. - **An unanswered question.** Each keyed item left unanswered is a miss. Question 4 left unanswered counts as no passage named. - **Question 4.** A reader names a passage when it quotes words of the document or gives a clause @@ -134,12 +141,15 @@ recorded is not a rule. name lie in the same paragraph; a heading, a list item, a table row and a terminology entry are each a paragraph of their own. A passage named by two or more readers of the run is a finding. - **Where a finding lies.** The record says, for each finding, whether it lies in text the change - wrote or altered: a sentence or paragraph the pull request adds, rewords or moves, read against - the branch it targets. A finding there blocks the change. A finding anywhere else is a finding - against the document, not against the change: it is recorded in - [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not - block. The change is judged on what it changed; the document is proved by the keyed questions, - which every run answers over the whole of it. + wrote or altered: a paragraph in which the pull request, read against the branch it targets, + adds, rewords, moves or deletes a sentence, or changes a word, a term or a link. A finding + there blocks the change. A finding anywhere else is a finding against the document, not + against the change: it is recorded in [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a + change of its own, and does not block. The change is judged on what it changed; the document + is proved by the keyed questions, which every run answers over the whole of it. +- **Open points.** An open point in marking an answer is marked for the reader. An open point on + whether a reader named a passage, or whether two readers named the same one, counts the + passage. An open point on where a finding lies treats it as the change's own text. ## Pass criteria @@ -166,26 +176,37 @@ fresh cohort names by fewer than two is clear. ## Changing the test -Nothing in this test is changed to let a run through. A threshold, a question, a key or a -fixture is changed only before a run's readers start, never changes a mark already given, and -is recorded with its reason: a threshold, a question or the rules here by a change to this -process, a key by step 4 of `CONTRIBUTING.md` when the text it quotes is reworded or when a -load-bearing obligation is added, and each in [acceptance/CHANGES.md](acceptance/CHANGES.md). A -question is reworded only so that it asks for what its key requires, and a key's required +Nothing in this test is changed to let a particular run through. A threshold, a question, a +key, a fixture or a rule here is changed only before a run's readers start, never changes a mark +already given, and is changed in one of three ways, each recorded with its reason: + +- A key's quotation follows the text it quotes, and a key gains an item for a new load-bearing + obligation, with the question that asks for it, under step 4 of `CONTRIBUTING.md`, in the + specification pull request that makes the change; `acceptance/README.md` says what that may + and may not do. +- A threshold, a fixture, a rule here, what an existing key item requires, or the wording of a + question changes by a change to this process, with its own pull request. +- An open point a run recorded is settled by either route, whichever it needs. + +A question is reworded only so that it asks for what its key requires, and a key's required elements are never more than its question asks for. A run made after such a change is a fresh -run under the new rules, not a re-mark of an old one. The one time a threshold was relaxed to -let a change through, after three runs, the relaxation was reversed on the editor's instruction -and is recorded here so that it is not tried again. Removing the clarity rating and the request -for suggestions was not that: it changed what the cohort is asked, not how strictly an answer is -marked. +run under the new rules, not a re-mark of an old one. The test for a change to a rule is what +the rule measured: a rule that measured something other than whether a cold reader could follow +the text is a defect in this file and is corrected, and the correction says what the rule +measured instead; a rule that measured that, and found the text wanting, is not touched, and the +text is fixed. The one time a threshold was relaxed to let a change through, after three runs, +the relaxation was reversed on the editor's instruction and is recorded here so that it is not +tried again. Removing the clarity rating and the request for suggestions was not that: it +changed what the cohort is asked, not how strictly an answer is marked. ## After a failure -Fix the text the findings name and run a fresh cohort; readers are never reused. A keyed miss is -fixed in the text that carries the obligation, wherever the change left it. A finding on text the -change wrote or altered is fixed there. A finding elsewhere goes to `acceptance/ROUTED.md` and -is not fixed in this change. The number of runs it took is recorded; a document that needed three -runs is telling its editor something. +Fix the text the run's misses and findings name and run a fresh cohort; readers are never reused. +A keyed miss is fixed in the text that carries the obligation, wherever it lies, since the keyed +questions prove the document and a document one reader in five misreads is not merged to, whoever +wrote the sentence. A finding on text the change wrote or altered is fixed there. A finding +elsewhere goes to `acceptance/ROUTED.md` and is not fixed in this change. The number of runs it +took is recorded; a document that needed three runs is telling its editor something. ## When it runs @@ -217,8 +238,8 @@ returned it, the exact prompt every reader received, the command the readers wer and the instruction a delegated marker was given are posted in one or more comments on the pull request, linked from the record and never edited, hidden or deleted after posting. -The changelog line for the change names the record. CI refuses a specification change whose pull -request adds no run file; it checks for at least one, and a pull request that changes two +The changelog line for the change names every run made on it, the passing run last. CI refuses a +specification change whose pull request adds no run file; it checks for at least one, and a pull request that changes two documents adds a run for each. Each published version's changelog entry carries the lines of the changes it contains, so a version's cohort record is the set of runs those lines name. A version with no such line has not been accepted and MUST NOT be described as published. The question sets @@ -232,15 +253,17 @@ reads that list before cutting a release. ## Changes to this process The process files are this file, `acceptance/README.md`, `acceptance/runs/README.md`, -`acceptance/ROUTED.md`, `CONTRIBUTING.md` and the checks that enforce them: `tools/changes.py`, -the key-quotation checks in `spec-qa.py`, the CI workflow and the pull request template. A change -to any of them, or to a question set, is a change to this process. They are not specifications, -and a change to them is not tested by a cohort: the cohort measures whether a low-strength reader -can follow a specification, and the process is followed by the editor. Such a change names the -runs or the case that showed the need for it, and goes through steps 1, 3 and 6 of -`CONTRIBUTING.md`, with no scope, key or cohort step; its review reads every process file it +`CONTRIBUTING.md` and the checks that enforce them: `tools/changes.py`, the key-quotation checks +in `spec-qa.py`, the CI workflow and the pull request template. A change to any of them is a +change to this process, and so is a change to a threshold, a fixture, the wording of a question +or what a key item requires, as [Changing the test](#changing-the-test) says. They are not +specifications, and a change to them is not tested by a cohort: the cohort measures whether a +low-strength reader can follow a specification, and the process is followed by the editor. Such +a change names the runs or the case that showed the need for it, and goes through steps 1, 3 and +6 of `CONTRIBUTING.md`, with no scope, key or cohort step; its review reads every process file it changes. It applies to every run whose readers start after it merges; a run made before then may be made under it, following all of it at a commit named in the record before the readers start. It is recorded by a line added at the top of [acceptance/CHANGES.md](acceptance/CHANGES.md), -naming its pull request, not in the changelog, which is for the three specifications. A change to -a key or a fixture follows `acceptance/README.md`. +naming its pull request, not in the changelog, which is for the three specifications. +`acceptance/ROUTED.md` and `acceptance/runs/` are records, not process files: a specification +pull request writes to them. diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md index a076ff1..cc01848 100644 --- a/acceptance/CHANGES.md +++ b/acceptance/CHANGES.md @@ -6,7 +6,7 @@ not a specification and is not versioned, and a change to it is not tested by a to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). - Pull request NNN: the test is made to prove what it exists to prove, the clarity of the - document to a cold reader, after thirty-four runs on two changes (19 to 30, 31 to 33 and 35 + document to a cold reader, after thirty-three runs on two changes (19 to 30, 31 to 33 and 35 to 52) in which keyed answers reached five of five and the runs still failed. Four defects in the process were found. Keys required elements the question never asked for (detector 2b's "need not remove or forbid", detector 2c's "at a version tracked together", and on pull @@ -33,4 +33,4 @@ to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). traceable-miss row is narrower and the confusion row is applied to the change rather than the document; both are recorded here as the reason the earlier rules could not converge, and neither changes a mark already given. Prompted by runs 19 to 30 on pull request 14 and runs - 31 to 52 on the defined-terms change. + 31 to 33 and 35 to 52 on the defined-terms change. diff --git a/acceptance/README.md b/acceptance/README.md index 49a097b..dcf39c0 100644 --- a/acceptance/README.md +++ b/acceptance/README.md @@ -48,11 +48,18 @@ Everything else in a key is explanation for the marker and never marks an answer A key quotes the document. When a clause is reworded, the key line that quotes it is updated in the same commit, and the change is one the acceptance run itself then tests. A key that no longer -matches the document is a defect in this directory, not a reason to mark a reader wrong. A key -update under step 4 of `CONTRIBUTING.md` changes a quotation to follow the text, or adds a keyed -item for a new load-bearing obligation; it does not change what an existing item requires. A -change to what an item requires, or to a question, is a change to the process and is recorded in -[CHANGES.md](CHANGES.md) with its reason, before the readers of the next run start. +matches the document is a defect in this directory, not a reason to mark a reader wrong. + +A key, a question set or a fixture changes in one of two ways, as `ACCEPTANCE.md` says under +"Changing the test", and always before the readers of the next run start: + +- **In a specification pull request, under step 4 of `CONTRIBUTING.md`**: a quotation follows + the text it quotes, or a keyed item is added for a new load-bearing obligation together with + the question that asks for it. The pull request's own changelog line records it. Such an + update does not change what an existing item requires, nor an existing question's wording. +- **In a process change, with its own pull request**: what an existing item requires, the + wording of a question, a fixture, or the rules of marking. It is recorded in + [CHANGES.md](CHANGES.md) with its reason. Questions are fixed so that runs can be compared across versions. Adding a keyed question is allowed when a new clause is load-bearing; removing or softening one to make a run pass is not; diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index bb82718..eb35815 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -8,7 +8,9 @@ editor reads this list before cutting a release. A finding is open until a change that reworks its passage passes a cohort, when the line names that run and moves below the rule, or until the editor closes it with a reason recorded here. A -finding raised again by a later run is noted on its line, not listed twice. +finding raised again by a later run is noted on its line, not listed twice. The rows below were +raised by runs 31 to 52 under the earlier rules, on untouched text of the change those runs +tested, and are listed here so that they are not lost. ## Open @@ -24,8 +26,7 @@ finding raised again by a later run is noted on its line, not listed twice. | `DETECTOR-SPEC.md` | 8.1 | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | | `DETECTOR-SPEC.md` | 5.3 | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | | `SPEC.md` | 1 (Owner) | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | open | -| `SPEC.md` | 4 | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 (one reader, 29) | open; a method question for the editor | -| `SPEC.md` | 2 and 3.1 | "technique" used for a way of writing the Rule in section 2 and for a way of searching in clause 3.1 | runs 29, 38 | pull request 14 rewords both | +| `SPEC.md` | 4 | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | open; a method question for the editor | ## Closed diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index b0a3f97..2f6fd96 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -33,13 +33,12 @@ Wrong readings the text invites, which count against the document on one reader: ## 2. Load-bearing statements - **2a.** Required: (1) a detector conforms when it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression); - (2) yes, a detector that predates the document can conform, because conformance is judged on - the evidence and a declaration is not a condition of it. Wrong if it says a declaration is + (2) yes, a detector that predates the document can conform. Wrong if it says a declaration is required, or that partial satisfaction is conformance. Key quotation, section [8](../../DETECTOR-SPEC.md#8-conformance): "A Detector Conforms if it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression)." Supporting, [8.1](../../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps): "A maintainer - MAY declare the version of this document the detector conforms to". "Judged on evidence", "the - declaration is optional" or "graded on evidence by anyone" each state the reason in element - (2); one of them is enough. + MAY declare the version of this document the detector conforms to". The reason, that + conformance is judged on evidence and a declaration is optional, is explanation: an answer + that gives it, or gives it instead of a plain yes, states element (2). - **2b.** Required: (1) it must let the project switch the route off in its configuration, or make the route detectable (by a rule the project can write, or by a mechanical check the detector documents; naming either means of detection, or just "detectable", states this); @@ -61,7 +60,7 @@ Wrong readings the text invites, which count against the document on one reader: ## 3. Application: Lintomatic 4.2 -| Clause | Accepted grade | Condition on the reasoning, and what the fixture supports | +| Clause | Accepted grade | What the fixture supports (no row carries a condition on the reasoning) | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it) | Yes | Project rules via the `Rule` interface, first class in every report | | [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, with a distinguishing exit code | @@ -87,9 +86,9 @@ reason that it is a SHOULD, or that the MUST which fails is [6.3](../../DETECTOR Wrong readings the text invites, which count against the document on one reader: -- [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) graded Partial on the grounds that thirty-eight of forty resolve, which has read the - clause as admitting a proportion. -- [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) graded No because the comment takes no reason, where the reasoning quotes [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) or [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason), +- [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) graded Partial with reasoning that says most, or thirty-eight of forty, bundled rules + resolve, which has read the clause as admitting a proportion. +- [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) graded No with reasoning that says the comment takes no reason and quotes [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) or [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason), which has read [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason)'s SHOULD into [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable). A reader who grades [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) Yes because they missed the last paragraph of the fixture has misread the diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index 182d6b2..36bfafd 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -57,33 +57,34 @@ Wrong readings the text invites, which count against the document on one reader: ## 3. Application: the ledger remediation -| Clause | Accepted grades | Condition on the reasoning, and what the fixture supports | -| ----------------------------------------------------------------------------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | No condition. Class, hazard and the next wider rule are named; Partial or No is the reading that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | -| [3.2](../../SPEC.md#32-build-the-net) | Yes | A rule in the project's static analyser, reading code, not a test | -| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | Commit B is red on the fixture and the originating instance and survives as its own commit | -| [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | The reasoning names the instance left unfixed behind an inline ignore on the agent's own authority; five findings were recorded | -| [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | The reasoning names the suppression no recorded decision covers; the rule is otherwise permanent and blocking. [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | -| [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | The reasoning names the documentation on a wiki, not versioned with the rule; the message is terse and carries an identifier | +| Clause | Accepted grades | Condition on the reasoning | What the fixture supports | +| ----------------------------------------------------------------------------------------- | ------------------ | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | None | Class, hazard and the next wider rule are named; Partial or No is the reading that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | +| [3.2](../../SPEC.md#32-build-the-net) | Yes | None | A rule in the project's static analyser, reading code, not a test | +| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | None | Commit B is red on the fixture and the originating instance and survives as its own commit | +| [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | Mentions the instance left unfixed, the inline ignore, or `Report::render()` | Five findings were recorded, but one instance was left unfixed behind an inline ignore on the agent's own authority | +| [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | Mentions the suppression, the inline ignore, or that no recorded decision covers it | The rule is permanent and blocking, but the suppression was not a recorded decision. [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | +| [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | Mentions the wiki, or that the documentation is not versioned or co-located with the rule | The message is terse and carries an identifier, but the documentation lives on a wiki, not versioned with the rule | Section [7](../../SPEC.md#7-conformance): the **remediation does not conform**; a reader is correct who says so and names [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed. The **defence does not conform** either; a reader is correct who says so and names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the reasons, or [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which -section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent. +section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent. Other clauses named +alongside are not marked. Section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): the **inline ignore on `Report::render()`** was the owner's decision, not the -practitioner's. A reader who names it is correct, with or without the decision to leave the -wider rule unbuilt, which section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) also gives to the owner. A reader who names only the wider -rule, or neither, is wrong. +practitioner's. A reader who names it is correct; any other decision named alongside it is not +marked. A reader who names only the decision to leave the wider rule unbuilt, or neither, is +wrong. Wrong readings the text invites, which count against the document on one reader: -- [3.5](../../SPEC.md#35-enforce-permanently-and-block) graded Yes because the rule fails the build, which has not applied the clause's opening - sentence on unrecorded suppression. -- [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) graded Yes because the URL resolves, which has read the versioning sentence of [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) - differently. -- The defence graded as conforming because the sweep is what failed, where the reasoning quotes - section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence. +- [3.5](../../SPEC.md#35-enforce-permanently-and-block) graded Yes with reasoning that says the rule blocks, or is permanent, and does not mention + the suppression. +- [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) graded Yes with reasoning that says the identifier resolves, or the URL works, and does not + mention versioning or co-location. +- The defence graded as conforming with reasoning that says only the sweep failed and that + quotes section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence. A reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the fixture, which is the reader's miss. A reader who attributes the suppression to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) rather than [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) has read the narrowing passage of [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) as diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index c11ae32..af28752 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -45,26 +45,26 @@ Wrong readings the text invites, which count against the document on one reader: grade, not a grade of its own. Wrong if it gives a grade per part. Key quotation, section [9](../../TOOLING-SPEC.md#9-conformance): "The project-level verdict is a single grade; which part of the Toolchain satisfied each clause is evidence for that grade, not a second grade." -- **2c.** Required: (1) two, graded and declared separately; (2) neither implies the other. - Wrong if it gives one grade, or says the artefact grade follows from the project grade or the - reverse. Key quotation, [9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately): "The two verdicts MUST be graded and declared separately, and +- **2c.** Required: (1) two; (2) neither implies the other. Wrong if it gives one grade, or says + the artefact grade follows from the project grade or the reverse. That the two are graded and + declared separately is explanation; an answer that says it has stated element (1). Key quotation, [9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately): "The two verdicts MUST be graded and declared separately, and neither implies the other." ## 3. Application: the Zigpipe project -| Clause | Accepted grades | Condition on the reasoning, and what the fixture supports | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap; a reader who notes that the detector specification was not supplied and grades from this document's summary of it is correct | -| [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)`. The two are blocking, since their firing stops `zig-qa run`; a reader who grades No on that condition is correct | -| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | -| [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | -| [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | Lintomatic, then the tests, stopping on a Lintomatic failure | -| [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | No condition. `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is the reading that counts the two `(none)` routes, since 5.1 says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | -| [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration) | Yes | Derived from `lintomatic.toml`, not hand-maintained | -| [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | Yes | The `proj.*` rules appear in the same listing | -| [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) | Yes or Partial | `quality-exceptions.toml` is a defined path that `zig-qa run` loads. Partial is accepted only where the reasoning says the fixture names a place for exceptions and none for calibrations or conventions, which the project record entry also covers | -| [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | The reasoning mentions that nothing rejects a justification that could be pasted onto any entry; a justification is required and an empty one rejected | -| [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | `zig-qa list --exceptions` uses the same format and operation as the rule listing | +| Clause | Accepted grades | Condition on the reasoning | What the fixture supports | +| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | None | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap; a reader who notes that the detector specification was not supplied and grades from this document's summary of it is correct | +| [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | None. A No given on the condition that the two bundled rules are blocking is a No: their firing stops `zig-qa run`, so they are | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)` | +| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | None | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | +| [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | None | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | +| [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | None | Lintomatic, then the tests, stopping on a Lintomatic failure | +| [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | None | `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is the reading that counts the two `(none)` routes, since [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | +| [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration) | Yes | None | Derived from `lintomatic.toml`, not hand-maintained | +| [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | Yes | None | The `proj.*` rules appear in the same listing | +| [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) | Yes or Partial | For Partial: mentions that the fixture names a place for exceptions and none for calibrations or conventions | `quality-exceptions.toml` is a defined path that `zig-qa run` loads; the project record entry also covers calibrations and conventions | +| [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | Mentions that nothing rejects a generic justification, one that could be pasted onto any entry | A justification is required and an empty one rejected | +| [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | None | `zig-qa list --exceptions` uses the same format and operation as the rule listing | Section [9](../../TOOLING-SPEC.md#9-conformance) verdict: **does not conform**; a reader is correct who says so and names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses. **Not with agent support**; a reader is correct who says so because @@ -76,14 +76,15 @@ optional." Wrong readings the text invites, which count against the document on one reader: -- [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) graded Yes with reasoning that quotes only the first sentence of the clause, which has - not read the sentence on a generic justification. -- [8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) applied to Zigpipe on the basis that its tooling includes a shipped package, which has - read "where the Toolchain is one the project ships" (section [9](../../TOOLING-SPEC.md#9-conformance)) as reaching the consuming - project. -- [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) graded Yes on the grounds that Lintomatic "mostly" conforms, where the reasoning quotes - [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), which has read it as admitting a proportion. -- Zigpipe held ungradeable for want of a declaration, which has read [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) as a condition. +- [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) graded Yes with reasoning that says a justification is required or an empty one rejected + and does not mention a generic justification, which has not read the sentence on one. +- Section [8](../../TOOLING-SPEC.md#8-self-audit) said to apply to Zigpipe with reasoning that says its tooling includes a shipped + package, which has read "where the Toolchain is one the project ships" (section [9](../../TOOLING-SPEC.md#9-conformance)) as + reaching the consuming project. +- [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) graded Yes with reasoning that says Lintomatic mostly or largely conforms and quotes [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), + which has read it as admitting a proportion. +- The declaration said to affect the verdict, with reasoning that Zigpipe cannot be graded without + one, which has read [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) as a condition. A reader who grades [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) Yes because they missed the two undocumented rules has misread the fixture, which is the reader's miss. diff --git a/acceptance/toolchain/QUESTIONS.md b/acceptance/toolchain/QUESTIONS.md index 35a2b91..86de411 100644 --- a/acceptance/toolchain/QUESTIONS.md +++ b/acceptance/toolchain/QUESTIONS.md @@ -26,7 +26,7 @@ the document you are relying on. Grade the Zigpipe project's toolchain, as described in the fixture, against every MUST clause in sections [4](../../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) to [6](../../TOOLING-SPEC.md#6-the-project-record): [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record), [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes), [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure), [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them), [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration), [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones), [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences). For each give Yes, Partial or No and one line of reasoning. Then state whether the toolchain conforms under section -9, whether it conforms with agent support, and whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies to it. Finally, state +[9](../../TOOLING-SPEC.md#9-conformance), whether it conforms with agent support, and whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies to it. Finally, state whether the absence of a declaration affects the verdict. ## 4. Confusion From 8a63448820785166faeef81263fc1143b3028710 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 11:52:20 +0100 Subject: [PATCH 03/34] Acceptance process: review 2 findings applied Question 1 defined once (first six bullets, same clause counts once, a bullet on neither list not counted); the verdict items of question 3 ask for their reason and the keys state each as an accepted answer with a condition; a finding lies in the change's text also where the readers set it against that text, read against the merge base; the cohort's model is the version the editor names; a keyed miss whose fix would change an obligation is routed and waited for; the question set is the one list of load-bearing questions; ROUTED.md defines its status column and links its clauses. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 102 +++++++++++++++--------------- CONTRIBUTING.md | 6 +- acceptance/CHANGES.md | 11 ++-- acceptance/README.md | 9 +-- acceptance/ROUTED.md | 33 +++++----- acceptance/detector/KEY.md | 11 ++-- acceptance/detector/QUESTIONS.md | 2 +- acceptance/method/KEY.md | 30 ++++----- acceptance/method/QUESTIONS.md | 4 +- acceptance/toolchain/KEY.md | 22 ++++--- acceptance/toolchain/QUESTIONS.md | 4 +- 11 files changed, 123 insertions(+), 111 deletions(-) diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index c5bd8cd..39110a7 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -29,7 +29,8 @@ serves that, and a rule that stops it proving that is a defect in this file, fix ``` A reader is set aside, and its answers do not count, when its answer text is empty, when its - process reports an error, or when the model version it reports is not the cohort's model. A run + process reports an error, or when the model version it reports is not the cohort's model, the + one version the editor names for the run before its readers start. A run with a reader set aside is void: it takes a run number and has a record, and a fresh cohort runs. No reader is set aside for any other cause. @@ -54,24 +55,17 @@ asks for nothing the question does not. Each reader answers all of them. bullets. Keyed against the key's list of the document's obligations: a bullet is correct when it names one of them. -2. **Load-bearing statements.** Three or more questions, fixed per document, each asking for a - short answer and the sentence or sentences of the document it rests on, quoted. Each has one - or more parts, and the key lists the element an answer must state for each part: - - - `SPEC.md`: what a practitioner does before fixing a defect, in order; who may baseline or - suppress, and whether an agent may; what "conforms" means for a remediation and what a - verdict rests on. - - `DETECTOR-SPEC.md`: what "conforms" means and whether a tool that predates the document can - conform; what a detector must do about an inline suppression route and whether one of those - things is enough; where a bundled rule's documentation must live, at what version, and - whether it must be reachable offline. - - `TOOLING-SPEC.md`: where conformance is measured and what the toolchain is; how a grade is - assigned when parts come from different sources; how many grades a project that ships - tooling has and whether one implies the other. +2. **Load-bearing statements.** Three or more questions, fixed per document and listed in its + question set under `acceptance/`, each asking for a short answer and the sentence or + sentences of the document it rests on, quoted. Each has one or more parts, and the key lists + the element an answer must state for each part. An obligation is load-bearing, and gets a + question, when a reader who misreads it would act against the method; the editor decides, and + the pull request that adds the question says why. 3. **Application.** A fixture case supplied with the questions, graded clause by clause with - Yes, Partial or No and one line of reasoning. Keyed against the intended verdicts, with the - readings the key names as ones the text invites counting against the document. + Yes, Partial or No and one line of reasoning, then the verdicts the question set asks for, + each with its reason. Keyed against the accepted grades and answers, with the readings the + key names as ones the text invites counting against the document. 4. **Confusion.** Any passage the reader found confusing on its one reading, quoted with its clause number, and nothing else: not what would make it clearer, not a proposed change. @@ -98,9 +92,10 @@ unless the editor shows that a rule here was misapplied, and a re-mark says whic Every mark is given by a rule in this file or in the key, as follows. Where neither settles a mark, the marker records the point in the run record and marks it as the open-point rule below -says, and the editor settles it in this file or in the key, as [Changing the test](#changing-the-test) -allows, before the next run. A run record states no judgement that is not written here or in -the key; a judgement an earlier run recorded is not a rule. +says, and the editor settles it in this file or in the key, as +[Changing the test](#changing-the-test) allows, before the next run. A run record states no +judgement that is not written here or in the key; a judgement an earlier run recorded is not a +rule. - **Keyed items.** Question 1 as a whole is one keyed item, and so is each item of question 2 and each answer of question 3 for which the key gives an accepted grade or answer. @@ -111,20 +106,21 @@ the key; a judgement an earlier run recorded is not a rule. or apostrophe style, and markdown link or emphasis markup; a quotation that drops or changes a word quotes the document by its longest such run. - **Question 1** is correct when at least four bullets each name an obligation the key lists and - no bullet asserts one the key marks wrong. Several bullets that name different clauses of one - line of the key's list each count. A bullet asserting an obligation that is in neither list is - recorded as an open point for the key and is not counted either way. + no bullet asserts one the key marks wrong. Only the first six bullets are marked. Several + bullets that name different clauses of one line of the key's list each count; two that name + the same clause count once. A bullet asserting an obligation that is in neither list is not + counted either way, and is recorded as an open point for the key. - **Question 2.** An answer is the reader's sentences and its quotation read together. It is correct when it states every element the key lists as required for that item, in the document's words or the reader's own, asserts nothing the key marks wrong, and quotes the document. An element stated only in the quotation is stated. Anything in the key beyond the required elements and the wrong assertions is explanation for the marker, and an answer is never marked down for leaving it out. -- **Question 3.** Each item is correct when its grade is one the key accepts and, where the key - sets a condition on the reasoning for that grade, the reasoning meets it. A grade the key - accepts with no condition is correct whatever the reasoning. A grade given on a stated - condition is read as the grade for the case the fixture presents, where the key says which - case that is; otherwise the item is unanswered. +- **Question 3.** Each item is correct when its grade or answer is one the key accepts and, + where the key sets a condition on the reasoning for it, the reasoning meets it. A grade or + answer the key accepts with no condition is correct whatever the reasoning. A grade given on a + stated condition is read as the grade for the case the fixture presents, where the key says + which case that is; otherwise the item is unanswered. - **A miss that counts against the document.** The key names, for question 1 and for each item of question 3, the wrong readings the text invites, each as a grade or answer together with what the reasoning states or leaves out. A miss that matches one blocks whatever the count, @@ -134,22 +130,26 @@ the key; a judgement an earlier run recorded is not a rule. below; a second miss on the same item fails the threshold, and no judgement is needed. - **An unanswered question.** Each keyed item left unanswered is a miss. Question 4 left unanswered counts as no passage named. -- **Question 4.** A reader names a passage when it quotes words of the document or gives a clause - number with a description that identifies a sentence or paragraph. A remark that is not about - a passage of the document, such as that another document was not supplied, is recorded as a - note and is not a passage. Two readers name the same passage when the words or sentences they - name lie in the same paragraph; a heading, a list item, a table row and a terminology entry are - each a paragraph of their own. A passage named by two or more readers of the run is a finding. +- **Question 4.** A reader names a passage when it quotes the document, as defined above, or + gives a clause number with a description or a shorter quotation that identifies a sentence or + paragraph. A quotation whose words occur in more than one paragraph, given with no clause + number, names the paragraph where they first occur. A remark that is not about a passage of + the document, such as that another document was not supplied, is recorded as a note and is + not a passage. Two readers name the same passage when the words or sentences they name lie in + the same paragraph; a heading, a list item, a table row and a terminology entry are each a + paragraph of their own. A passage named by two or more readers of the run is a finding. - **Where a finding lies.** The record says, for each finding, whether it lies in text the change - wrote or altered: a paragraph in which the pull request, read against the branch it targets, - adds, rewords, moves or deletes a sentence, or changes a word, a term or a link. A finding + wrote or altered: a paragraph in which the pull request, read against its merge base with the + branch it targets, adds, rewords, moves or deletes a sentence, or changes a word, a term or a + link; or a paragraph the readers naming the finding set against such a paragraph. A finding there blocks the change. A finding anywhere else is a finding against the document, not against the change: it is recorded in [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not block. The change is judged on what it changed; the document is proved by the keyed questions, which every run answers over the whole of it. -- **Open points.** An open point in marking an answer is marked for the reader. An open point on - whether a reader named a passage, or whether two readers named the same one, counts the - passage. An open point on where a finding lies treats it as the change's own text. +- **Open points.** An open point in marking an answer is marked for the reader, except that a + question 1 bullet on neither list is not counted, as above. An open point on whether a reader + named a passage, or whether two readers named the same one, counts the passage. An open point + on where a finding lies treats it as the change's own text. ## Pass criteria @@ -164,7 +164,7 @@ is four of five. | A miss the key says counts against the document | **None**, whatever the count | | A passage found confusing by two or more readers, in text the change wrote or altered | **None.** It is a finding and blocks until the text is changed | | A passage found confusing by two or more readers, elsewhere | Recorded in `acceptance/ROUTED.md` and routed; does not block | -| Anything from one reader only | Recorded; applied at the editor's discretion; does not block | +| Anything else from one reader only | Recorded; applied at the editor's discretion; does not block | The asymmetry is deliberate. A load-bearing statement is the sentence the whole document exists to make, and one reader in five getting it wrong means one agent in five will act on the wrong @@ -202,11 +202,12 @@ changed what the cohort is asked, not how strictly an answer is marked. ## After a failure Fix the text the run's misses and findings name and run a fresh cohort; readers are never reused. -A keyed miss is fixed in the text that carries the obligation, wherever it lies, since the keyed -questions prove the document and a document one reader in five misreads is not merged to, whoever -wrote the sentence. A finding on text the change wrote or altered is fixed there. A finding -elsewhere goes to `acceptance/ROUTED.md` and is not fixed in this change. The number of runs it -took is recorded; a document that needed three runs is telling its editor something. +A failed keyed item is fixed in the text that carries the obligation, wherever it lies, since the +keyed questions prove the document as a whole, whoever wrote the sentence; where that fix would +change an obligation, it is routed to a change of its own, and this change waits for it. A +finding on text the change wrote or altered is fixed there. A finding elsewhere goes to +`acceptance/ROUTED.md` and is not fixed in this change. The number of runs it took is recorded; +a document that needed three runs is telling its editor something. ## When it runs @@ -239,11 +240,12 @@ and the instruction a delegated marker was given are posted in one or more comme request, linked from the record and never edited, hidden or deleted after posting. The changelog line for the change names every run made on it, the passing run last. CI refuses a -specification change whose pull request adds no run file; it checks for at least one, and a pull request that changes two -documents adds a run for each. Each published version's changelog entry carries the lines of the -changes it contains, so a version's cohort record is the set of runs those lines name. A version -with no such line has not been accepted and MUST NOT be described as published. The question sets -and answer keys live under `acceptance/` in this repository so that a run can be repeated and +specification change whose pull request adds no run file; it checks for at least one, and a pull +request that changes two documents adds a run for each. Each published version's changelog entry +carries the lines of the changes it contains, so a version's cohort record is the set of runs +those lines name. A version with no such line has not been accepted and MUST NOT be described as +published. The question sets and answer keys live under `acceptance/` in this repository so that +a run can be repeated and compared. `acceptance/ROUTED.md` lists every finding routed from a run, open until a change that fixes diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 013bb9d..733e05b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -29,9 +29,9 @@ review. It is a patch version when released. SHOULD or MAY in a numbered section or appendix. It needs everything a clarity edit needs, plus the five tests in SCOPE.md answered in the pull request, and it is at least a minor version when released. `tools/changes.py` tells the two apart mechanically and CI runs it on every pull -request. A change to the process files ACCEPTANCE.md names, this file among them, is neither: -ACCEPTANCE.md says how it is made, and it is recorded in `acceptance/CHANGES.md`, not in -CHANGELOG.md. +request. A change to the acceptance process, as ACCEPTANCE.md defines one under "Changes to +this process", is neither: ACCEPTANCE.md says how it is made, and it is recorded in +`acceptance/CHANGES.md`, not in CHANGELOG.md. ## What a pull request contains diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md index cc01848..29a20db 100644 --- a/acceptance/CHANGES.md +++ b/acceptance/CHANGES.md @@ -1,13 +1,14 @@ # Changes to the acceptance process -Changes to the process files that [ACCEPTANCE.md](../ACCEPTANCE.md) names, recorded here as it -describes: a line is added at the top for each change, naming its pull request. The process is -not a specification and is not versioned, and a change to it is not tested by a cohort. Changes -to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). +Changes to the acceptance process, as [ACCEPTANCE.md](../ACCEPTANCE.md) defines one under +"Changes to this process", recorded here as it describes: a line is added at the top for each +change, naming its pull request. The process is not a specification and is not versioned, and a +change to it is not tested by a cohort. Changes to the three specifications are recorded in +[CHANGELOG.md](../CHANGELOG.md). - Pull request NNN: the test is made to prove what it exists to prove, the clarity of the document to a cold reader, after thirty-three runs on two changes (19 to 30, 31 to 33 and 35 - to 52) in which keyed answers reached five of five and the runs still failed. Four defects in + to 52) in which keyed answers reached five of five and the runs still failed. Five defects in the process were found. Keys required elements the question never asked for (detector 2b's "need not remove or forbid", detector 2c's "at a version tracked together", and on pull request 14 method 2d's order of the kinds), so readers who answered the question as put were diff --git a/acceptance/README.md b/acceptance/README.md index dcf39c0..aaeba1f 100644 --- a/acceptance/README.md +++ b/acceptance/README.md @@ -25,8 +25,8 @@ in [CHANGES.md](CHANGES.md). 4. The result is recorded as a file under [runs/](runs/), stating what `ACCEPTANCE.md` lists, with the answer sheets posted on the pull request and linked from it, and the change's `Unreleased` changelog line names that file. A run that fails is followed by a change to the - text the findings name and a fresh cohort; a finding on text the change did not write or - alter is added to [ROUTED.md](ROUTED.md) instead. + text its misses and findings name and a fresh cohort; a finding on text the change did not + write or alter is added to [ROUTED.md](ROUTED.md) instead. ## How a key is written @@ -50,8 +50,9 @@ A key quotes the document. When a clause is reworded, the key line that quotes i the same commit, and the change is one the acceptance run itself then tests. A key that no longer matches the document is a defect in this directory, not a reason to mark a reader wrong. -A key, a question set or a fixture changes in one of two ways, as `ACCEPTANCE.md` says under -"Changing the test", and always before the readers of the next run start: +A key, a question set or a fixture changes by one of two routes, as `ACCEPTANCE.md` says under +"Changing the test", always before the readers of the next run start, and an open point a run +recorded is settled by whichever route it needs: - **In a specification pull request, under step 4 of `CONTRIBUTING.md`**: a quotation follows the text it quotes, or a keyed item is added for a new load-bearing obligation together with diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index eb35815..60fc616 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -8,25 +8,26 @@ editor reads this list before cutting a release. A finding is open until a change that reworks its passage passes a cohort, when the line names that run and moves below the rule, or until the editor closes it with a reason recorded here. A -finding raised again by a later run is noted on its line, not listed twice. The rows below were -raised by runs 31 to 52 under the earlier rules, on untouched text of the change those runs -tested, and are listed here so that they are not lost. +finding raised again by a later run is noted on its line, not listed twice. The status column +holds `open`, or the `editor/` branch or pull request number of the change that is +reworking the passage. The rows below were raised by runs 31 to 52 under the earlier rules, on +untouched text of the change those runs tested, and are listed here so that they are not lost. ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | -------------------------------------- | -| `TOOLING-SPEC.md` | 9 and 9.1 | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 2, 4.2, 8.1 | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 9.2 | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 4.3 | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | run 33 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 4.1 | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 5.3 | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | open | -| `DETECTOR-SPEC.md` | 7.1 | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | 8.1 | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | 5.3 | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | -| `SPEC.md` | 1 (Owner) | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | open | -| `SPEC.md` | 4 | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | open; a method question for the editor | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | ---------------------------------------- | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | run 33 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | ## Closed diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index 2f6fd96..aaddb69 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -8,7 +8,7 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting an obligation the document does not state, is a correct restatement. +bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. - Support bespoke rules written by the project ([4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it)). - Provide a harness that runs a single rule against supplied code ([4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code)). @@ -78,11 +78,12 @@ No grade above carries a condition on the reasoning: the accepted grade is corre reasoning, and a reader who adds a caveat about what the fixture does not say and still gives the accepted grade is correct. -Section [8](../../DETECTOR-SPEC.md#8-conformance) verdict: **does not conform**; a reader is correct who says so and names [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), the -two undocumented bundled rules, or partial conformance not counting, as the reason. +The two answers the question then asks for, each with the condition its reason must meet: -Clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation): **does not change the verdict**; a reader is correct who says so and gives as the -reason that it is a SHOULD, or that the MUST which fails is [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together). +- **Conformance under section [8](../../DETECTOR-SPEC.md#8-conformance).** Accepted answer: does not conform. Condition: the reason + names [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), the two undocumented bundled rules, or partial conformance not counting. +- **Whether clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation) changes the verdict.** Accepted answer: it does not. Condition: the reason + says that it is a SHOULD, or that the MUST which fails is [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together). Wrong readings the text invites, which count against the document on one reader: diff --git a/acceptance/detector/QUESTIONS.md b/acceptance/detector/QUESTIONS.md index 2666b2f..5c288f8 100644 --- a/acceptance/detector/QUESTIONS.md +++ b/acceptance/detector/QUESTIONS.md @@ -25,7 +25,7 @@ the document you are relying on. Grade Lintomatic 4.2, as described in the fixture, against every MUST clause in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression): [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it), [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), [4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding), [5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure), [5.2](../../DETECTOR-SPEC.md#52-the-detector-must-support-invocation-over-a-subset-at-minimum-a-single-file), [5.3](../../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran), [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally), [6.1](../../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation), [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access), [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable). For each give Yes, Partial or No and one line of reasoning. Then state whether Lintomatic conforms under section [8](../../DETECTOR-SPEC.md#8-conformance), and why. Finally, -state whether clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation) changes the verdict. +state whether clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation) changes the verdict, and why. ## 4. Confusion diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index 36bfafd..b756f47 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -8,7 +8,7 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting an obligation the document does not state, is a correct restatement. +bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. - Attribute the defect to a class, bounded on both sides, with the hazard stated ([3.1](../../SPEC.md#31-attribute-the-defect-to-a-class)). - Express the class as a rule in a detector that reads code, never a test ([3.2](../../SPEC.md#32-build-the-net)). @@ -43,7 +43,7 @@ Wrong readings the text invites, which count against the document on one reader: quotation, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): "An Agent MUST NOT Baseline, suppress or knowingly leave an Instance unfixed on its own authority, whatever the Instance count turns out to be." "The owner" names the defined term and states that the decision is a human's; an answer need not add "a human" - to be correct. The default owner paragraph in the terminology entry for Owner still requires a + to be correct. The default-owner sentences of the terminology entry for Owner still require a recorded decision, so an answer naming whoever instructed the practitioner, for a project that has recorded nobody, names the owner and is correct. - **2c.** Required: (1) all six clauses of section [3](../../SPEC.md#3-the-method) were followed for that defect; (2) the verdict @@ -61,21 +61,22 @@ Wrong readings the text invites, which count against the document on one reader: | ----------------------------------------------------------------------------------------- | ------------------ | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | None | Class, hazard and the next wider rule are named; Partial or No is the reading that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | | [3.2](../../SPEC.md#32-build-the-net) | Yes | None | A rule in the project's static analyser, reading code, not a test | -| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | None | Commit B is red on the fixture and the originating instance and survives as its own commit | +| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes, Partial or No | For Partial or No: attributes the inline ignore to this clause, a fair reading of Part B | Commit B is red on the fixture and the originating instance and survives as its own commit | | [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | Mentions the instance left unfixed, the inline ignore, or `Report::render()` | Five findings were recorded, but one instance was left unfixed behind an inline ignore on the agent's own authority | | [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | Mentions the suppression, the inline ignore, or that no recorded decision covers it | The rule is permanent and blocking, but the suppression was not a recorded decision. [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | | [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | Mentions the wiki, or that the documentation is not versioned or co-located with the rule | The message is terse and carries an identifier, but the documentation lives on a wiki, not versioned with the rule | -Section [7](../../SPEC.md#7-conformance): the **remediation does not conform**; a reader is correct who says so and names -[3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed. The **defence does not conform** either; a reader is -correct who says so and names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the reasons, or [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which -section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent. Other clauses named -alongside are not marked. +The three answers the question then asks for, each with the condition its reason must meet: -Section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): the **inline ignore on `Report::render()`** was the owner's decision, not the -practitioner's. A reader who names it is correct; any other decision named alongside it is not -marked. A reader who names only the decision to leave the wider rule unbuilt, or neither, is -wrong. +- **The remediation under section [7](../../SPEC.md#7-conformance).** Accepted answer: does not conform. Condition: the reason + names [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed; other clauses named alongside are not marked. +- **The defence alone.** Accepted answer: does not conform. Condition: the reason names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation), or + [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also + requires to be absent; other clauses named alongside are not marked. +- **The decision section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) says was not the practitioner's.** Accepted answer: the inline ignore on + `Report::render()`, which was the owner's decision. Condition: none; any other decision named + alongside it is not marked. An answer naming only the decision to leave the wider rule unbuilt, + or naming none, is wrong. Wrong readings the text invites, which count against the document on one reader: @@ -86,9 +87,8 @@ Wrong readings the text invites, which count against the document on one reader: - The defence graded as conforming with reasoning that says only the sweep failed and that quotes section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence. -A reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the fixture, which is the reader's miss. A reader who -attributes the suppression to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) rather than [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) has read the narrowing passage of [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) as -covering it, which is a fair reading and is not marked wrong. +A reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the fixture, which is the reader's miss, whether or not +the inline ignore was attributed to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) instead. ## 4. Confusion diff --git a/acceptance/method/QUESTIONS.md b/acceptance/method/QUESTIONS.md index 34fa82d..9a2f393 100644 --- a/acceptance/method/QUESTIONS.md +++ b/acceptance/method/QUESTIONS.md @@ -24,8 +24,8 @@ the document you are relying on. Grade the ledger remediation, as described in the fixture, against each of the six clauses of section [3](../../SPEC.md#3-the-method): [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class), [3.2](../../SPEC.md#32-build-the-net), [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire), [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), [3.5](../../SPEC.md#35-enforce-permanently-and-block), [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation). For each give Yes, Partial or No and one line of -reasoning. Then state whether the remediation conforms under section [7](../../SPEC.md#7-conformance), whether the defence -alone conforms, and which decision in the fixture, if any, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) says was not the +reasoning. Then state whether the remediation conforms under section [7](../../SPEC.md#7-conformance) and why, whether the defence +alone conforms and why, and which decision in the fixture, if any, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) says was not the practitioner's to take. ## 4. Confusion diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index af28752..7353852 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -8,7 +8,7 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting an obligation the document does not state, is a correct restatement. +bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. - Route defences only through detectors that conform to the detector specification, wrapping permitted ([4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification)). @@ -66,13 +66,19 @@ Wrong readings the text invites, which count against the document on one reader: | [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | Mentions that nothing rejects a generic justification, one that could be pasted onto any entry | A justification is required and an empty one rejected | | [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | None | `zig-qa list --exceptions` uses the same format and operation as the rule listing | -Section [9](../../TOOLING-SPEC.md#9-conformance) verdict: **does not conform**; a reader is correct who says so and names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or -[4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses. **Not with agent support**; a reader is correct who says so because -section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, there being no summary. **Section [8](../../TOOLING-SPEC.md#8-self-audit) does not apply**; a reader is correct -who says so because Zigpipe ships nothing to other projects, or because section [8](../../TOOLING-SPEC.md#8-self-audit) bears on -zig-qa's own artefact grade and not on Zigpipe. **The absence of a declaration does not affect -the verdict**; a reader is correct who says so. Key quotation, [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance): "The declaration is -optional." +The four answers the question then asks for, each with the condition its reason must meet: + +- **Conformance under section [9](../../TOOLING-SPEC.md#9-conformance).** Accepted answer: does not conform. Condition: the reason + names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses; other clauses named alongside are not marked. +- **Conformance with agent support.** Accepted answer: no. Condition: the reason says that + section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, or that there is no summary. +- **Whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies.** Accepted answer: it does not. Condition: the reason says that + Zigpipe ships nothing to other projects, or that section [8](../../TOOLING-SPEC.md#8-self-audit) bears on zig-qa's own artefact + grade and not on Zigpipe. +- **Whether the absence of a declaration affects the verdict.** Accepted answer: it does not. + Condition: the reason says that the declaration is optional, or not a condition of + conformance, or that a verdict rests on the evidence. Key quotation, [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance): "The declaration is + optional." Wrong readings the text invites, which count against the document on one reader: diff --git a/acceptance/toolchain/QUESTIONS.md b/acceptance/toolchain/QUESTIONS.md index 86de411..d4e5cd7 100644 --- a/acceptance/toolchain/QUESTIONS.md +++ b/acceptance/toolchain/QUESTIONS.md @@ -26,8 +26,8 @@ the document you are relying on. Grade the Zigpipe project's toolchain, as described in the fixture, against every MUST clause in sections [4](../../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) to [6](../../TOOLING-SPEC.md#6-the-project-record): [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record), [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes), [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure), [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them), [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration), [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones), [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences). For each give Yes, Partial or No and one line of reasoning. Then state whether the toolchain conforms under section -[9](../../TOOLING-SPEC.md#9-conformance), whether it conforms with agent support, and whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies to it. Finally, state -whether the absence of a declaration affects the verdict. +[9](../../TOOLING-SPEC.md#9-conformance) and why, whether it conforms with agent support and why, and whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies to it +and why. Finally, state whether the absence of a declaration affects the verdict, and why. ## 4. Confusion From 4cc568884c13ed12c7b31a56f41459e26519de30 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 11:58:31 +0100 Subject: [PATCH 04/34] Acceptance process: review 3 findings applied Two routes for changing the test, with a changed or removed obligation's key item following the text in the specification pull request; an open point holds up only its own pull request; a fresh run follows a change to the document, not to the record; detector 2b names both things; the section 7 and section 9 reasons accept every failing clause; the restatement rule reads plainly; ROUTED.md defines a finding as ACCEPTANCE.md does. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 30 ++++++++++++++++-------------- CONTRIBUTING.md | 4 ++-- acceptance/README.md | 20 +++++++++++++------- acceptance/ROUTED.md | 6 +++--- acceptance/detector/KEY.md | 13 +++++++------ acceptance/method/KEY.md | 7 ++++--- acceptance/toolchain/KEY.md | 10 ++++++---- 7 files changed, 51 insertions(+), 39 deletions(-) diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index 39110a7..d1bd6af 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -141,8 +141,8 @@ rule. - **Where a finding lies.** The record says, for each finding, whether it lies in text the change wrote or altered: a paragraph in which the pull request, read against its merge base with the branch it targets, adds, rewords, moves or deletes a sentence, or changes a word, a term or a - link; or a paragraph the readers naming the finding set against such a paragraph. A finding - there blocks the change. A finding anywhere else is a finding against the document, not + link; or a paragraph that any reader naming the finding sets against such a paragraph. A + finding there blocks the change. A finding anywhere else is a finding against the document, not against the change: it is recorded in [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not block. The change is judged on what it changed; the document is proved by the keyed questions, which every run answers over the whole of it. @@ -178,15 +178,18 @@ fresh cohort names by fewer than two is clear. Nothing in this test is changed to let a particular run through. A threshold, a question, a key, a fixture or a rule here is changed only before a run's readers start, never changes a mark -already given, and is changed in one of three ways, each recorded with its reason: +already given, and is changed by one of two routes, each recorded with its reason: -- A key's quotation follows the text it quotes, and a key gains an item for a new load-bearing - obligation, with the question that asks for it, under step 4 of `CONTRIBUTING.md`, in the - specification pull request that makes the change; `acceptance/README.md` says what that may - and may not do. -- A threshold, a fixture, a rule here, what an existing key item requires, or the wording of a - question changes by a change to this process, with its own pull request. -- An open point a run recorded is settled by either route, whichever it needs. +- In the specification pull request that changes the text, under step 4 of `CONTRIBUTING.md`: + a key's quotation follows the text it quotes; a key gains an item, with the question that asks + for it, for a new load-bearing obligation; and where the change alters or removes the + obligation an existing item tests, the item and its question follow the text, since that is + the text changing and not the test. `acceptance/README.md` says what that may and may not do. +- By a change to this process, with its own pull request: a threshold, a fixture, a rule here, + what an existing key item requires of an unchanged obligation, or the wording of a question. + +An open point a run recorded is settled by whichever route it needs, before the next run of the +pull request whose run recorded it; a run of another pull request is not held up by it. A question is reworded only so that it asks for what its key requires, and a key's required elements are never more than its question asks for. A run made after such a change is a fresh @@ -214,9 +217,9 @@ a document that needed three runs is telling its editor something. On the final text of each pull request, after the editor's rewrite and any key update, one cohort per changed document. It is the last gate before the merge to `next`, so that every change on `next` is one the cohort has accepted, and the release branch needs no cohort of its own unless it -changes specification text. A fresh cohort runs after every change to the pull request's own +changes specification text. A fresh cohort runs after every change to the changed document's text that follows a run, and after every void run; a text that failed is never run again -unchanged. A version number or date that `PUBLISHING.md` or step 5 of `CONTRIBUTING.md` has the +unchanged, and a commit that adds the run record or the changelog line changes no document. A version number or date that `PUBLISHING.md` or step 5 of `CONTRIBUTING.md` has the editor set is not such a change, and neither is a merge of `next` into the branch that brings in text another cohort accepted and alters no sentence this change wrote. @@ -245,8 +248,7 @@ request that changes two documents adds a run for each. Each published version's carries the lines of the changes it contains, so a version's cohort record is the set of runs those lines name. A version with no such line has not been accepted and MUST NOT be described as published. The question sets and answer keys live under `acceptance/` in this repository so that -a run can be repeated and -compared. +a run can be repeated and compared. `acceptance/ROUTED.md` lists every finding routed from a run, open until a change that fixes its passage passes a cohort, or until the editor closes it with a recorded reason. The editor diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 733e05b..ea5a525 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -69,8 +69,8 @@ survives, the change is yours. changed document. Its record is committed under `acceptance/runs/`, the `Unreleased` changelog line names it, and CI refuses a specification change that lacks one. For an obligation, the editor sets the document's header to `-dev, unpublished` in the - same commit if it still names a published version. A change to the pull request's own text - after the run means a fresh run, as ACCEPTANCE.md says under "When it runs". A finding the + same commit if it still names a published version. A change to the document's text after the + run means a fresh run, as ACCEPTANCE.md says under "When it runs". A finding the run routes to `acceptance/ROUTED.md` is not fixed in this pull request. 6. **Merge** into `next` by merge commit, and the branch is deleted. diff --git a/acceptance/README.md b/acceptance/README.md index aaeba1f..ec5c3b2 100644 --- a/acceptance/README.md +++ b/acceptance/README.md @@ -32,12 +32,15 @@ in [CHANGES.md](CHANGES.md). A key says, for each keyed item, exactly what is marked and nothing a marker has to guess: -- **Question 1**: the obligations a bullet may name, and the assertions that make a bullet wrong. +- **Question 1**: the obligations a bullet may name, the assertions that make a bullet wrong, + and the wrong readings the text invites, if any. - **Question 2**: for each item, the elements an answer is required to state, numbered, one for each part of the question; the assertions that make an answer wrong; and the sentence or - sentences of the document the answer rests on, quoted verbatim. The required elements are - what the question asks for, no more: an element the question does not ask for is not - required, however load-bearing, until the question is reworded to ask for it. + sentences of the document the answer rests on, quoted verbatim, which show the marker where + the elements come from and which `spec-qa.py` checks against the document; a reader's + quotation need not be that one. The required elements are what the question asks for, no + more: an element the question does not ask for is not required, however load-bearing, until + the question is reworded to ask for it. - **Question 3**: for each item, the grade or grades accepted, any condition on the reasoning, and the wrong readings the text invites, which count against the document when one reader takes them. @@ -55,9 +58,12 @@ A key, a question set or a fixture changes by one of two routes, as `ACCEPTANCE. recorded is settled by whichever route it needs: - **In a specification pull request, under step 4 of `CONTRIBUTING.md`**: a quotation follows - the text it quotes, or a keyed item is added for a new load-bearing obligation together with - the question that asks for it. The pull request's own changelog line records it. Such an - update does not change what an existing item requires, nor an existing question's wording. + the text it quotes; a keyed item is added for a new load-bearing obligation together with the + question that asks for it; or, where the change alters or removes the obligation an existing + item tests, that item and its question follow the text, which is the text changing and not + the test. The pull request's own changelog line records it. Such an update does not change + what an item requires of an obligation the change leaves as it was, nor the wording of a + question whose obligation is unchanged. - **In a process change, with its own pull request**: what an existing item requires, the wording of a question, a fixture, or the rules of marking. It is recorded in [CHANGES.md](CHANGES.md) with its reason. diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 60fc616..24beef9 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -1,8 +1,8 @@ # Passages routed out of a cohort -A finding is a passage two or more readers of one cohort named as confusing, under question 4, -that lies outside the text the change being tested wrote or altered. `ACCEPTANCE.md` routes such -a finding here rather than blocking the change whose cohort raised it, so that each change is +A finding is a passage two or more readers of one cohort named as confusing under question 4. +`ACCEPTANCE.md` routes here a finding that lies outside the text the change being tested wrote +or altered, rather than blocking the change whose cohort raised it, so that each change is judged on what it changed and the document is still made clearer, one change at a time. The editor reads this list before cutting a release. diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index aaddb69..5b4d4aa 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -7,8 +7,8 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement -A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. +A bullet is correct when it names one of these obligations. Four or more correct bullets, with +no bullet making an assertion listed below as wrong, is a correct restatement. - Support bespoke rules written by the project ([4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it)). - Provide a harness that runs a single rule against supplied code ([4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code)). @@ -39,10 +39,11 @@ Wrong readings the text invites, which count against the document on one reader: MAY declare the version of this document the detector conforms to". The reason, that conformance is judged on evidence and a declaration is optional, is explanation: an answer that gives it, or gives it instead of a plain yes, states element (2). -- **2b.** Required: (1) it must let the project switch the route off in its configuration, or - make the route detectable (by a rule the project can write, or by a mechanical check the - detector documents; naming either means of detection, or just "detectable", states this); - (2) yes, either one alone is enough. Wrong if it says the detector must remove or forbid the +- **2b.** Required: (1) the two things, both named: let the project switch the route off in its + configuration, or make the route detectable (by a rule the project can write, or by a + mechanical check the detector documents; naming either means of detection, or just + "detectable", states that alternative); an answer naming only one of the two things misses + this element; (2) yes, either one alone is enough. Wrong if it says the detector must remove or forbid the route, or must require a reason ([7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason) is SHOULD). Key quotation, [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable): "For each one it MUST do at least one of two things, and either alone satisfies this clause: make the route disableable by configuration; or make the route detectable, by a Rule the project can write in diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index b756f47..a990c7b 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -7,8 +7,8 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement -A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. +A bullet is correct when it names one of these obligations. Four or more correct bullets, with +no bullet making an assertion listed below as wrong, is a correct restatement. - Attribute the defect to a class, bounded on both sides, with the hazard stated ([3.1](../../SPEC.md#31-attribute-the-defect-to-a-class)). - Express the class as a rule in a detector that reads code, never a test ([3.2](../../SPEC.md#32-build-the-net)). @@ -69,7 +69,8 @@ Wrong readings the text invites, which count against the document on one reader: The three answers the question then asks for, each with the condition its reason must meet: - **The remediation under section [7](../../SPEC.md#7-conformance).** Accepted answer: does not conform. Condition: the reason - names [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed; other clauses named alongside are not marked. + names [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), [3.5](../../SPEC.md#35-enforce-permanently-and-block) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed; other clauses named alongside are not + marked. - **The defence alone.** Accepted answer: does not conform. Condition: the reason names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation), or [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent; other clauses named alongside are not marked. diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index 7353852..2d2bea3 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -7,8 +7,8 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement -A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. +A bullet is correct when it names one of these obligations. Four or more correct bullets, with +no bullet making an assertion listed below as wrong, is a correct restatement. - Route defences only through detectors that conform to the detector specification, wrapping permitted ([4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification)). @@ -69,9 +69,11 @@ Wrong readings the text invites, which count against the document on one reader: The four answers the question then asks for, each with the condition its reason must meet: - **Conformance under section [9](../../TOOLING-SPEC.md#9-conformance).** Accepted answer: does not conform. Condition: the reason - names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses; other clauses named alongside are not marked. + names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) or [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) among the failing clauses; other clauses named alongside are not + marked. - **Conformance with agent support.** Accepted answer: no. Condition: the reason says that - section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, or that there is no summary. + section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, or that there is no summary, or that the toolchain does not conform + under section [9](../../TOOLING-SPEC.md#9-conformance) at all. - **Whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies.** Accepted answer: it does not. Condition: the reason says that Zigpipe ships nothing to other projects, or that section [8](../../TOOLING-SPEC.md#8-self-audit) bears on zig-qa's own artefact grade and not on Zigpipe. From c31d49a0ac8e66931784ac09ef342d78347b0957 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 12:00:29 +0100 Subject: [PATCH 05/34] Acceptance process: CHANGES.md names pull request 22 Co-Authored-By: Claude Fable 5.1 --- acceptance/CHANGES.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md index 29a20db..daa3d44 100644 --- a/acceptance/CHANGES.md +++ b/acceptance/CHANGES.md @@ -6,7 +6,8 @@ change, naming its pull request. The process is not a specification and is not v change to it is not tested by a cohort. Changes to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). -- Pull request NNN: the test is made to prove what it exists to prove, the clarity of the +- [Pull request #22](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/22): + the test is made to prove what it exists to prove, the clarity of the document to a cold reader, after thirty-three runs on two changes (19 to 30, 31 to 33 and 35 to 52) in which keyed answers reached five of five and the runs still failed. Five defects in the process were found. Keys required elements the question never asked for (detector 2b's From e6bdf402e051155cf4b7752f62ad4dd8a5ef41a1 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 12:57:49 +0100 Subject: [PATCH 06/34] Keys and routed findings: open points from runs 53 to 59 The method key lists fixing the original defect with a reproducing test among the obligations, says a bullet on the verdict resting on reproduction describes the reviewer, and says that an answer to 2a naming no step does not state the steps. The toolchain key lists the governing principle, says a bullet on how conformance is measured is neither correct nor wrong, and accepts a section 9 reason that points to the grades given. acceptance/ROUTED.md gains the findings runs 53 to 59 routed, and the runs that raised existing rows again. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 33 +++++++++++++++++++-------------- acceptance/method/KEY.md | 9 +++++++-- acceptance/toolchain/KEY.md | 12 +++++++++--- 3 files changed, 35 insertions(+), 19 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 24beef9..901caed 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -10,24 +10,29 @@ A finding is open until a change that reworks its passage passes a cohort, when that run and moves below the rule, or until the editor closes it with a reason recorded here. A finding raised again by a later run is noted on its line, not listed twice. The status column holds `open`, or the `editor/` branch or pull request number of the change that is -reworking the passage. The rows below were raised by runs 31 to 52 under the earlier rules, on +reworking the passage. The rows raised by runs 31 to 52 were raised under the earlier rules, on untouched text of the change those runs tested, and are listed here so that they are not lost. ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | ---------------------------------------- | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | run 33 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- | ---------------------------------------- | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59 | `open` | ## Closed diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index a990c7b..86cf8ff 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -21,9 +21,13 @@ no bullet making an assertion listed below as wrong, is a correct restatement. than only mentioning it (section [2](../../SPEC.md#2-when-the-method-applies)). - Take a baseline, a suppression or an instance left unfixed to the owner, never on the practitioner's own authority (section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom)). +- Fix the original defect conventionally, with a test that reproduces it, once the defence is in + place (sections [2](../../SPEC.md#2-when-the-method-applies) and [6](../../SPEC.md#6-relationship-to-test-driven-development)). Wrong if it asserts: the practitioner fixes the reported instance first and defends afterwards; -a test may serve as the detector; an agent may baseline the remaining instances. +a test may serve as the detector; an agent may baseline the remaining instances. A bullet that +says the verdict on a remediation rests on reproduction (section [7](../../SPEC.md#7-conformance)) describes the reviewer, +not the practitioner, and is neither correct nor wrong. Wrong readings the text invites, which count against the document on one reader: none named. @@ -36,7 +40,8 @@ Wrong readings the text invites, which count against the document on one reader: order."; [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance): "The Practitioner MUST run the Rule across the entire codebase and record the total Instance count before fixing anything, and MUST then fix every Instance found rather than only the one". An answer that names the steps by their clause numbers in order states - both elements. + both elements; one that says only that section [3](../../SPEC.md#3-the-method)'s six clauses are followed in order, + naming no step, does not state element (1). - **2b.** Required: (1) the owner of the codebase, the human who holds that authority; (2) no, an agent may not do so on its own authority. Wrong if it lets the practitioner or an agent decide because the count was small, because the project recorded no owner, or at all. Key diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index 2d2bea3..e0388d4 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -24,11 +24,16 @@ no bullet making an assertion listed below as wrong, is a correct restatement. same way as the defences ([6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) to [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences)). - A shipped toolchain also fails its release on undocumented bundled defences and runs its own defences on its own source ([8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation), [8.2](../../TOOLING-SPEC.md#82-a-shipped-toolchain-must-run-its-own-bundled-defences-on-its-own-source)). +- Make everything the method requires of a practitioner possible without the project building + the mechanism first (section [1](../../TOOLING-SPEC.md#1-what-this-document-is-for), the governing principle). Wrong if it asserts: the toolchain must be a single product; a declaration is required to conform; the agent summary (section [7](../../TOOLING-SPEC.md#7-agent-context)) is a MUST. A bullet that calls agent support or the self-audit a further condition "where it applies", or says a declaration is not a condition, -asserts no obligation and is neither correct nor wrong. +asserts no obligation and is neither correct nor wrong. So is a bullet that states how +conformance is measured rather than what is required: the toolchain judged as one assembled +whole at the project level (sections [1](../../TOOLING-SPEC.md#1-what-this-document-is-for) and [9](../../TOOLING-SPEC.md#9-conformance)), the project-level verdict a single grade +(section [9](../../TOOLING-SPEC.md#9-conformance)), or a shipped artefact graded separately ([9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately)). Wrong readings the text invites, which count against the document on one reader: none named. @@ -69,8 +74,9 @@ Wrong readings the text invites, which count against the document on one reader: The four answers the question then asks for, each with the condition its reason must meet: - **Conformance under section [9](../../TOOLING-SPEC.md#9-conformance).** Accepted answer: does not conform. Condition: the reason - names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) or [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) among the failing clauses; other clauses named alongside are not - marked. + names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) or [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) among the failing clauses, or points to the grades the reader gave above, as "the + clauses graded No", where those grades have one of the three at No or Partial; other clauses + named alongside are not marked. - **Conformance with agent support.** Accepted answer: no. Condition: the reason says that section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, or that there is no summary, or that the toolchain does not conform under section [9](../../TOOLING-SPEC.md#9-conformance) at all. From ed30e694e1d80e342beef8910228f01602d01557 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:02:16 +0100 Subject: [PATCH 07/34] Detector: the detectable route, the gap duty, the remainder and the mechanism Four passages acceptance/ROUTED.md lists, each quoted by two or more cold readers of the defined-terms change. Clause 7.1's detectable route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph on what detectable means and the two ways; clause 8.1's gap duty falls only on a detector that declares a version; clause 5.3 says what the remainder is and when there is none; clause 6.1 says it asks for the mechanism, a missing page failing 6.3, and that a path in the message is not enough on its own. The detector key's 7.1 quotation follows the text. Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 16 ++++++++++++++ DETECTOR-SPEC.md | 44 ++++++++++++++++++++++++++------------ acceptance/detector/KEY.md | 17 ++++++++------- 3 files changed, 55 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f5f8122..bfeb0df 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,22 @@ pass it. ## Detector specification (DETECTOR-SPEC.md) +### Unreleased + +- Clarity, no obligation changed: four passages that cold readers of an unrelated change read as + the document pulling against itself or could not follow, each listed in `acceptance/ROUTED.md`. + Clause [7.1](DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable)'s detectable route is now a bespoke rule the project writes for the detector under + clause [4.1](DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it), after six runs read "a Rule the project can write in the Detector itself" as the + project editing the detector's own source, and a new paragraph says what detectable means and + what each of the two ways is. Clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps)'s duty to record a known gap falls only on a detector + that declares a version, and says so beside "The declaration is optional", with the method's + clause [3.2](SPEC.md#32-build-the-net) record applying either way. Clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) says what the remainder is, and that + there is none where the output prints every finding. Clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) says it asks for the + mechanism and not for every page, a missing page failing clause [6.3](DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), and that a + documentation path printed in the message is not enough on its own, after three readers of + run 54 set that sentence against the mechanism. Prompted by runs 32 to 54 on the defined-terms + change, in which two or more readers quoted each passage. Accepted by run NNN. + ### 1.0.0, 2026-09-08 First version, split out of toolchain specification 0.1.0 on the same day. States what a detector, diff --git a/DETECTOR-SPEC.md b/DETECTOR-SPEC.md index cdc6c9e..7583801 100644 --- a/DETECTOR-SPEC.md +++ b/DETECTOR-SPEC.md @@ -157,7 +157,10 @@ any. ### 5.3 The result MUST reach the practitioner in the output of the command they ran Where the [Detector] writes fuller detail elsewhere, the invoked command's own output MUST -carry both a usable summary and the location of the remainder. +carry both a usable summary and the location of the remainder. The remainder is the findings +the summary leaves out. Where the output prints every finding, each with where it is and its +[Identifier], there is no remainder, and a report that only adds detail to those same findings, +such as a code snippet, need not be named in the output. **Why**: method specification clause 8.2. A report the [Practitioner] has to go and find is a report that arrives after the decision it was meant to inform. @@ -183,13 +186,16 @@ Two cases, by who wrote the [Rule]: | The project's own | The project | Print the [Identifier] unaltered; the lookup is the [Toolchain]'s to give | In both, the mechanism is keyed on **the [Identifier] exactly as printed**. A command, an index -file or a URL are all acceptable forms. Each case in turn: +file or a URL are all acceptable forms. This clause asks for the mechanism, not for every page: +where the mechanism works but a [Bundled rule]'s [Identifier] has no documentation to resolve +to, that fails clause 6.3, not this clause. Each case in turn: **For a [Bundled rule]**, the [Detector] supplies the documentation and the lookup, and clauses 6.2 and 6.3 say where. The mechanism MUST resolve an [Identifier] presented alone, because the reader who needs it most has the [Identifier] from a log, a -ticket or a colleague and not the [Message]. A [Message] that carries its own -documentation path resolves that [Message], not the [Identifier]. +ticket or a colleague and not the [Message]. A documentation path printed in the [Message] is +therefore not enough on its own: it helps only a reader who has the [Message], and the +[Identifier] alone still leads nowhere. **For a project's own [Rule]**, the [Detector] cannot know the documentation, so what it owes is the half it can give: the [Identifier] printed unaltered under clause 4.3, and no @@ -236,7 +242,7 @@ An automated check over every [Identifier] a [Bundled rule] can print, applying family list or pattern of clause 6.5 where one is used, that blocks the release when any of them lands on no page. -**Why**: clause 6.1 is the clause most easily believed to be satisfied whilst being broken, because +**Why**: clause 6.3 is the clause most easily believed to be satisfied whilst being broken, because the documentation is written by the same person who wrote the [Rule] and its absence is invisible from the inside. Where the [Detector] is shipped inside a [Toolchain], the [toolchain specification](TOOLING-SPEC.md)'s self-audit makes this check a MUST for the [Toolchain]; a @@ -269,9 +275,18 @@ nothing to run. An inline ignore comment, a per-line directive and a generated [Baseline] are all such routes. The [Detector] MAY ship them. For each one it MUST do at least one of two things, and either alone satisfies this clause: make the route disableable by configuration; or make the route -detectable, by a [Rule] the project can write in the [Detector] itself or by a mechanical check -the [Detector] documents. Either lets a project which decides to forbid the route enforce that -decision. A route that can be neither switched off nor seen does not [Conform]. +detectable, by a bespoke [Rule] the project writes for the [Detector] under clause 4.1 or by a +mechanical check the [Detector] documents. Either lets a project which decides to forbid the +route enforce that decision. A route that can be neither switched off nor seen does not [Conform]. + +A route is detectable when the project can find every use of it. The first way is a bespoke +[Rule]: the project registers a [Rule] of its own, as clause 4.1 provides for, that matches the +route wherever it is used, and the [Detector] runs it like any other bespoke [Rule]; the +[Detector] ships no such [Rule] itself, and need not. That way is open only where the +[Detector]'s [Rules] can see the route: a [Detector] whose [Rules] never see comments cannot make +an ignore comment detectable this way. The second way is a mechanical check the [Detector] +documents, such as a text search for the route's spelling that the [Detector]'s documentation +gives. **Why**: the method specification's position is that [Suppression] is an [Owner] decision under its clause 3.4 and section 4, and an [Owner] cannot decide something they are @@ -310,14 +325,15 @@ for a [Detector]. The declaration is machine-readable, in whatever form the [Detector]'s ecosystem uses to record dependencies. The same declaration is where a gap against this document is recorded once it is -known: a [Detector] that has learnt, from its own checks or from a [Practitioner]'s -report under the method's clause 3.2, that it fails a MUST in sections 4 to 7 MUST record that gap -alongside the version it declares, in the same file or one it names. A declaration with a non-empty -gap record is a statement of where the [Detector] stands and is not a claim of +known: a [Detector] that declares a version and has learnt, from its own checks or from a +[Practitioner]'s report under the method's clause 3.2, that it fails a MUST in sections 4 to 7 +MUST record that gap alongside that version, in the same file or one it names. A declaration +with a non-empty gap record is a statement of where the [Detector] stands and is not a claim of [Conformance]. -The declaration is optional. It is how a maintainer claims [Conformance]; it is not a -condition of it. A [Detector] that predates this document, or whose maintainer has never read +The declaration is optional, and so the duty above falls only on a [Detector] that has one; a +gap a [Practitioner] finds is recorded under the method's clause 3.2 in any case. It is how a +maintainer claims [Conformance]; it is not a condition of it. A [Detector] that predates this document, or whose maintainer has never read it, MAY be graded [Conforming] on evidence by anyone who exercises the clauses above against it, and a verdict on any [Detector], declared or not, rests on that exercise and not on the claim. A declaration tells the reader what the maintainer believes and what they know to be missing; diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index 5b4d4aa..1f4348b 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -40,14 +40,15 @@ Wrong readings the text invites, which count against the document on one reader: conformance is judged on evidence and a declaration is optional, is explanation: an answer that gives it, or gives it instead of a plain yes, states element (2). - **2b.** Required: (1) the two things, both named: let the project switch the route off in its - configuration, or make the route detectable (by a rule the project can write, or by a - mechanical check the detector documents; naming either means of detection, or just - "detectable", states that alternative); an answer naming only one of the two things misses - this element; (2) yes, either one alone is enough. Wrong if it says the detector must remove or forbid the - route, or must require a reason ([7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason) is SHOULD). Key quotation, [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable): "For each one it - MUST do at least one of two things, and either alone satisfies this clause: make the route - disableable by configuration; or make the route detectable, by a Rule the project can write in - the Detector itself or by a mechanical check the Detector documents". An answer need not say + configuration, or make the route detectable (by a bespoke rule the project writes for the + detector, or by a mechanical check the detector documents; naming either means of detection, + or just "detectable", states that alternative); an answer naming only one of the two things + misses this element; (2) yes, either one alone is enough. Wrong if it says the detector must + remove or forbid the route, or must require a reason ([7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason) is SHOULD). Key quotation, + [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable): "For each one it MUST do at least one of two things, and either alone satisfies this + clause: make the route disableable by configuration; or make the route detectable, by a + bespoke Rule the project writes for the Detector under clause [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it) or by a mechanical check the + Detector documents". An answer need not say that the detector is not required to remove or forbid the route; that follows from element (2) and the question does not ask it. - **2c.** Required: (1) with the rule, in the installed copy; (2) at a version tracked together From 4081aa45e771f14fb832e708ba612c88f5008d8e Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:16:08 +0100 Subject: [PATCH 08/34] Toolchain key and routed findings: open points from runs 61 and 62 The toolchain key counts a bullet on not describing partial conformance as conformance, and says a bullet calling section 7 recommended or a SHOULD asserts no obligation. acceptance/ROUTED.md gains clause 3.4's "tremendous" sentence from run 61 and notes that run on the two rows it raised again. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 5 +++-- acceptance/toolchain/KEY.md | 6 ++++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 901caed..76acd8f 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -31,8 +31,9 @@ untouched text of the change those runs tested, and are listed here so that they | `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56 | `open` | | `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | | `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | ## Closed diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index e0388d4..aadab7a 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -26,11 +26,13 @@ no bullet making an assertion listed below as wrong, is a correct restatement. defences on its own source ([8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation), [8.2](../../TOOLING-SPEC.md#82-a-shipped-toolchain-must-run-its-own-bundled-defences-on-its-own-source)). - Make everything the method requires of a practitioner possible without the project building the mechanism first (section [1](../../TOOLING-SPEC.md#1-what-this-document-is-for), the governing principle). +- Not describe partial conformance as conformance (section [9](../../TOOLING-SPEC.md#9-conformance)). Wrong if it asserts: the toolchain must be a single product; a declaration is required to conform; the agent summary (section [7](../../TOOLING-SPEC.md#7-agent-context)) is a MUST. A bullet that calls agent support or the -self-audit a further condition "where it applies", or says a declaration is not a condition, -asserts no obligation and is neither correct nor wrong. So is a bullet that states how +self-audit a further condition "where it applies", says that section [7](../../TOOLING-SPEC.md#7-agent-context) is recommended or a +SHOULD, or says a declaration is not a condition, asserts no obligation and is neither correct +nor wrong. So is a bullet that states how conformance is measured rather than what is required: the toolchain judged as one assembled whole at the project level (sections [1](../../TOOLING-SPEC.md#1-what-this-document-is-for) and [9](../../TOOLING-SPEC.md#9-conformance)), the project-level verdict a single grade (section [9](../../TOOLING-SPEC.md#9-conformance)), or a shipped artefact graded separately ([9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately)). From c1c986e0804af06129ee4f4308168f71e227341a Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:17:27 +0100 Subject: [PATCH 09/34] Run 060: detector route rule cohort, failed on two findings Every keyed item within threshold, one reader leaving clause 7.1 ungraded. Two findings in text the change wrote or altered: clause 6.1's sentence on a documentation path in the Message, whose term two readers could not find defined, and clause 8.1's "anyone who exercises the clauses above". Nothing routed. Co-Authored-By: Claude Fable 5.1 --- .../runs/060-detector-route-rule-run-1.md | 138 ++++++++++++++++++ 1 file changed, 138 insertions(+) create mode 100644 acceptance/runs/060-detector-route-rule-run-1.md diff --git a/acceptance/runs/060-detector-route-rule-run-1.md b/acceptance/runs/060-detector-route-rule-run-1.md new file mode 100644 index 0000000..20317b4 --- /dev/null +++ b/acceptance/runs/060-detector-route-rule-run-1.md @@ -0,0 +1,138 @@ +# Run 60: DETECTOR-SPEC.md, detector route rule, first run + +- **Document**: DETECTOR-SPEC.md +- **Pull request**: not yet opened at the time of the run; it is to be opened from branch + `editor/detector-route-rule`. The readers read commit ed30e69. Change: clause 7.1's detectable + route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph + on what detectable means; clause 8.1's gap duty falls only on a detector that declares a + version; clause 5.3 says what the remainder is; clause 6.1 says it asks for the mechanism, a + missing page failing clause 6.3, and that a path in the message is not enough on its own. +- **Run within the pull request**: 1. +- **Cohort**: five readers, none reused from any earlier run. Model named for the run before the + readers started: `claude-haiku-5-5`. Every reader reported `claude-haiku-5-5`. +- **Isolation**: each reader a separate `claude -p` process, started through the `ccy-claude` + wrapper (which only reattaches the session's credential) in a fresh empty directory made by + `mktemp -d` outside any repository, with the packet on standard input; no tools, no settings, + no project context, no saved session; the machine applies no managed settings. The editor + attests the rest of the isolation. Command: + `ccy-claude -p --model haiku --safe-mode --tools '' --setting-sources '' --strict-mcp-config --no-session-persistence --output-format json < prompt.txt`. + The prompt (sha256 `d3b016b18288e3589df60de97f746a31179924dbdd5faa339c055fbf95d89125`) is + `QUESTIONS.md`, `DETECTOR-SPEC.md` and `fixtures/lintomatic.md` at ed30e69, each after one + `===== FILE: =====` line, one blank line between files; the marker rebuilt it from that + commit and it matched byte for byte. +- **Readers set aside**: none. Every reader's output has `is_error` false, subtype `success`, + non-empty answer text and the cohort's model. +- **Marked by**: a delegated marker, a fresh `dbf-spec-editor` agent (Claude Opus) given the + marking instruction and its agent definition, having seen neither the drafting nor the review. + +## Marks + +Readers are numbered 1 to 5 as their sheets are. Question 3 columns: the eleven clauses, then +the two answers (S8 conformance under section 8, 6.4 whether clause 6.4 changes the verdict). + +| Reader | Q1 | 2a | 2b | 2c | 4.1 | 4.2 | 4.3 | 5.1 | 5.2 | 5.3 | 5.4 | 6.1 | 6.2 | 6.3 | 7.1 | S8 | 6.4 | +| ------ | ------- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---- | --- | --- | +| 1 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 2 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 3 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 4 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 5 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | miss | ok | ok | + +Question 1: correct bullets per reader 6, 6, 6, 6 and 6, one for each line of the key's list +(4.1; 4.2; 4.3; 5.1 to 5.4; 6.1 to 6.3; 7.1). Reader 1's third bullet adds 4.4 as a SHOULD and +its sixth adds what conformance means beside 7.1; both are, by the key, neither correct nor +wrong, and each bullet still names its obligation. Reader 2's fourth bullet says every finding +must reach the output, which the key says names 5.3 and is not wrong. No bullet asserts anything +the key marks wrong. + +Question 2: every reader states every required element of each item and quotes the document. +For 2a all five quote section 8's "A Detector Conforms if it satisfies every MUST in sections 4 +to 7" or clause 8.1's "A Detector that predates this document, or whose maintainer has never +read it, MAY be graded Conforming on evidence"; for 2b all five quote clause 7.1's "For each one +it MUST do at least one of two things, and either alone satisfies this clause", and name both +things and that either is enough; for 2c all five quote the headings or first sentences of 6.2 +and 6.3. Readers 1 and 2, under 2b, add that the detectable route is closed to a detector whose +rules never see comments, so that it needs the disable route or a documented mechanical check; +that is not among the key's wrong assertions for 2b, and the answers are correct by the rule. + +Question 3: readers 1 to 4 give every accepted grade; reader 5 gives every accepted grade on the +ten rows it answers. S8: all five say Lintomatic does not conform, naming 6.3. 6.4: all five say +it does not change the verdict, because it is a SHOULD. + +### Misses + +- **Reader 5, question 3, clause 7.1**: unanswered. Its table ends at 6.3 and no grade for 7.1 + appears anywhere on the sheet, so the item is a miss under "An unanswered question". There is + no reasoning, so nothing quotes the document; it is not a reading the key names (7.1 graded + No on 7.2's reason). The other four readers are correct on the item, so the threshold of all + but one is met. + +## Question 4: passages named + +Paragraphs as `ACCEPTANCE.md` defines them; "changed" means the diff 9e914fe..ed30e69 adds, +rewords, moves or deletes a sentence in the paragraph, or changes a word, term or link in it. + +| # | Passage (clause, words or description) | Readers | Changed by the diff | +| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- | ------------------- | +| 1 | Clause 6.1, "For a Bundled rule" paragraph: "A documentation path printed in the [Message] is therefore not enough on its own"; both readers say [Message] is not defined in the document they were given | 1, 5 | yes | +| 2 | Clause 8.1, second paragraph: "MAY be graded [Conforming] on evidence by anyone who exercises the clauses above against it" (5, whether the MAY is a permission or a possibility); "by anyone who exercises the clauses above against it" (3, whether "the clauses above" are sections 4 to 7 or earlier clauses of section 8) | 3, 5 | yes | +| 3 | Clause 6.1, "For a project's own Rule" paragraph: "Where the [Identifier] is itself a URL, as method clause 3.6 allows"; the reader could not check method clause 3.6 | 1 | no | +| 4 | Clause 6.1 heading: "The Detector MUST provide a mechanism that resolves a printed identifier to its documentation", set against passage 5 | 2 | no | +| 5 | Clause 6.1 table, project's own row: "Print the Identifier unaltered; the lookup is the Toolchain's to give", set against passage 4 | 2 | no | +| 6 | Clause 5.3, first paragraph: "a report that only adds detail to those same findings, such as a code snippet, need not be named in the output" | 2 | yes | +| 7 | Clause 8.1, first paragraph: "a [Detector] that declares a version and has learnt ... that it fails a MUST in sections 4 to 7 MUST record that gap alongside that version", set against section 8's definition and the optional declaration | 3 | yes | +| 8 | Clause 4.3, first paragraph: "in its default output and in every machine-readable format it offers"; "machine-readable" is not defined | 4 | no | + +## Findings and where they lie + +- **Finding A, passage 1, clause 6.1 "For a Bundled rule" paragraph** (readers 1 and 5): the + sentence both readers quote was written by the change, replacing "A [Message] that carries its + own documentation path resolves that [Message], not the [Identifier]." Text the change wrote. + Blocks. +- **Finding B, passage 2, clause 8.1 second paragraph** (readers 3 and 5): the sentence both + readers quote is not itself reworded, but the change rewrote the paragraph's first sentence + ("The declaration is optional, and so the duty above falls only on a [Detector] that has one; + ...") and added its second. The rule places a finding by paragraph, so the paragraph is the + change's. Blocks. + +Passages 3 to 8 are each named by one reader: recorded, not findings. Nothing is routed to +`acceptance/ROUTED.md` from this run. + +## Points the rules did not settle + +1. **Passage 1, a term defined in a document not supplied.** Readers 1 and 5 each quote the + sentence with its clause number, which `ACCEPTANCE.md` defines as naming a passage, and the + confusion each gives is that [Message] is defined in the method specification, which they + were not given. The rule that a remark about another document not being supplied is a note, + not a passage, covers a remark "not about a passage of the document"; these remarks are about + a quoted passage, so they were read as naming it. Had it been open, the open-point rule on + whether a reader named a passage counts the passage in any case. Finding A turns on this + point. Reader 1's passage 3 (method clause 3.6) is the same case, counted the same way, and + is named by one reader only. + +## Process change + +The run was made under the acceptance process change of pull request 22 before it merged, +following all of it at commit e6bdf40, named before the readers started. + +## Against the criteria + +- Load-bearing statements: five of five on 2a, 2b and 2c. Met. +- Restatement and application: five of five on question 1 and on every question 3 item but + 7.1, where four of five are correct (reader 5 left it unanswered). Met. +- A miss the key says counts against the document: none. +- Passages named by two or more readers in text the change wrote or altered: two (findings A + and B). Not met. +- Passages named by two or more readers elsewhere: none. + +## Findings applied + +None at the time of marking; for the editor, under "After a failure" in `ACCEPTANCE.md`. + +## Verdict + +FAIL, on findings A and B alone; every keyed item passed its threshold. + +## Answer sheets + +Posted on the pull request: LINK TO BE ADDED. From 4cbecad23a52dbdf54b48292f3c1c8d422f4e956 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:17:27 +0100 Subject: [PATCH 10/34] Detector: a Message row in section 2, and who may grade a detector After run 60: section 2's table gains a Message row, since two readers could not find the term defined; clause 8.1 says anyone may grade a detector by checking it against each clause of sections 4 to 7, in place of "anyone who exercises the clauses above against it". Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 7 ++++++- DETECTOR-SPEC.md | 7 ++++--- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bfeb0df..9d6fc71 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -82,7 +82,12 @@ pass it. mechanism and not for every page, a missing page failing clause [6.3](DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), and that a documentation path printed in the message is not enough on its own, after three readers of run 54 set that sentence against the mechanism. Prompted by runs 32 to 54 on the defined-terms - change, in which two or more readers quoted each passage. Accepted by run NNN. + change, in which two or more readers quoted each passage. + [Run 60](acceptance/runs/060-detector-route-rule-run-1.md) failed on two findings in the + revised text, every keyed item within threshold: two readers could not find Message defined, + so section [2](DETECTOR-SPEC.md#2-terminology)'s table gains a Message row; and two readers stopped at clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps)'s + "anyone who exercises the clauses above", which now says anyone may grade a detector by + checking it against each clause of sections [4](DETECTOR-SPEC.md#4-authoring-rules) to [7](DETECTOR-SPEC.md#7-suppression). Accepted by run NNN. ### 1.0.0, 2026-09-08 diff --git a/DETECTOR-SPEC.md b/DETECTOR-SPEC.md index 7583801..06a4238 100644 --- a/DETECTOR-SPEC.md +++ b/DETECTOR-SPEC.md @@ -39,6 +39,7 @@ definition; the ones this document leans on most are, in short: | -------------- | ------------------------------------------------------------------------------ | | [Detector] | A tool that reads code without executing it and reports what it finds | | [Rule] | One check a [Detector] evaluates | +| [Message] | The text a [Rule] prints when it fires | | [Identifier] | The stable name printed with a finding that leads to its documentation | | [Practitioner] | Whoever is doing the work, a person or an [Agent] | | [Toolchain] | Everything a project assembles to run its checks through, [Detectors] included | @@ -334,9 +335,9 @@ with a non-empty gap record is a statement of where the [Detector] stands and is The declaration is optional, and so the duty above falls only on a [Detector] that has one; a gap a [Practitioner] finds is recorded under the method's clause 3.2 in any case. It is how a maintainer claims [Conformance]; it is not a condition of it. A [Detector] that predates this document, or whose maintainer has never read -it, MAY be graded [Conforming] on evidence by anyone who exercises the clauses above against -it, and a verdict on any [Detector], declared or not, rests on that exercise and not on the -claim. A declaration tells the reader what the maintainer believes and what they know to be missing; +it, MAY be graded [Conforming] on evidence: anyone may grade it, by checking it against each +clause of sections 4 to 7, and a verdict on any [Detector], declared or not, rests on that +check and not on the claim. A declaration tells the reader what the maintainer believes and what they know to be missing; the reader still runs the [Harness]. The shape is illustrative rather than prescribed. In a Composer manifest: From 99074928ac750fd2098b309725424dd221403539 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:29:03 +0100 Subject: [PATCH 11/34] Routed findings from runs 63 and 64 acceptance/ROUTED.md gains the detector's clause 6.1 paragraph on a project's own rule, from run 63, and notes run 64 on the four method rows it raised again. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 76acd8f..55462f4 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,25 +15,26 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- | ---------------------------------------- | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------------------------------- | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | run 63 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59, 64 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61, 64 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | ## Closed From 890d8e3d40e327caaf8776d890b97e4940bad536 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:29:20 +0100 Subject: [PATCH 12/34] Run 063: detector route rule cohort, failed on one finding Every keyed item five of five. One finding in text the change wrote: clause 5.3's sentence on a report that only adds detail. One finding in untouched text, clause 6.1's paragraph on a project's own rule, routed to acceptance/ROUTED.md. Co-Authored-By: Claude Fable 5.1 --- .../runs/063-detector-route-rule-run-2.md | 144 ++++++++++++++++++ 1 file changed, 144 insertions(+) create mode 100644 acceptance/runs/063-detector-route-rule-run-2.md diff --git a/acceptance/runs/063-detector-route-rule-run-2.md b/acceptance/runs/063-detector-route-rule-run-2.md new file mode 100644 index 0000000..c3fa574 --- /dev/null +++ b/acceptance/runs/063-detector-route-rule-run-2.md @@ -0,0 +1,144 @@ +# Run 63: DETECTOR-SPEC.md, detector route rule, second run + +- **Document**: DETECTOR-SPEC.md +- **Pull request**: not yet opened at the time of the run; it is to be opened from branch + `editor/detector-route-rule`. The readers read commit 4cbecad. Change: clause 7.1's detectable + route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph + on what detectable means; clause 8.1's gap duty falls only on a detector that declares a + version; clause 5.3 says what the remainder is; clause 6.1 says it asks for the mechanism, a + missing page failing clause 6.3, and that a path in the message is not enough on its own. + Revised after run 60 in section 2's table (a Message row) and clause 8.1's second paragraph. +- **Run within the pull request**: 2 (run 60 came before). +- **Cohort**: five readers, none reused from any earlier run. Model named for the run before the + readers started: `claude-haiku-5-5`. Every reader reported `claude-haiku-5-5`. +- **Isolation**: each reader a separate `claude -p` process, started through the `ccy-claude` + wrapper (which only reattaches the session's credential) in a fresh empty directory made by + `mktemp -d` outside any repository, with the packet on standard input; no tools, no settings, + no project context, no saved session; the machine applies no managed settings. The editor + attests the rest of the isolation. Command: + `ccy-claude -p --model haiku --safe-mode --tools '' --setting-sources '' --strict-mcp-config --no-session-persistence --output-format json < prompt.txt`. + The prompt (sha256 `30b41f341b26e48b3c8bf1b1d55a25283ffd43c835328eaf5cd9ec021d265e09`) is + `QUESTIONS.md`, `DETECTOR-SPEC.md` and `fixtures/lintomatic.md` at 4cbecad, each after one + `===== FILE: =====` line, one blank line between files; the marker rebuilt it from that + commit and it matched byte for byte. +- **Readers set aside**: none. Every reader's output has `is_error` false, subtype `success`, + non-empty answer text and the cohort's model. +- **Marked by**: a delegated marker, a fresh `dbf-spec-editor` agent (Claude Opus) given the + marking instruction and its agent definition, having seen neither the drafting nor the review. + +## Marks + +Readers are numbered 1 to 5 as their sheets are. Question 3 columns: the eleven clauses, then +the two answers (S8 conformance under section 8, 6.4 whether clause 6.4 changes the verdict). + +| Reader | Q1 | 2a | 2b | 2c | 4.1 | 4.2 | 4.3 | 5.1 | 5.2 | 5.3 | 5.4 | 6.1 | 6.2 | 6.3 | 7.1 | S8 | 6.4 | +| ------ | ------- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 1 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 2 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 3 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 4 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 5 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | + +Question 1: correct bullets per reader 6, 6, 6, 6 and 6, one for each line of the key's list +(4.1; 4.2; 4.3; 5.1 to 5.4; 6.1 to 6.3; 7.1). Reader 3's third bullet adds 4.4 as a SHOULD; +readers 2 and 3 add to their sixth bullet that a written reason is a SHOULD; reader 5's sixth +bullet adds the SHOULDs of 7.2, 4.4 and 6.4. Each restates a SHOULD as advice, which the key +says is neither correct nor wrong, and each bullet still names its obligation. No bullet asserts +anything the key marks wrong. + +Question 2: every reader states every required element of each item and quotes the document. +For 2a all five state that conformance is every MUST in sections 4 to 7 and that a detector +predating the document can conform, quoting section 8's definition or clause 8.1's "A Detector +that predates this document, or whose maintainer has never read it, MAY be graded Conforming on +evidence" (reader 1's quotation of section 8 drops "[Detector]", and quotes the document by the +nine-word run "if it satisfies every MUST in sections 4 to 7"; its 8.1 quotation is verbatim). +For 2b all five name both things and that either is enough, quoting clause 7.1's "For each one +it MUST do at least one of two things, and either alone satisfies this clause". Readers 2, 3 +and 5 add that the bespoke-rule way of detection is closed to a detector whose rules never see +comments; reader 2 goes on to say that for an ignore comment the configuration switch or a +documented mechanical check is what remains. Neither is among the key's wrong assertions for +2b (removing or forbidding the route, or requiring a reason), and the answers are correct by the +rule. For 2c all five state the installed copy, the version tracked together and no network +access, quoting the headings of 6.2 and 6.3. + +Question 3: all five give every accepted grade. Reader 2 grades 4.3 "Yes, with a caveat" about +machine-readable formats; the key accepts Yes with no condition on the reasoning and says a +caveat with the accepted grade is correct. S8: all five say Lintomatic does not conform, naming +6.3. 6.4: all five say it does not change the verdict, because it is a SHOULD. + +### Misses + +None. + +## Question 4: passages named + +Paragraphs as `ACCEPTANCE.md` defines them; "changed" means the diff 9e914fe..4cbecad adds, +rewords, moves or deletes a sentence in the paragraph, or changes a word, term or link in it. + +| # | Passage (clause, words or description) | Readers | Changed by the diff | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ------------------- | +| 1 | Clause 5.3, first paragraph: "a report that only adds detail to those same findings, such as a code snippet, need not be named in the output" (1, took two readings to see whether the exemption covers the HTML report; 2, the subject of "need not be named" took two readings to settle) | 1, 2 | yes | +| 2 | Clause 6.1, "For a project's own Rule" paragraph: "Where the [Identifier] is itself a URL, as method clause 3.6 allows, the [Detector] prints that URL and has done what this clause asks" (1 and 4, could not check method clause 3.6); "The lookup for such a [Rule] is an obligation on the project's assembled [Toolchain]" (5, set against passages 7 and 8) | 1, 4, 5 | no | +| 3 | Clause 4.3, first paragraph: "in its default output and in every machine-readable format it offers"; "machine-readable" not said, so whether the HTML report is covered | 2 | no | +| 4 | Clause 6.4, Why paragraph: "the [toolchain specification]'s self-audit makes this check a MUST for the [Toolchain]"; the reader was not given that document | 2 | yes | +| 5 | Clause 8.1, second paragraph: "A Detector that predates this document, or whose maintainer has never read it, MAY be graded Conforming on evidence"; whether the MAY is a permission to the grader or an eligibility of the detector, settled by the next sentence | 3 | yes | +| 6 | Clause 7.1, second paragraph: "That way is open only where the [Detector]'s [Rules] can see the route: a [Detector] whose [Rules] never see comments cannot make an ignore comment detectable this way"; what it means for a rule to "see" a comment | 4 | yes | +| 7 | Section 1, the governing principle paragraph: "a [Conforming] [Detector] MUST make that possible without the project building the mechanism first", set against passages 2 and 8 | 5 | no | +| 8 | Clause 6.1 table, project's own row: "the lookup is the [Toolchain]'s to give", set against passage 7 | 5 | no | + +Notes, not passages: reader 5 says the fixture's `explain` paragraph and its last paragraph +disagree about which rules print `No documentation declared`; that is about the fixture, not +the document. + +## Findings and where they lie + +- **Finding A, passage 1, clause 5.3 first paragraph** (readers 1 and 2): the sentence both + readers quote was added by the change, with "The remainder is the findings the summary leaves + out." Text the change wrote. Blocks. +- **Finding B, passage 2, clause 6.1 "For a project's own Rule" paragraph** (readers 1, 4 and + 5): the diff does not touch this paragraph, and no reader naming it sets it against a + paragraph the diff changed (reader 5 sets it against section 1's governing principle and the + 6.1 table's project row, both unchanged). Not the change's text. Routed to + `acceptance/ROUTED.md`; does not block. + +Passages 3 to 8 are each named by one reader: recorded, not findings. Passages 4, 5 and 6 lie +in changed paragraphs but are single. + +## Points the rules did not settle + +1. **Finding B, readers 1 and 4: a passage named for a reference to a document not supplied.** + Both quote the URL sentence with its clause number, which `ACCEPTANCE.md` defines as naming a + passage, and the confusion each gives is that method clause 3.6 is in a document they were + not given. The rule that a remark that another document was not supplied is a note covers a + remark "not about a passage of the document"; these remarks are attached to a quoted + passage. Marked as the open-point rule says: an open point on whether a reader named a + passage counts the passage. Without readers 1 and 4, reader 5 alone names the paragraph and + there is no finding B. Finding A does not turn on this point. Reader 2's passage 4 is the + same case, single either way. + +## Process change + +The run was made under the acceptance process change of pull request 22 before it merged, +following all of it at commit 4081aa4, named before the readers started. + +## Against the criteria + +- Load-bearing statements: five of five on 2a, 2b and 2c. Met. +- Restatement and application: five of five on question 1 and on every question 3 item. Met. +- A miss the key says counts against the document: none. +- Passages named by two or more readers in text the change wrote or altered: one (finding A). + Not met. +- Passages named by two or more readers elsewhere: one (finding B), routed to + `acceptance/ROUTED.md`. + +## Findings applied + +None at the time of marking; for the editor, under "After a failure" in `ACCEPTANCE.md`. + +## Verdict + +FAIL, on finding A alone; every keyed item passed its threshold. + +## Answer sheets + +Posted on the pull request: LINK TO BE ADDED. From d157143b7f4e6e36d62370de418ff7b69be5dce1 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:29:20 +0100 Subject: [PATCH 13/34] Detector 5.3: a fuller report elsewhere is not a remainder After run 63: the sentence on a report that only adds detail is a sentence of its own, with its subject stated: a fuller report written elsewhere, such as a page with a code snippet for each finding, only adds detail to findings the output already carries, is not a remainder, and need not be named. Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 7 ++++++- DETECTOR-SPEC.md | 5 +++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9d6fc71..532b83d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -87,7 +87,12 @@ pass it. revised text, every keyed item within threshold: two readers could not find Message defined, so section [2](DETECTOR-SPEC.md#2-terminology)'s table gains a Message row; and two readers stopped at clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps)'s "anyone who exercises the clauses above", which now says anyone may grade a detector by - checking it against each clause of sections [4](DETECTOR-SPEC.md#4-authoring-rules) to [7](DETECTOR-SPEC.md#7-suppression). Accepted by run NNN. + checking it against each clause of sections [4](DETECTOR-SPEC.md#4-authoring-rules) to [7](DETECTOR-SPEC.md#7-suppression). + [Run 63](acceptance/runs/063-detector-route-rule-run-2.md) failed on one finding in the revised + text, every keyed item five of five: two readers needed two readings of clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s + sentence on a report that only adds detail, which is now a sentence of its own with its subject + stated; three readers named clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation)'s paragraph on a project's own rule, which this + change does not touch and `acceptance/ROUTED.md` now lists. Accepted by run NNN. ### 1.0.0, 2026-09-08 diff --git a/DETECTOR-SPEC.md b/DETECTOR-SPEC.md index 06a4238..42369c6 100644 --- a/DETECTOR-SPEC.md +++ b/DETECTOR-SPEC.md @@ -160,8 +160,9 @@ any. Where the [Detector] writes fuller detail elsewhere, the invoked command's own output MUST carry both a usable summary and the location of the remainder. The remainder is the findings the summary leaves out. Where the output prints every finding, each with where it is and its -[Identifier], there is no remainder, and a report that only adds detail to those same findings, -such as a code snippet, need not be named in the output. +[Identifier], there is no remainder. A fuller report written elsewhere, such as a page with a +code snippet for each finding, only adds detail to findings the output already carries; it is +not a remainder, and the output need not name it. **Why**: method specification clause 8.2. A report the [Practitioner] has to go and find is a report that arrives after the decision it was meant to inform. From 5852a8562abd0828b9519726ddcce3490d13b388 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:40:51 +0100 Subject: [PATCH 14/34] Routed findings: four detector rows closed by run 66, two toolchain rows from run 67 acceptance/ROUTED.md gains a Closed table, with the detector's clauses 7.1, 8.1, 5.3 and 6.1 rows closed by editor/detector-route-rule's run 66, notes run 66 on the clause 6.1 row on a project's own rule, and gains the toolchain's clause 6.2 heading and documented-check paragraph from run 67. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 47 ++++++++++++++++++++++++-------------------- 1 file changed, 26 insertions(+), 21 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 55462f4..f995c21 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,27 +15,32 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------------------------------- | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | run 63 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59, 64 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61, 64 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61, 64 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------------------------------- | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | run 67 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59, 64 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61, 64 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | ## Closed -None yet. +A row moves here when a change that reworks its passage passes a cohort, naming that run, or +when the editor closes it with a reason. + +| Document | Clause | Passage | Raised by | Closed by | +| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ------------------------------------ | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule`, run 66 | From c78b76c413f7b3c6617621b00a1590f6c53a5389 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:40:57 +0100 Subject: [PATCH 15/34] Run 066: detector route rule cohort passed Every keyed item five of five and no finding in text the change wrote or altered. One finding in untouched text, clause 6.1's paragraph on a project's own rule, raised again and kept in acceptance/ROUTED.md. The changelog names runs 60, 63 and 66. Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 5 +- .../runs/066-detector-route-rule-run-3.md | 145 ++++++++++++++++++ 2 files changed, 149 insertions(+), 1 deletion(-) create mode 100644 acceptance/runs/066-detector-route-rule-run-3.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 532b83d..9cc1a76 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -92,7 +92,10 @@ pass it. text, every keyed item five of five: two readers needed two readings of clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s sentence on a report that only adds detail, which is now a sentence of its own with its subject stated; three readers named clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation)'s paragraph on a project's own rule, which this - change does not touch and `acceptance/ROUTED.md` now lists. Accepted by run NNN. + change does not touch and `acceptance/ROUTED.md` now lists. Accepted by + [run 66](acceptance/runs/066-detector-route-rule-run-3.md), every keyed item five of five; its + one finding is that same clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) paragraph, named again by three readers, which stays in + `acceptance/ROUTED.md` for a change of its own. ### 1.0.0, 2026-09-08 diff --git a/acceptance/runs/066-detector-route-rule-run-3.md b/acceptance/runs/066-detector-route-rule-run-3.md new file mode 100644 index 0000000..0d80045 --- /dev/null +++ b/acceptance/runs/066-detector-route-rule-run-3.md @@ -0,0 +1,145 @@ +# Run 66: DETECTOR-SPEC.md, detector route rule, third run + +- **Document**: DETECTOR-SPEC.md +- **Pull request**: not yet opened at the time of the run; it is to be opened from branch + `editor/detector-route-rule`. The readers read commit 44fdd9e. Change: clause 7.1's detectable + route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph + on what detectable means; clause 8.1's gap duty falls only on a detector that declares a + version; clause 5.3 says what the remainder is; clause 6.1 says it asks for the mechanism, a + missing page failing clause 6.3, and that a path in the message is not enough on its own; + section 2's table gains a Message row. Revised after run 63 in clause 5.3's sentence on a + fuller report elsewhere. +- **Run within the pull request**: 3 (runs 60 and 63 came before). +- **Cohort**: five readers, none reused from any earlier run. Model named for the run before the + readers started: `claude-haiku-5-5`. Every reader reported `claude-haiku-5-5`. +- **Isolation**: each reader a separate `claude -p` process, started through the `ccy-claude` + wrapper (which only reattaches the session's credential) in a fresh empty directory made by + `mktemp -d` outside any repository, with the packet on standard input; no tools, no settings, + no project context, no saved session; the machine applies no managed settings. The editor + attests the rest of the isolation. Command: + `ccy-claude -p --model haiku --safe-mode --tools '' --setting-sources '' --strict-mcp-config --no-session-persistence --output-format json < prompt.txt`. + The prompt (sha256 `5f991fd47d3b8d092464ca41bfdfd9f5707647f46e3b274929a0c5a44f5a50c4`) is + `QUESTIONS.md`, `DETECTOR-SPEC.md` and `fixtures/lintomatic.md` at 44fdd9e, each after one + `===== FILE: =====` line, one blank line between files; the marker rebuilt it from that + commit and it matched byte for byte. +- **Readers set aside**: none. Every reader's output has `is_error` false, subtype `success`, + non-empty answer text and the cohort's model. +- **Marked by**: a delegated marker, a fresh `dbf-spec-editor` agent (Claude Opus) given the + marking instruction and its agent definition, having seen neither the drafting nor the review. + +## Marks + +Readers are numbered 1 to 5 as their sheets are. Question 3 columns: the eleven clauses, then +the two answers (S8 conformance under section 8, 6.4 whether clause 6.4 changes the verdict). + +| Reader | Q1 | 2a | 2b | 2c | 4.1 | 4.2 | 4.3 | 5.1 | 5.2 | 5.3 | 5.4 | 6.1 | 6.2 | 6.3 | 7.1 | S8 | 6.4 | +| ------ | ------- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 1 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 2 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 3 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 4 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 5 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | + +Question 1: correct bullets per reader 6, 6, 6, 6 and 6, one for each line of the key's list +(4.1; 4.2; 4.3; 5.1 to 5.4; 6.1 to 6.3; 7.1). Readers 2 and 3 add to their sixth bullet that a +detector conforms when it meets every MUST in sections 4 to 7, which the key says states what +conformance means and is neither correct nor wrong; the bullet still names 7.1. Reader 4's +fourth bullet says every finding goes in the command's output, which the key says names 5.3 and +is not marked wrong for the plural. Reader 5's fifth bullet adds that a project rule's +identifier is printed unaltered, which is 4.3 restated inside the 6.1 to 6.3 bullet. No bullet +asserts anything the key marks wrong. + +Question 2: every reader states every required element of each item and quotes the document. +For 2a all five state that conformance is every MUST in sections 4 to 7 and that a detector +predating the document can conform, quoting section 8's definition or clause 8.1's "A Detector +that predates this document, or whose maintainer has never read it, MAY be graded Conforming on +evidence". For 2b all five name both things, disableable by configuration or detectable, and +that either is enough, quoting clause 7.1's "For each one it MUST do at least one of two +things, and either alone satisfies this clause". Readers 1, 3, 4 and 5 add that the bespoke-rule +way of detection is closed to a detector whose rules never see comments, which is the +document's own sentence and is not among the key's wrong assertions for 2b (removing or +forbidding the route, or requiring a reason). For 2c readers 1 to 4 state the installed copy, a +version tracked together and no network access, quoting the 6.2 and 6.3 headings; reader 5 +states all three in its own words (citing the headings without quotation marks) and quotes +clause 6.3's "Every [Identifier] a [Bundled rule] can print MUST resolve, under clause 6.1, to a +page in that shipped documentation", which quotes the document. + +Question 3: all five give every accepted grade. Readers 1 to 5 add caveats to 4.2 (the fixture +does not say `--only` prints finding lines) and 4.3 (no machine-readable format described); +reader 5 adds to 4.2 "If it does not, this is No", and reader 3 adds to its 6.3 No that the +grade would need revisiting if its reading of the fixture were wrong. Each gives the accepted +grade as its grade for the fixture as written, and the key says a reader who adds a caveat about +what the fixture does not say and still gives the accepted grade is correct. S8: all five say +Lintomatic does not conform, naming 6.3. 6.4: all five say it does not change the verdict, +because it is a SHOULD. + +### Misses + +None. + +## Question 4: passages named + +Paragraphs as `ACCEPTANCE.md` defines them; "changed" means the diff 9e914fe..44fdd9e adds, +rewords, moves or deletes a sentence in the paragraph, or changes a word, term or link in it. + +| # | Passage (clause, words or description) | Readers | Changed by the diff | +| --- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ------------------- | +| 1 | Clause 6.1, "For a project's own Rule" paragraph: "Where the [Identifier] is itself a URL, as method clause 3.6 allows, the [Detector] prints that URL and has done what this clause asks" and "The lookup for such a [Rule] is an obligation on the project's assembled [Toolchain], under clause 4.2 of the [toolchain specification]" (1, could not check the documents cited); "What this clause forbids is a documentation path printed alongside a shorter [Identifier] that cannot be looked up on its own" (3 and 4, what "shorter" is measured against) | 1, 3, 4 | no | +| 2 | Clause 4.3, first paragraph: "A prefix the [Detector] adds from the invoking directory or the [Rule]'s location is derived from the file path. A namespace the [Rule author] assigns once in configuration is not." (no reason given) | 2 | no | +| 3 | Clause 6.3, the "dangling one" paragraph: "The failure to guard against is not the absent document but the **dangling one**", set against the clause's first paragraph, "Every Identifier a Bundled rule can print MUST resolve, under clause 6.1, to a page in that shipped documentation"; whether the MUST covers absent documentation | 3 | no | +| 4 | Section 2, "These are additional."; what the terms are additional to | 3 | no | +| 5 | Clause 7.1, second paragraph: "That way is open only where the [Detector]'s [Rules] can see the route: a [Detector] whose [Rules] never see comments cannot make an ignore comment detectable this way"; what a bespoke rule can see | 4 | yes | +| 6 | Clause 7.2, first paragraph: "The [Detector] SHOULD NOT supply a default one, including each entry of a generated [Baseline]"; what "one" refers back to | 5 | no | +| 7 | Clause 8.1, second paragraph: "a gap a [Practitioner] finds is recorded under the method's clause 3.2 in any case", set against the same paragraph's "falls only on a [Detector] that has one"; where the record is kept with no declaration | 5 | yes | + +Notes, not passages: none. + +## Findings and where they lie + +- **Finding A, passage 1, clause 6.1 "For a project's own Rule" paragraph** (readers 1, 3 and + 4): the diff 9e914fe..44fdd9e changes the paragraph before it ("In both, ...") and the "For a + Bundled rule" paragraph, but not this one; no reader naming it sets it against a paragraph the + diff changed (reader 1 points at documents not supplied, readers 3 and 4 at the comparison in + "shorter"). Not the change's text. Routed to `acceptance/ROUTED.md`, where the paragraph's row + raised by run 63 is noted as raised again by run 66; does not block. + +Passages 2 to 7 are each named by one reader: recorded, not findings. Passages 5 and 7 lie in +changed paragraphs but are single. + +## Points the rules did not settle + +1. **Finding A, reader 1: a passage named for a reference to a document not supplied.** Reader 1 + quotes two sentences of the paragraph with the clause number, which `ACCEPTANCE.md` defines + as naming a passage, and its confusion is that method clause 3.6 and the toolchain + specification were not given. The rule that a remark that another document was not supplied + is a note covers a remark "not about a passage of the document"; this remark is attached to + quoted passages. Marked as the open-point rule says: an open point on whether a reader named + a passage counts the passage. The finding does not turn on this point, since readers 3 and 4 + name the same paragraph, and the finding is routed either way. + +## Process change + +The run was made under the acceptance process change of pull request 22 before it merged, +following all of it at commit 9907492, named before the readers started. + +## Against the criteria + +- Load-bearing statements: five of five on 2a, 2b and 2c. Met. +- Restatement and application: five of five on question 1 and on every question 3 item. Met. +- A miss the key says counts against the document: none. +- Passages named by two or more readers in text the change wrote or altered: none. Met. +- Passages named by two or more readers elsewhere: one (finding A), routed to + `acceptance/ROUTED.md`. + +## Findings applied + +None at the time of marking; finding A is for `acceptance/ROUTED.md`. + +## Verdict + +PASS. Every keyed item passed its threshold, no miss counts against the document, and the one +finding lies outside the text the change wrote or altered. + +## Answer sheets + +Posted on the pull request: LINK TO BE ADDED. From f3ec69ebb2b2fc10fc592765ba20848acbdcb00a Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:47:43 +0100 Subject: [PATCH 16/34] Routed findings: rows from runs 19 to 30, two closed by defined-terms run 64 acceptance/ROUTED.md gains the passages runs 19 to 30 on the semantic-detectors change named in text that change no longer touches (the Narrowing entry, section 4's lead-in to the owner's decisions, clause 3.1's paragraph on independent techniques and the section 3 table's 3.1 row), notes those runs on the rows they raised again, and closes the method's two-bounds paragraph and bundled-defence paragraph by editor/defined-terms's run 64. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index f995c21..c6c3d28 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -10,7 +10,7 @@ A finding is open until a change that reworks its passage passes a cohort, when that run and moves below the rule, or until the editor closes it with a reason recorded here. A finding raised again by a later run is noted on its line, not listed twice. The status column holds `open`, or the `editor/` branch or pull request number of the change that is -reworking the passage. The rows raised by runs 31 to 52 were raised under the earlier rules, on +reworking the passage. The rows raised by runs 19 to 52 were raised under the earlier rules, on untouched text of the change those runs tested, and are listed here so that they are not lost. ## Open @@ -25,22 +25,28 @@ untouched text of the change those runs tested, and are listed here so that they | `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | | `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | run 67 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56, 64 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | | `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59, 64 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61, 64 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61, 64 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed A row moves here when a change that reworks its passage passes a cohort, naming that run, or when the editor closes it with a reason. -| Document | Clause | Passage | Raised by | Closed by | -| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ------------------------------------ | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule`, run 66 | +| Document | Clause | Passage | Raised by | Closed by | +| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------ | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 20, 22, 24, 30, 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 20, 30, 39, 42 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | runs 20, 54 | `editor/detector-route-rule`, run 66 | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on the two bounds | "Both bounds measure breadth within one level; the level itself is a separate question, answered against the report." | runs 19, 56 | `editor/defined-terms`, run 64 | +| `SPEC.md` | [3.5](../SPEC.md#35-enforce-permanently-and-block), the paragraph on a bundled defence | "For a Bundled defence, the project whose entry point demonstrates it is any project the Toolchain is genuinely installed into", and its substitutes sentence | runs 25, 29, 53, 59 | `editor/defined-terms`, run 64 | From ae5f8dab06df43dd63f0a8b34d1fcf17940974db Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:58:29 +0100 Subject: [PATCH 17/34] Runs 060, 063 and 066: pull request 23 and the posted answer sheets Each record names pull request 23 and links the three comments carrying its evidence. Co-Authored-By: Claude Fable 5.1 --- acceptance/runs/060-detector-route-rule-run-1.md | 7 +++++-- acceptance/runs/063-detector-route-rule-run-2.md | 7 +++++-- acceptance/runs/066-detector-route-rule-run-3.md | 7 +++++-- 3 files changed, 15 insertions(+), 6 deletions(-) diff --git a/acceptance/runs/060-detector-route-rule-run-1.md b/acceptance/runs/060-detector-route-rule-run-1.md index 20317b4..1b11d48 100644 --- a/acceptance/runs/060-detector-route-rule-run-1.md +++ b/acceptance/runs/060-detector-route-rule-run-1.md @@ -1,7 +1,7 @@ # Run 60: DETECTOR-SPEC.md, detector route rule, first run - **Document**: DETECTOR-SPEC.md -- **Pull request**: not yet opened at the time of the run; it is to be opened from branch +- **Pull request**: [23](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23), opened after the run from branch `editor/detector-route-rule`. The readers read commit ed30e69. Change: clause 7.1's detectable route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph on what detectable means; clause 8.1's gap duty falls only on a detector that declares a @@ -135,4 +135,7 @@ FAIL, on findings A and B alone; every keyed item passed its threshold. ## Answer sheets -Posted on the pull request: LINK TO BE ADDED. +Posted on the pull request in three comments: +[the command, models, marker instruction and agent definition](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6124578731), +[the answer sheets](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6124580496) and +[the prompt](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6124582261). diff --git a/acceptance/runs/063-detector-route-rule-run-2.md b/acceptance/runs/063-detector-route-rule-run-2.md index c3fa574..61604ee 100644 --- a/acceptance/runs/063-detector-route-rule-run-2.md +++ b/acceptance/runs/063-detector-route-rule-run-2.md @@ -1,7 +1,7 @@ # Run 63: DETECTOR-SPEC.md, detector route rule, second run - **Document**: DETECTOR-SPEC.md -- **Pull request**: not yet opened at the time of the run; it is to be opened from branch +- **Pull request**: [23](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23), opened after the run from branch `editor/detector-route-rule`. The readers read commit 4cbecad. Change: clause 7.1's detectable route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph on what detectable means; clause 8.1's gap duty falls only on a detector that declares a @@ -141,4 +141,7 @@ FAIL, on finding A alone; every keyed item passed its threshold. ## Answer sheets -Posted on the pull request: LINK TO BE ADDED. +Posted on the pull request in three comments: +[the command, models, marker instruction and agent definition](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6124582736), +[the answer sheets](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6124583990) and +[the prompt](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6124585481). diff --git a/acceptance/runs/066-detector-route-rule-run-3.md b/acceptance/runs/066-detector-route-rule-run-3.md index 0d80045..72cc618 100644 --- a/acceptance/runs/066-detector-route-rule-run-3.md +++ b/acceptance/runs/066-detector-route-rule-run-3.md @@ -1,7 +1,7 @@ # Run 66: DETECTOR-SPEC.md, detector route rule, third run - **Document**: DETECTOR-SPEC.md -- **Pull request**: not yet opened at the time of the run; it is to be opened from branch +- **Pull request**: [23](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23), opened after the run from branch `editor/detector-route-rule`. The readers read commit 44fdd9e. Change: clause 7.1's detectable route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph on what detectable means; clause 8.1's gap duty falls only on a detector that declares a @@ -142,4 +142,7 @@ finding lies outside the text the change wrote or altered. ## Answer sheets -Posted on the pull request: LINK TO BE ADDED. +Posted on the pull request in three comments: +[the command, models, marker instruction and agent definition](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6124587014), +[the answer sheets](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6124587505) and +[the prompt](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6124587875). From 8defdae4aeb9e7feb7c0531ab2d5559d3bb1e195 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:01:19 +0100 Subject: [PATCH 18/34] Detector 5.3 and 8.1: the location owed only where the summary leaves a finding out After run 69, a defined-terms cohort on this document with this change merged in: clause 5.3's obligation owes the location of the remainder only where the summary leaves a finding out, so it no longer pulls against "need not name it"; clause 8.1 says a practitioner's record of a gap lives in their own project's record, and that a detector without a declaration can still be found to conform. Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 14 ++++++++++---- DETECTOR-SPEC.md | 15 ++++++++------- 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9cc1a76..11f482c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -92,10 +92,16 @@ pass it. text, every keyed item five of five: two readers needed two readings of clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s sentence on a report that only adds detail, which is now a sentence of its own with its subject stated; three readers named clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation)'s paragraph on a project's own rule, which this - change does not touch and `acceptance/ROUTED.md` now lists. Accepted by - [run 66](acceptance/runs/066-detector-route-rule-run-3.md), every keyed item five of five; its - one finding is that same clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) paragraph, named again by three readers, which stays in - `acceptance/ROUTED.md` for a change of its own. + change does not touch and `acceptance/ROUTED.md` now lists. + [Run 66](acceptance/runs/066-detector-route-rule-run-3.md) passed, every keyed item five of + five, its one finding being that same clause [6.1](DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) paragraph, named again by three readers, + which stays in `acceptance/ROUTED.md` for a change of its own. Run 69, a cohort of the + defined-terms change on this document with this change merged in, then had two readers set + clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s "MUST carry both a usable summary and the location of the remainder" against its + "need not name it", and two readers stop at clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps)'s gap sentence and its MAY: clause + [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) now owes the location only where the summary leaves a finding out, and clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) says + where a practitioner's record of a gap lives and that a detector without a declaration can + still be found to conform. Accepted by run NNN. ### 1.0.0, 2026-09-08 diff --git a/DETECTOR-SPEC.md b/DETECTOR-SPEC.md index 42369c6..b4e8160 100644 --- a/DETECTOR-SPEC.md +++ b/DETECTOR-SPEC.md @@ -158,8 +158,8 @@ any. ### 5.3 The result MUST reach the practitioner in the output of the command they ran Where the [Detector] writes fuller detail elsewhere, the invoked command's own output MUST -carry both a usable summary and the location of the remainder. The remainder is the findings -the summary leaves out. Where the output prints every finding, each with where it is and its +carry a usable summary and, where the summary leaves any finding out, the location of the +remainder. The remainder is the findings the summary leaves out. Where the output prints every finding, each with where it is and its [Identifier], there is no remainder. A fuller report written elsewhere, such as a page with a code snippet for each finding, only adds detail to findings the output already carries; it is not a remainder, and the output need not name it. @@ -334,11 +334,12 @@ with a non-empty gap record is a statement of where the [Detector] stands and is [Conformance]. The declaration is optional, and so the duty above falls only on a [Detector] that has one; a -gap a [Practitioner] finds is recorded under the method's clause 3.2 in any case. It is how a -maintainer claims [Conformance]; it is not a condition of it. A [Detector] that predates this document, or whose maintainer has never read -it, MAY be graded [Conforming] on evidence: anyone may grade it, by checking it against each -clause of sections 4 to 7, and a verdict on any [Detector], declared or not, rests on that -check and not on the claim. A declaration tells the reader what the maintainer believes and what they know to be missing; +gap a [Practitioner] finds is recorded in their own project's record under the method's clause +3.2 in any case. The declaration is how a maintainer claims [Conformance]; it is not a condition +of it, and a [Detector] without one can still be found to [Conform]. A [Detector] that predates +this document, or whose maintainer has never read it, MAY be graded [Conforming] on evidence: +anyone may grade it, by checking it against each clause of sections 4 to 7, and a verdict on +any [Detector], declared or not, rests on that check and not on the claim. A declaration tells the reader what the maintainer believes and what they know to be missing; the reader still runs the [Harness]. The shape is illustrative rather than prescribed. In a Composer manifest: From bc6ae14586c1a392c1808989114ebc71a46b9d27 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:01:43 +0100 Subject: [PATCH 19/34] Method key: clause 3.3's narrowing sentence; routed findings from runs 68 to 71 The method key lists the sentence behind a narrowing, and the owner's decision where it cannot be written, among the obligations, after three readers of run 70 restated it. acceptance/ROUTED.md gains Appendix A's closing paragraph from run 70 and the detector's clause 5.3 from run 69, and notes runs 68, 70 and 71 on the rows they raised again. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 60 +++++++++++++++++++++------------------- acceptance/method/KEY.md | 2 ++ 2 files changed, 33 insertions(+), 29 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index c6c3d28..eced9c7 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,38 +15,40 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------------------------------- | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | run 67 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------- | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | run 69 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed A row moves here when a change that reworks its passage passes a cohort, naming that run, or when the editor closes it with a reason. -| Document | Clause | Passage | Raised by | Closed by | -| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------ | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 20, 22, 24, 30, 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 20, 30, 39, 42 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | runs 20, 54 | `editor/detector-route-rule`, run 66 | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on the two bounds | "Both bounds measure breadth within one level; the level itself is a separate question, answered against the report." | runs 19, 56 | `editor/defined-terms`, run 64 | -| `SPEC.md` | [3.5](../SPEC.md#35-enforce-permanently-and-block), the paragraph on a bundled defence | "For a Bundled defence, the project whose entry point demonstrates it is any project the Toolchain is genuinely installed into", and its substitutes sentence | runs 25, 29, 53, 59 | `editor/defined-terms`, run 64 | +| Document | Clause | Passage | Raised by | Closed by | +| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------ | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 20, 22, 24, 30, 36, 42, 45, 48, 51, 54, and 71 on a branch without the rework | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 20, 30, 39, 42, and 71 on a branch without the rework | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | runs 20, 54 | `editor/detector-route-rule`, run 66 | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on the two bounds | "Both bounds measure breadth within one level; the level itself is a separate question, answered against the report." | runs 19, 56, and 70 on a branch without the rework | `editor/defined-terms`, run 64 | +| `SPEC.md` | [3.5](../SPEC.md#35-enforce-permanently-and-block), the paragraph on a bundled defence | "For a Bundled defence, the project whose entry point demonstrates it is any project the Toolchain is genuinely installed into", and its substitutes sentence | runs 25, 29, 53, 59 | `editor/defined-terms`, run 64 | diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index 86cf8ff..e2ffd60 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -23,6 +23,8 @@ no bullet making an assertion listed below as wrong, is a correct restatement. practitioner's own authority (section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom)). - Fix the original defect conventionally, with a test that reproduces it, once the defence is in place (sections [2](../../SPEC.md#2-when-the-method-applies) and [6](../../SPEC.md#6-relationship-to-test-driven-development)). +- Write down, for any code the rule is narrowed to exclude, why the hazard cannot arise there, + and take the exclusion to the owner where that sentence cannot be written ([3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire), Part B). Wrong if it asserts: the practitioner fixes the reported instance first and defends afterwards; a test may serve as the detector; an agent may baseline the remaining instances. A bullet that From dbdf841674660faea021eb5cb534c3763fb886dd Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:11:45 +0100 Subject: [PATCH 20/34] Detector 5.3 and 7.1: a usable summary defined, detectability in one paragraph of two ways --- DETECTOR-SPEC.md | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/DETECTOR-SPEC.md b/DETECTOR-SPEC.md index b4e8160..1de2e6e 100644 --- a/DETECTOR-SPEC.md +++ b/DETECTOR-SPEC.md @@ -159,10 +159,10 @@ any. Where the [Detector] writes fuller detail elsewhere, the invoked command's own output MUST carry a usable summary and, where the summary leaves any finding out, the location of the -remainder. The remainder is the findings the summary leaves out. Where the output prints every finding, each with where it is and its -[Identifier], there is no remainder. A fuller report written elsewhere, such as a page with a -code snippet for each finding, only adds detail to findings the output already carries; it is -not a remainder, and the output need not name it. +remainder. A summary is usable when each finding in it carries where it is and its +[Identifier]. The remainder is the findings the summary leaves out: where the output prints +every finding there is none, and a fuller report written elsewhere, such as a page with a code +snippet for each finding, is not one and need not be named. **Why**: method specification clause 8.2. A report the [Practitioner] has to go and find is a report that arrives after the decision it was meant to inform. @@ -281,14 +281,12 @@ detectable, by a bespoke [Rule] the project writes for the [Detector] under clau mechanical check the [Detector] documents. Either lets a project which decides to forbid the route enforce that decision. A route that can be neither switched off nor seen does not [Conform]. -A route is detectable when the project can find every use of it. The first way is a bespoke -[Rule]: the project registers a [Rule] of its own, as clause 4.1 provides for, that matches the -route wherever it is used, and the [Detector] runs it like any other bespoke [Rule]; the -[Detector] ships no such [Rule] itself, and need not. That way is open only where the -[Detector]'s [Rules] can see the route: a [Detector] whose [Rules] never see comments cannot make -an ignore comment detectable this way. The second way is a mechanical check the [Detector] -documents, such as a text search for the route's spelling that the [Detector]'s documentation -gives. +A route is detectable when the project can find every use of it, in either of two ways: by a +bespoke [Rule] the project registers under clause 4.1 that matches the route wherever it is +used, which the [Detector] runs like any other bespoke [Rule] and need not ship itself; or by a +mechanical check the [Detector] documents, such as a text search for the route's spelling. The +first way needs the [Detector]'s [Rules] to be able to match the route's text, so a [Detector] +whose [Rules] never see comments offers only the second for an ignore comment. **Why**: the method specification's position is that [Suppression] is an [Owner] decision under its clause 3.4 and section 4, and an [Owner] cannot decide something they are From 13b497ba3ddc0eba771e3c516bc89e4ffe14cb2d Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:18:48 +0100 Subject: [PATCH 21/34] Toolchain key: the 4.3 row says which case the fixture presents; ROUTED 5.3 row adds run 75 Run 72's open point: three readers graded clause 4.3 on the condition that the ignore comment is Lintomatic's only suppression route, and the key did not say which case the fixture presents. The 4.3 row now says so, and such a grade is the grade for this case. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 2 +- acceptance/toolchain/KEY.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index eced9c7..68a0be9 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -18,7 +18,7 @@ untouched text of the change those runs tested, and are listed here so that they | Document | Clause | Passage | Raised by | Status | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------- | | `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | run 69 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | | `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index aadab7a..e098c90 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -63,7 +63,7 @@ Wrong readings the text invites, which count against the document on one reader: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | None | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap; a reader who notes that the detector specification was not supplied and grades from this document's summary of it is correct | | [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | None. A No given on the condition that the two bundled rules are blocking is a No: their firing stops `zig-qa run`, so they are | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)` | -| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | None | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | +| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | None | The fixture names one suppression route of Lintomatic's, the inline ignore comment, disabled project-wide, with exceptions only through the record the toolchain reads; a grade given on the condition that the comment is Lintomatic's only route is the grade for this case | | [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | None | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | | [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | None | Lintomatic, then the tests, stopping on a Lintomatic failure | | [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | None | `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is the reading that counts the two `(none)` routes, since [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | From f32e54224a351627a5412cfb2cfee9f9942d7aea Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:19:41 +0100 Subject: [PATCH 22/34] Run 073: detector route rule cohort, failed on clauses 5.3 and 7.1 Every keyed item at threshold. Two findings in text the change wrote: clause 5.3's "usable summary" and whether a report written elsewhere is the remainder, and clause 7.1's second paragraph on when a route is detectable. Nothing routed. Co-Authored-By: Claude Fable 5.1 --- .../runs/073-detector-route-rule-run-4.md | 151 ++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 acceptance/runs/073-detector-route-rule-run-4.md diff --git a/acceptance/runs/073-detector-route-rule-run-4.md b/acceptance/runs/073-detector-route-rule-run-4.md new file mode 100644 index 0000000..2f070f2 --- /dev/null +++ b/acceptance/runs/073-detector-route-rule-run-4.md @@ -0,0 +1,151 @@ +# Run 73: DETECTOR-SPEC.md, detector route rule, fourth run + +- **Document**: DETECTOR-SPEC.md +- **Pull request**: [23](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23), from branch + `editor/detector-route-rule`. The readers read commit d67174c. Change: clause 7.1's detectable + route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph + on what detectable means; clause 8.1's gap duty falls only on a detector that declares a + version; clause 5.3 says what the remainder is; clause 6.1 says it asks for the mechanism, a + missing page failing clause 6.3, and that a path in the message is not enough on its own; + section 2's table gains a Message row. Revised after run 69 (a cohort of another change with + this one merged in) in clause 5.3's obligation sentence, the location of the remainder owed + only where the summary leaves a finding out, and in clause 8.1's second paragraph, where a + practitioner's record of a gap lives and that a detector without a declaration can still be + found to conform; so this is a fresh run. +- **Run within the pull request**: 4 (runs 60, 63 and 66 came before). +- **Cohort**: five readers, none reused from any earlier run. Model named for the run before the + readers started: `claude-haiku-5-5`. Every reader reported `claude-haiku-5-5`. +- **Isolation**: each reader a separate `claude -p` process, started through the `ccy-claude` + wrapper (which only reattaches the session's credential) in a fresh empty directory made by + `mktemp -d` outside any repository, with the packet on standard input; no tools, no settings, + no project context, no saved session; the machine applies no managed settings. The editor + attests the rest of the isolation. Command: + `ccy-claude -p --model haiku --safe-mode --tools '' --setting-sources '' --strict-mcp-config --no-session-persistence --output-format json < prompt.txt`. + The prompt (sha256 `0df0b01c02c5d2fd83b37d218f88785980384c3a0ecadce92abf7088211c9249`) is + `QUESTIONS.md`, `DETECTOR-SPEC.md` and `fixtures/lintomatic.md` at d67174c, each after one + `===== FILE: =====` line, one blank line between files; the marker rebuilt it from that + commit and it matched byte for byte. +- **Readers set aside**: none. Every reader's output has `is_error` false, subtype `success`, + non-empty answer text and the cohort's model. +- **Marked by**: a delegated marker, a fresh `dbf-spec-editor` agent (Claude Opus) given the + marking instruction and its agent definition, having seen neither the drafting nor the review. + +## Marks + +Readers are numbered 1 to 5 as their sheets are. Question 3 columns: the eleven clauses, then +the two answers (S8 conformance under section 8, 6.4 whether clause 6.4 changes the verdict). + +| Reader | Q1 | 2a | 2b | 2c | 4.1 | 4.2 | 4.3 | 5.1 | 5.2 | 5.3 | 5.4 | 6.1 | 6.2 | 6.3 | 7.1 | S8 | 6.4 | +| ------ | ------- | --- | --- | --- | --- | ---- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 1 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 2 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 3 | correct | ok | ok | ok | ok | miss | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 4 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 5 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | + +Question 1: correct bullets per reader 6, 6, 6, 6 and 6. Readers 1, 2, 3 and 5 give one bullet +per line of the key's list (4.1; 4.2; 4.3; 5.1 to 5.4; 6.1 to 6.3; 7.1). Reader 4 splits 5.1 to +5.3 from 5.4, which are different clauses of one line and each count, and joins 6.1 to 6.3 with +7.1 in its sixth bullet. Reader 2's fourth bullet says every finding must reach the command's +output, which the key says names 5.3 and is not marked wrong for the plural; its fifth bullet +says every identifier the detector can print must have a page, which names the 6.1 to 6.3 line +and is not among the key's wrong assertions. Reader 5 adds, after its six bullets, a line on +what conformance means; it is not among the first six bullets and states no obligation. No +bullet asserts anything the key marks wrong. + +Question 2: every reader states every required element of each item and quotes the document. +For 2a all five state that conformance is every MUST in sections 4 to 7 and that a detector +predating the document can conform; readers 1 to 4 quote section 8's definition, and all five +quote clause 8.1's "A Detector that predates this document, or whose maintainer has never read +it, MAY be graded Conforming on evidence" (reader 5 also its "a Detector without one can still +be found to Conform"). For 2b all five name both things, disableable by configuration or +detectable, and that either alone is enough, quoting clause 7.1's "For each one it MUST do at +least one of two things, and either alone satisfies this clause". Readers 2, 3 and 5 add that +the bespoke-rule way is closed to a detector whose rules never see comments, the document's own +sentence, which is not among the key's wrong assertions for 2b. For 2c all five state that the +documentation ships with the rule (in the installed copy), at a version tracked together, and +is reachable without network access, quoting the 6.2 and 6.3 headings; reader 4 adds clause +6.2's sentence that a website catalogue is the wrong place, which agrees with the key. + +Question 3: readers 1, 2, 4 and 5 give every accepted grade, each with caveats on 4.2 (the +fixture does not say `--only` prints finding lines) and 4.3 (no machine-readable format +described); the key says a caveat with the accepted grade is correct. S8: all five say +Lintomatic does not conform, naming 6.3. 6.4: all five say it does not change the verdict, +because it is a SHOULD. + +### Misses + +- **Reader 3, clause 4.2: Partial** (accepted: Yes). Reasoning: `--only` runs one rule and exits + 1 when it fires, but the clause requires the harness to report "what it printed" and the + fixture never says `--only` prints the finding line; "it becomes Yes if `--only` output + includes the finding". The reasoning quotes three words of the document, below the six that + `ACCEPTANCE.md` defines as quoting it, so it does not quote the document. The key names no + wrong reading for 4.2. A single miss, the reader's, tolerated by the threshold. Reader 3's + question 4 note puts the close call on the fixture, not the document. + +## Question 4: passages named + +Paragraphs as `ACCEPTANCE.md` defines them; "changed" means the diff 9e914fe..d67174c adds, +rewords, moves or deletes a sentence in the paragraph, or changes a word, term or link in it. + +| # | Passage (clause, words or description) | Readers | Changed by the diff | +| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- | ------------------- | +| 1 | Clause 8.1, first paragraph: "The same declaration is where a gap against this document is recorded once it is known: a [Detector] that declares a version and has learnt… that it fails a MUST in sections 4 to 7 MUST record that gap…"; whether a MUST in section 8 counts toward conformance | 1 | yes | +| 2 | Clause 5.3, first paragraph: "the invoked command's own output MUST carry a usable summary" (2, what makes a summary usable); "The remainder is the findings the summary leaves out." (5, whether a fuller HTML report is a remainder, settled only by the sentences after) | 2, 5 | yes | +| 3 | Clause 7.1, second paragraph: "a [Detector] whose [Rules] never see comments cannot make an ignore comment detectable this way" (2, whether a property of the detector's API or of its rules); "That way is open only where the [Detector]'s [Rules] can see the route: …" (4, whether the route was open to Lintomatic) | 2, 4 | yes | +| 4 | Section 1, "The governing principle" paragraph: "a [Conforming] [Detector] MUST make that possible without the project building the mechanism first"; an unnumbered MUST outside sections 4 to 7, and whether it bears on conformance | 4 | no | +| 5 | Clause 4.2, first paragraph: "a [Detector] failing 4.1 is graded here on its [Bundled rules]", set beside the sentence before it that the clause applies to every rule | 5 | no | + +Notes, not passages: reader 3 wrote "none", adding that its 4.2 close call came from the fixture +not stating what `--only` prints, not from the document. + +## Findings and where they lie + +- **Finding A, passage 2, clause 5.3 first paragraph** (readers 2 and 5): the diff 9e914fe..d67174c + rewords this paragraph's obligation sentence and adds its three following sentences, so it is + text the change wrote or altered. Reader 2's words ("usable summary") are in the part of the + sentence the change kept, but the rule is by paragraph. Blocks. +- **Finding B, passage 3, clause 7.1 second paragraph** (readers 2 and 4): the diff adds this + paragraph. Text the change wrote. Blocks. + +Passages 1, 4 and 5 are each named by one reader: recorded, not findings. Passage 1 lies in a +changed paragraph but is single. Nothing is routed to `acceptance/ROUTED.md` from this run. + +## Points the rules did not settle + +1. **Reader 3's 4.2 grade carries a condition for another grade.** `ACCEPTANCE.md` reads a grade + given on a stated condition as the grade for the case the fixture presents where the key says + which case that is, and otherwise as unanswered; the key says no row carries a condition. + Reader 3 gives Partial as its grade and states the condition only for Yes. Marked as Partial, + the grade given; read either way (Partial, or unanswered) the item is a miss, so the mark does + not turn on the point. + +## Process change + +The run was made under the acceptance process change of pull request 22 before it merged, +following all of it at commit bc6ae14, named before the readers started. + +## Against the criteria + +- Load-bearing statements: five of five on 2a, 2b and 2c. Met. +- Restatement and application: five of five on question 1 and on every question 3 item but + 4.2, which is four of five. Met. +- A miss the key says counts against the document: none. +- Passages named by two or more readers in text the change wrote or altered: two (findings A and + B). Not met. +- Passages named by two or more readers elsewhere: none. + +## Findings applied + +None at the time of marking; findings A and B are on the change's own text, to be fixed there +before a fresh cohort. + +## Verdict + +FAIL. Every keyed item passed its threshold and no miss counts against the document, but +findings A (clause 5.3, first paragraph) and B (clause 7.1, second paragraph) lie in text the +change wrote or altered, and each blocks. + +## Answer sheets + +Posted on the pull request: LINK PLACEHOLDER. From cca75a72e4d7f3887e02cd0b5d8d4c9e12933b91 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:19:41 +0100 Subject: [PATCH 23/34] Run 077: detector route rule cohort, failed on clause 5.3's remainder sentence Every keyed item at threshold, one miss on clause 4.2 tolerated. One finding in text the change wrote: clause 5.3's remainder sentence, four readers unable to tell what "is not one" refers to. Nothing routed. Co-Authored-By: Claude Fable 5.1 --- .../runs/077-detector-route-rule-run-5.md | 154 ++++++++++++++++++ 1 file changed, 154 insertions(+) create mode 100644 acceptance/runs/077-detector-route-rule-run-5.md diff --git a/acceptance/runs/077-detector-route-rule-run-5.md b/acceptance/runs/077-detector-route-rule-run-5.md new file mode 100644 index 0000000..7d1852f --- /dev/null +++ b/acceptance/runs/077-detector-route-rule-run-5.md @@ -0,0 +1,154 @@ +# Run 77: DETECTOR-SPEC.md, detector route rule, fifth run + +- **Document**: DETECTOR-SPEC.md +- **Pull request**: [23](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23), from branch + `editor/detector-route-rule`. The readers read commit dbdf841. Change: clause 7.1's detectable + route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph + on what detectable means; clause 8.1's gap duty falls only on a detector that declares a + version; clause 5.3 says what the remainder is; clause 6.1 says it asks for the mechanism, a + missing page failing clause 6.3, and that a path in the message is not enough on its own; + section 2's table gains a Message row. Revised after run 73 in the two paragraphs its findings + named: clause 5.3's paragraph now defines a usable summary (each finding carries where it is + and its Identifier) and the remainder (the findings the summary leaves out; none where the + output prints every finding; a fuller report written elsewhere is not one and need not be + named), and clause 7.1's second paragraph now states the two ways a route is detectable in one + paragraph (a bespoke Rule registered under clause 4.1, or a mechanical check the Detector + documents); so this is a fresh run. +- **Run within the pull request**: 5 (runs 60, 63, 66 and 73 came before). +- **Cohort**: five readers, none reused from any earlier run. Model named for the run before the + readers started: `claude-haiku-5-5`. Every reader reported `claude-haiku-5-5`. +- **Isolation**: each reader a separate `claude -p` process, started through the `ccy-claude` + wrapper (which only reattaches the session's credential) in a fresh empty directory made by + `mktemp -d` outside any repository, with the packet on standard input; no tools, no settings, + no project context, no saved session; the machine applies no managed settings. The editor + attests the rest of the isolation. Command: + `ccy-claude -p --model haiku --safe-mode --tools '' --setting-sources '' --strict-mcp-config --no-session-persistence --output-format json < prompt.txt`. + The prompt (sha256 `2a96d5b0c6339f5402c9883018d35c0353c42dfad581c94d9b80a8b428882bc9`) is + `QUESTIONS.md`, `DETECTOR-SPEC.md` and `fixtures/lintomatic.md` at dbdf841, each after one + `===== FILE: =====` line, one blank line between files; the marker rebuilt it from that + commit and it matched byte for byte. +- **Readers set aside**: none. Every reader's output has `is_error` false, subtype `success`, + non-empty answer text and the cohort's model. +- **Marked by**: a delegated marker, a fresh `dbf-spec-editor` agent (Claude Opus) given the + marking instruction and its agent definition, having seen neither the drafting nor the review. + +## Marks + +Readers are numbered 1 to 5 as their sheets are. Question 3 columns: the eleven clauses, then +the two answers (S8 conformance under section 8, 6.4 whether clause 6.4 changes the verdict). + +| Reader | Q1 | 2a | 2b | 2c | 4.1 | 4.2 | 4.3 | 5.1 | 5.2 | 5.3 | 5.4 | 6.1 | 6.2 | 6.3 | 7.1 | S8 | 6.4 | +| ------ | ------- | --- | --- | --- | --- | ---- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 1 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 2 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 3 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 4 | correct | ok | ok | ok | ok | miss | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 5 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | + +Question 1: correct bullets among the first six, per reader, 6, 6, 6, 5 and 6. Readers 1 and 5 +give one bullet per line of the key's list. Reader 2 joins 4.1 and 4.2 in its first bullet and +splits 5.1 to 5.3 from 5.4. Reader 3 gives seven bullets; only the first six are marked (4.1, +4.2, 4.3, 5.1 to 5.3, 5.4, 6.1 to 6.3), and its seventh, on 7.1, is not marked. Reader 4 gives +five bullets, each naming a line of the key's list. Readers 1, 2 and 4 say the command's output +carries each finding with its location and identifier, which names 5.3 and is not marked wrong +for the plural. No bullet asserts anything the key marks wrong. + +Question 2: every reader states every required element of each item and quotes the document. +For 2a all five state that conformance is every MUST in sections 4 to 7 and that a detector +predating the document can conform, quoting section 8's definition and clause 8.1's "A Detector +that predates this document, or whose maintainer has never read it, MAY be graded Conforming on +evidence"; readers 1 and 4 also quote "a Detector without one can still be found to Conform". +For 2b all five name both things, disableable by configuration or detectable, and that either +alone is enough, quoting clause 7.1's "For each one it MUST do at least one of two things, and +either alone satisfies this clause" (reader 2 with an ellipsis; its longest verbatim run is well +over six words). Reader 4 adds that a detector whose rules never see comments has only the +documented mechanical check, the document's own sentence, which is not among the key's wrong +assertions for 2b. For 2c all five state that the documentation ships with the rule in the +installed copy, at a version tracked together, and is reachable without network access, +quoting the 6.2 and 6.3 headings; readers 3, 4 and 5 add clause 6.3's "Every Identifier a +Bundled rule can print MUST resolve", which agrees with the key. + +Question 3: readers 1, 2, 3 and 5 give every accepted grade, several with caveats on 4.2 (the +fixture does not say `--only` prints finding lines) and 4.3 (no machine-readable format +described); the key says a caveat with the accepted grade is correct. Readers 1, 3 and 5 state +that they read the two undocumented rules as bundled; reader 3 adds what would follow if they +were project rules, but gives No as its grade without condition. All five grade 5.3 Yes, every +finding being printed so no remainder needs naming, and 6.1 Yes. S8: all five say Lintomatic +does not conform, naming 6.3. 6.4: all five say it does not change the verdict, because it is a +SHOULD. + +### Misses + +- **Reader 4, clause 4.2: Partial** (accepted: Yes). Reasoning: `--only` runs a single rule and + exits 1 when it fires, which covers "whether the Rule fired", but the fixture never says a + single-rule run prints its findings, so "what it printed" is not established. The reasoning + quotes four and three words of clause 4.2, below the six that `ACCEPTANCE.md` defines as + quoting the document, so it does not quote the document; it puts the gap on the fixture's + silence. The key names no wrong reading for 4.2. A single miss, the reader's, tolerated by the + threshold. + +## Question 4: passages named + +Paragraphs as `ACCEPTANCE.md` defines them; "changed" means the diff 9e914fe..dbdf841 adds, +rewords, moves or deletes a sentence in the paragraph, or changes a word, term or link in it. + +| # | Passage (clause, words or description) | Readers | Changed by the diff | +| --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------- | ------------------- | +| 1 | Clause 5.3, first paragraph: "The remainder is the findings the summary leaves out: where the output prints every finding there is none, and a fuller report written elsewhere, such as a page with a code snippet for each finding, is not one and need not be named."; what "is not one" refers to (the remainder or the summary), and the double negative | 2, 3, 4, 5 | yes | +| 2 | Clause 7.1, second paragraph: "The first way needs the [Detector]'s [Rules] to be able to match the route's text, so a [Detector] whose [Rules] never see comments offers only the second for an ignore comment."; how the project's bespoke rule relates to the detector's matching, given "need not ship itself" | 1 | yes | +| 3 | Clause 6.4, Why paragraph: "Where the [Detector] is shipped inside a [Toolchain], the [toolchain specification]'s self-audit makes this check a MUST for the [Toolchain]; a [Detector] released on its own is asked for it as a SHOULD."; whether a detector inside a toolchain keeps the SHOULD | 1 | yes | +| 4 | Clause 6.1, "For a project's own [Rule]" paragraph: "The lookup for such a [Rule] is an obligation on the project's assembled [Toolchain], under clause 4.2 of the [toolchain specification]"; the referenced clause could not be checked | 3 | no | +| 5 | Clause 7.1, Why paragraph: "the method specification's position is that [Suppression] is an [Owner] decision under its clause 3.4 and section 4"; it depends on definitions in SPEC.md | 3 | no | + +Notes, not passages: reader 1's third item is about the fixture's `explain` paragraph and its +last paragraph (whether the two rules printing `No documentation declared` are bundled), not a +passage of the document. Reader 2 notes that defined terms link to SPEC.md, which was not +supplied: a remark about another document. + +## Findings and where they lie + +- **Finding A, passage 1, clause 5.3 first paragraph** (readers 2, 3, 4 and 5): the diff + 9e914fe..dbdf841 rewords this paragraph's obligation sentence and adds the sentences after it, + including the one all four readers quote. Text the change wrote. Blocks. + +Passages 2 to 5 are each named by one reader: recorded, not findings. Passages 2 and 3 lie in +changed paragraphs but are single. Nothing is routed to `acceptance/ROUTED.md` from this run. + +## Points the rules did not settle + +1. **Reader 3's passages 4 and 5.** Each quotes six or more words of the document with its + clause number, which `ACCEPTANCE.md` counts as naming a passage, but the confusion stated is + that SPEC.md or TOOLING-SPEC.md was not supplied, which it records as a note and not a + passage. Under the open-point rule, an open point on whether a reader named a passage counts + the passage, so both are listed as passages. Each is named by reader 3 alone, so the verdict + does not turn on the point. + +## Process change + +The run was made under the acceptance process change of pull request 22 before it merged, +following all of it at commit bc6ae14, named before the readers started. + +## Against the criteria + +- Load-bearing statements: five of five on 2a, 2b and 2c. Met. +- Restatement and application: five of five on question 1 and on every question 3 item but + 4.2, which is four of five. Met. +- A miss the key says counts against the document: none. +- Passages named by two or more readers in text the change wrote or altered: one (finding A). + Not met. +- Passages named by two or more readers elsewhere: none. + +## Findings applied + +None at the time of marking; finding A is on the change's own text, to be fixed there before a +fresh cohort. + +## Verdict + +FAIL. Every keyed item passed its threshold and no miss counts against the document, but +finding A (clause 5.3, first paragraph, the sentence defining the remainder, named by four of +five readers) lies in text the change wrote, and blocks. + +## Answer sheets + +Posted on the pull request: LINK PLACEHOLDER. From c5f17b5e6cfca24aa0dfe12f45fafee38a6e4987 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:19:41 +0100 Subject: [PATCH 24/34] Detector 5.3: the remainder in three plain sentences After run 77: the remainder is the findings the summary leaves out; where the output prints every finding there is no remainder; a fuller report written elsewhere is not a remainder and the output need not name it. The changelog names runs 73 and 77. Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 8 +++++++- DETECTOR-SPEC.md | 6 +++--- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 11f482c..e24f7ea 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -101,7 +101,13 @@ pass it. "need not name it", and two readers stop at clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps)'s gap sentence and its MAY: clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) now owes the location only where the summary leaves a finding out, and clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) says where a practitioner's record of a gap lives and that a detector without a declaration can - still be found to conform. Accepted by run NNN. + still be found to conform. [Run 73](acceptance/runs/073-detector-route-rule-run-4.md) failed + on clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s "usable summary" and on clause [7.1](DETECTOR-SPEC.md#71-every-route-that-silences-a-finding-must-be-detectable)'s second paragraph; clause + [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) now says what a usable summary is and what the remainder is, and clause [7.1](DETECTOR-SPEC.md#71-every-route-that-silences-a-finding-must-be-detectable) states + the two ways a route is detectable in one paragraph. [Run + 77](acceptance/runs/077-detector-route-rule-run-5.md) failed on the remainder sentence alone, + four readers unable to tell what "is not one" referred to; it is now three plain sentences. + Accepted by run NNN. ### 1.0.0, 2026-09-08 diff --git a/DETECTOR-SPEC.md b/DETECTOR-SPEC.md index 1de2e6e..d9823de 100644 --- a/DETECTOR-SPEC.md +++ b/DETECTOR-SPEC.md @@ -160,9 +160,9 @@ any. Where the [Detector] writes fuller detail elsewhere, the invoked command's own output MUST carry a usable summary and, where the summary leaves any finding out, the location of the remainder. A summary is usable when each finding in it carries where it is and its -[Identifier]. The remainder is the findings the summary leaves out: where the output prints -every finding there is none, and a fuller report written elsewhere, such as a page with a code -snippet for each finding, is not one and need not be named. +[Identifier]. The remainder is the findings the summary leaves out. Where the output prints +every finding, there is no remainder. A fuller report written elsewhere, such as a page with a +code snippet for each finding, is not a remainder, and the output need not name it. **Why**: method specification clause 8.2. A report the [Practitioner] has to go and find is a report that arrives after the decision it was meant to inform. From ac149963ff6916d0c6bbb34d6a5678400216082f Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:20:49 +0100 Subject: [PATCH 25/34] Changelog: clause links point at the headings as they are Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e24f7ea..20c5805 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -102,8 +102,8 @@ pass it. [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) now owes the location only where the summary leaves a finding out, and clause [8.1](DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) says where a practitioner's record of a gap lives and that a detector without a declaration can still be found to conform. [Run 73](acceptance/runs/073-detector-route-rule-run-4.md) failed - on clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s "usable summary" and on clause [7.1](DETECTOR-SPEC.md#71-every-route-that-silences-a-finding-must-be-detectable)'s second paragraph; clause - [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) now says what a usable summary is and what the remainder is, and clause [7.1](DETECTOR-SPEC.md#71-every-route-that-silences-a-finding-must-be-detectable) states + on clause [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran)'s "usable summary" and on clause [7.1](DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable)'s second paragraph; clause + [5.3](DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) now says what a usable summary is and what the remainder is, and clause [7.1](DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) states the two ways a route is detectable in one paragraph. [Run 77](acceptance/runs/077-detector-route-rule-run-5.md) failed on the remainder sentence alone, four readers unable to tell what "is not one" referred to; it is now three plain sentences. From 08b9e5658bbfa022a59cede83305d00513c7d401 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:33:37 +0100 Subject: [PATCH 26/34] Run 078: detector route rule cohort passed Every keyed item at threshold, one miss on clause 4.2 tolerated. No finding in text the change wrote; clause 4.2's sentence on a detector failing 4.1, named by two readers, routed to acceptance/ROUTED.md. The changelog names the run. Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 2 +- .../runs/078-detector-route-rule-run-6.md | 177 ++++++++++++++++++ 2 files changed, 178 insertions(+), 1 deletion(-) create mode 100644 acceptance/runs/078-detector-route-rule-run-6.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 20c5805..0325039 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -107,7 +107,7 @@ pass it. the two ways a route is detectable in one paragraph. [Run 77](acceptance/runs/077-detector-route-rule-run-5.md) failed on the remainder sentence alone, four readers unable to tell what "is not one" referred to; it is now three plain sentences. - Accepted by run NNN. + Accepted by [run 78](acceptance/runs/078-detector-route-rule-run-6.md). ### 1.0.0, 2026-09-08 diff --git a/acceptance/runs/078-detector-route-rule-run-6.md b/acceptance/runs/078-detector-route-rule-run-6.md new file mode 100644 index 0000000..a2ebfda --- /dev/null +++ b/acceptance/runs/078-detector-route-rule-run-6.md @@ -0,0 +1,177 @@ +# Run 78: DETECTOR-SPEC.md, detector route rule, sixth run + +- **Document**: DETECTOR-SPEC.md +- **Pull request**: [23](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23), from branch + `editor/detector-route-rule`. The readers read commit ac14996. Change: clause 7.1's detectable + route is a bespoke rule the project writes for the detector under clause 4.1, with a paragraph + on what detectable means; clause 8.1's gap duty falls only on a detector that declares a + version; clause 5.3 says what the remainder is; clause 6.1 says it asks for the mechanism, a + missing page failing clause 6.3, and that a path in the message is not enough on its own; + section 2's table gains a Message row. Revised after run 77 in the one paragraph its finding + named: clause 5.3's remainder sentence is now three plain sentences (the remainder is the + findings the summary leaves out; where the output prints every finding there is no remainder; + a fuller report written elsewhere is not a remainder and the output need not name it); so this + is a fresh run. +- **Run within the pull request**: 6 (runs 60, 63, 66, 73 and 77 came before). +- **Cohort**: five readers, none reused from any earlier run. Model named for the run before the + readers started: `claude-haiku-5-5`. Every reader reported `claude-haiku-5-5`. +- **Isolation**: each reader a separate `claude -p` process, started through the `ccy-claude` + wrapper (which only reattaches the session's credential) in a fresh empty directory made by + `mktemp -d` outside any repository, with the packet on standard input; no tools, no settings, + no project context, no saved session; the machine applies no managed settings. The editor + attests the rest of the isolation. Command: + `ccy-claude -p --model haiku --safe-mode --tools '' --setting-sources '' --strict-mcp-config --no-session-persistence --output-format json < prompt.txt`. + The prompt (sha256 `9d52a2f0c2c15cb1f4d1b7d628fc403560534fa35cdde941e26efe717846f35c`) is + `QUESTIONS.md`, `DETECTOR-SPEC.md` and `fixtures/lintomatic.md` at ac14996, each after one + `===== FILE: =====` line, one blank line between files; the marker rebuilt it from that + commit and it matched byte for byte. +- **Readers set aside**: none. Every reader's output has `is_error` false, subtype `success`, + non-empty answer text and the cohort's model. +- **Marked by**: a delegated marker, a fresh `dbf-spec-editor` agent (Claude Opus) given the + marking instruction and its agent definition, having seen neither the drafting nor the review. + +## Marks + +Readers are numbered 1 to 5 as their sheets are. Question 3 columns: the eleven clauses, then +the two answers (S8 conformance under section 8, 6.4 whether clause 6.4 changes the verdict). + +| Reader | Q1 | 2a | 2b | 2c | 4.1 | 4.2 | 4.3 | 5.1 | 5.2 | 5.3 | 5.4 | 6.1 | 6.2 | 6.3 | 7.1 | S8 | 6.4 | +| ------ | ------- | --- | --- | --- | --- | ---- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| 1 | correct | ok | ok | ok | ok | miss | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 2 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 3 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 4 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | +| 5 | correct | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | ok | + +Question 1: every reader has six correct bullets among the first six, one per line of the key's +list (4.1, 4.2, 4.3, 5.1 to 5.4, 6.1 to 6.3, 7.1). Reader 1 gives seven bullets; its seventh, on +section 8, is not marked. Readers 3 and 5 also restate SHOULDs (4.4, 6.4, 7.2) as SHOULDs, and +reader 4's sixth bullet adds what conformance means; the key says neither is correct nor wrong. +Reader 4 says the output carries a pointer to any findings the summary leaves out, which names +5.3. No bullet asserts anything the key marks wrong. + +Question 2: every reader states every required element of each item and quotes the document. +For 2a all five state that conformance is every MUST in sections 4 to 7 and that a detector +predating the document can conform, quoting clause 8.1's "A Detector that predates this +document, or whose maintainer has never read it, MAY be graded Conforming on evidence"; readers +1, 3 and 4 also quote section 8's definition, reader 2 its "satisfies every MUST in sections 4 +to 7". For 2b all five name both things, disableable by configuration or detectable, and that +either alone is enough, quoting clause 7.1's "For each one it MUST do at least one of two +things, and either alone satisfies this clause"; readers 2, 4 and 5 add that 7.2's reason is a +SHOULD, and reader 3 that a detector whose rules never see comments has only the mechanical +check, the document's own sentence; neither is among the key's wrong assertions. For 2c all five +state that the documentation ships with the rule (reader 2 "inside the installed rule package"), +at a version tracked together or the same version, and reachable without network access, +quoting clause 6.2's heading and clause 6.3's heading or its "Every Identifier a Bundled rule can +print MUST resolve" sentence. None names a website as acceptable; reader 4 quotes 6.2's Why +saying it is not. + +Question 3: readers 2 to 5 give every accepted grade; readers 2, 3 and 4 add caveats on 4.2 (the +fixture does not say `--only` prints finding lines) and 4.3 (no machine-readable format +described), and the key says a caveat with the accepted grade is correct. Reader 3 states that +it read the two undocumented rules as bundled and gives No for 6.3 without condition. All five +grade 5.3 Yes because every finding is printed so there is no remainder, and 6.1 Yes; readers 2, +3 and 4 say the two missing pages fail 6.3, not 6.1. No reader grades 6.3 Partial or 7.1 No. S8: +all five say Lintomatic does not conform, naming 6.3 (reader 1 adds its own 4.2 Partial as a +second reason). 6.4: all five say it does not change the verdict, because it is a SHOULD. + +### Misses + +- **Reader 1, clause 4.2: Partial** (accepted: Yes). Reasoning: `--only` exits 1 when the rule + fires, so whether it fired is reported, but the fixture never says `--only` prints what the + rule printed, which 4.2 requires. The quotation in the reasoning is from the fixture ("exits 1 + when the rule fires, 0 when it does not"); clause 4.2 is paraphrased, not quoted, so the + reasoning does not quote the document. It puts the gap on the fixture's silence. The key names + no wrong reading for 4.2. A single miss, the reader's, tolerated by the threshold. + +## Question 4: passages named + +Paragraphs as `ACCEPTANCE.md` defines them; "changed" means the diff 9e914fe..ac14996 adds, +rewords, moves or deletes a sentence in the paragraph, or changes a word, term or link in it. +That diff changes section 2's Message row, clause 5.3's first paragraph, clause 6.1's "In both" +and "For a Bundled rule" paragraphs, clause 6.4's Why paragraph, clause 7.1's first and second +paragraphs, and clause 8.1's first and second paragraphs. + +| # | Passage (clause, words or description) | Readers | Changed by the diff | +| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ------------------- | +| 1 | Clause 4.2, first paragraph: "a [Detector] failing 4.1 is graded here on its [Bundled rules]."; reader 2: the sentence's purpose unclear until read as applying 4.2 to bundled rules alone | 1, 2 | no | +| 2 | Clause 6.1, the table row for a project's own Rule: "Print the [Identifier] unaltered; the lookup is the [Toolchain]'s to give", set against the clause heading's "MUST provide a mechanism that resolves a printed identifier to its documentation" | 1 | no | +| 3 | Clause 6.1, the heading, which reader 1 sets against passage 2; counted as named by reader 1 under the open-point rule | 1 | no | +| 4 | Clause 8.1, the JSON example: `"method": "1.0.1"`, against the header's method specification version 1.1.0; what the field means | 2 | no | +| 5 | Clause 8.1, second paragraph: "A [Detector] that predates this document, or whose maintainer has never read it, MAY be graded [Conforming] on evidence: anyone may grade it, ... rests on that check and not on the claim."; how every detector is graded, under a heading about the optional declaration | 3 | yes | +| 6 | Clause 4.3, first paragraph: "The [Detector] MUST print it, unaltered, alongside every finding the [Rule] reports, in its default output and in every machine-readable format it offers."; whether the HTML report is a machine-readable format, set against 5.3's "fuller report" | 5 | no | +| 7 | Clause 5.3, first paragraph, which reader 5 cites ("5.3 calls the HTML a 'fuller report'") in passage 6; counted as named by reader 5 under the open-point rule | 5 | yes | + +Notes, not passages: reader 2's items on clause 6.1 ("Where the [Identifier] is itself a URL, as +method clause 3.6 allows": the method specification was not provided, so the cross-reference +could not be checked) and on clause 7.1's Why paragraph ([Owner], [Class] and [Hazard] are +defined only in the method specification, which was not given); reader 4's single item, the same +6.1 sentence and the references to method clauses 3.3 (4.2's Why) and 3.2 (8.1), with *Class* +and *Owner*, all because the method specification was not provided. Each is a remark that +another document was not supplied, which `ACCEPTANCE.md` records as a note and not a passage; +see the first point below. + +## Findings and where they lie + +- **Finding A, passage 1, clause 4.2 first paragraph** (readers 1 and 2): the diff + 9e914fe..ac14996 does not touch clause 4.2, and neither reader sets the paragraph against a + paragraph the change wrote or altered. A finding against the document, not the change: to be + recorded in `acceptance/ROUTED.md` as a new open row and routed to a change of its own. Does + not block. + +Passages 2 to 7 are each named by one reader: recorded, not findings. Passages 5 and 7 lie in +changed paragraphs but are single. + +## Points the rules did not settle + +1. **Remarks that the method specification was not supplied, made by quoting a sentence.** + Readers 2 and 4 each quote six or more words of clause 6.1's "For a project's own Rule" + paragraph, with the clause number, and say only that the method specification was not + provided, so the cross-reference could not be checked. `ACCEPTANCE.md` both says a quotation + with a clause number names a passage and says a remark "such as that another document was not + supplied" is a note and not a passage. Run 77 recorded this as an open point and counted the + passages; `ACCEPTANCE.md` and the key have not changed on it since, so it was not settled + before this run as "Changing the test" asks. This marker holds that the rule settles it: the + stated confusion in each item is exactly the rule's own example of a note, and the quotation + only locates the reference whose target was not supplied; so these are notes, and no open + point arises. The verdict turns on this. Were it held open and the passages counted, readers 2 + and 4 would form a finding on clause 6.1's "For a project's own Rule" paragraph, unchanged + and routed (it is already open in `acceptance/ROUTED.md`, runs 28, 63, 66); and reader 4's + "8.1's 'the method's clause 3.2'", whose words occur in both of clause 8.1's paragraphs, would + leave open whether reader 4 and reader 3 (passage 5) named the same changed paragraph, which + the open-point rule counts, so that finding would block. The editor settles the point in + `ACCEPTANCE.md` before any further run. +2. **Reader 1's passage on clause 6.1** sets the table row against the heading; whether it names + one passage or both is not settled, so both are counted (passages 2 and 3). Reader 5's + citation of clause 5.3 is counted the same way (passage 7). No other reader names any of + these paragraphs, so the verdict does not turn on either. + +## Process change + +The run was made under the acceptance process change of pull request 22 before it merged, +following all of it at commit 13b497b, named before the readers started. + +## Against the criteria + +- Load-bearing statements: five of five on 2a, 2b and 2c. Met. +- Restatement and application: five of five on question 1 and on every question 3 item but + 4.2, which is four of five. Met. +- A miss the key says counts against the document: none. +- Passages named by two or more readers in text the change wrote or altered: none, on the + marking of point 1. Met. +- Passages named by two or more readers elsewhere: one (finding A, clause 4.2), routed to + `acceptance/ROUTED.md`; does not block. + +## Findings applied + +None at the time of marking; finding A is to be added to `acceptance/ROUTED.md` by the editor. + +## Verdict + +PASS, on the marking of point 1. Every keyed item meets its threshold, no miss counts against +the document, and the one finding, clause 4.2's first paragraph, lies outside the text the change +wrote and is routed. + +## Answer sheets + +Posted on the pull request: LINK PLACEHOLDER. From 389ef3f967fe910150b14f77623c1150423210a5 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:35:36 +0100 Subject: [PATCH 27/34] Acceptance: a missing-document remark is a note however it quotes; a parted paragraph is the change's text Settles the open points of runs 76, 77, 78 and 80. ACCEPTANCE.md says a remark that another document was not supplied, or that a term is defined only there, is a note however much of the document it quotes to locate the reference; and that a paragraph the pull request parts from what it refers to, by inserting a paragraph between them or moving either, is the change's text. The Lintomatic fixture says `check --only` prints its findings as `check` does, and the detector key's 4.2 row says so and which case a conditional grade is for. ROUTED gains the toolchain 6.1 Project record sentence (run 76) and the detector 4.2 sentence on a detector failing 4.1 (run 78, taken by editor/defined-terms); the 4.3 row links the section, whose heading the toolchain change rewrites. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 10 ++++--- acceptance/ROUTED.md | 48 ++++++++++++++++--------------- acceptance/detector/KEY.md | 2 +- acceptance/fixtures/lintomatic.md | 3 +- 4 files changed, 34 insertions(+), 29 deletions(-) diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index d1bd6af..77eac8f 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -134,15 +134,17 @@ rule. gives a clause number with a description or a shorter quotation that identifies a sentence or paragraph. A quotation whose words occur in more than one paragraph, given with no clause number, names the paragraph where they first occur. A remark that is not about a passage of - the document, such as that another document was not supplied, is recorded as a note and is - not a passage. Two readers name the same passage when the words or sentences they name lie in + the document, such as that another document was not supplied or that a term is defined only + in another document, is recorded as a note and is not a passage, however much of the document + it quotes to locate the reference. Two readers name the same passage when the words or sentences they name lie in the same paragraph; a heading, a list item, a table row and a terminology entry are each a paragraph of their own. A passage named by two or more readers of the run is a finding. - **Where a finding lies.** The record says, for each finding, whether it lies in text the change wrote or altered: a paragraph in which the pull request, read against its merge base with the branch it targets, adds, rewords, moves or deletes a sentence, or changes a word, a term or a - link; or a paragraph that any reader naming the finding sets against such a paragraph. A - finding there blocks the change. A finding anywhere else is a finding against the document, not + link; a paragraph the pull request parts from what it refers to, by inserting a paragraph + between them or moving either; or a paragraph that any reader naming the finding sets against + such a paragraph. A finding there blocks the change. A finding anywhere else is a finding against the document, not against the change: it is recorded in [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not block. The change is judged on what it changed; the document is proved by the keyed questions, which every run answers over the whole of it. diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 68a0be9..f322fc5 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,29 +15,31 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------- | -| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind | run 76 | `open` | +| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index 5b4d4aa..dd559a8 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -64,7 +64,7 @@ Wrong readings the text invites, which count against the document on one reader: | Clause | Accepted grade | What the fixture supports (no row carries a condition on the reasoning) | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it) | Yes | Project rules via the `Rule` interface, first class in every report | -| [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, with a distinguishing exit code | +| [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, prints its findings as `check` does, and exits with a distinguishing code; the fixture says all three, so a grade given on the condition that `--only` prints as `check` does is the grade for this case | | [4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding) | Yes | `LM-NNNN` fixed at first release; project names taken as given, not rewritten; printed on every finding line | | [5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) | Yes | Local, no account, no server, no network | | [5.2](../../DETECTOR-SPEC.md#52-the-detector-must-support-invocation-over-a-subset-at-minimum-a-single-file) | Yes | `check ` accepts a single file | diff --git a/acceptance/fixtures/lintomatic.md b/acceptance/fixtures/lintomatic.md index f168d56..9d334fc 100644 --- a/acceptance/fixtures/lintomatic.md +++ b/acceptance/fixtures/lintomatic.md @@ -21,7 +21,8 @@ src/db.zig:88:3 proj.no-raw-sql Query built by string concatenation Fuller detail, including a snippet, is written to `.lintomatic/report.html`. The one-line form is printed whether or not the HTML report is enabled. -`lintomatic check --only ` runs a single rule. It accepts any rule name, +`lintomatic check --only ` runs a single rule and prints its findings as +`check` does. It accepts any rule name, bundled or project, and exits 1 when the rule fires, 0 when it does not, and 2 when the rule name is unknown. From 1f0c4f84da0d50cb87a89dc7f770ab1b36f5dfb1 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:36:14 +0100 Subject: [PATCH 28/34] Run records: links to the answer sheets posted on the pull request Co-Authored-By: Claude Fable 5.1 --- acceptance/runs/073-detector-route-rule-run-4.md | 2 +- acceptance/runs/077-detector-route-rule-run-5.md | 2 +- acceptance/runs/078-detector-route-rule-run-6.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/acceptance/runs/073-detector-route-rule-run-4.md b/acceptance/runs/073-detector-route-rule-run-4.md index 2f070f2..3d9006b 100644 --- a/acceptance/runs/073-detector-route-rule-run-4.md +++ b/acceptance/runs/073-detector-route-rule-run-4.md @@ -148,4 +148,4 @@ change wrote or altered, and each blocks. ## Answer sheets -Posted on the pull request: LINK PLACEHOLDER. +Posted on the pull request: [answer sheets](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6125141514); [command, models, marker instruction and agent definition](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6125141043); [the prompt](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6125141961). diff --git a/acceptance/runs/077-detector-route-rule-run-5.md b/acceptance/runs/077-detector-route-rule-run-5.md index 7d1852f..796f44c 100644 --- a/acceptance/runs/077-detector-route-rule-run-5.md +++ b/acceptance/runs/077-detector-route-rule-run-5.md @@ -151,4 +151,4 @@ five readers) lies in text the change wrote, and blocks. ## Answer sheets -Posted on the pull request: LINK PLACEHOLDER. +Posted on the pull request: [answer sheets](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6125143108); [command, models, marker instruction and agent definition](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6125142546); [the prompt](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6125143449). diff --git a/acceptance/runs/078-detector-route-rule-run-6.md b/acceptance/runs/078-detector-route-rule-run-6.md index a2ebfda..463a283 100644 --- a/acceptance/runs/078-detector-route-rule-run-6.md +++ b/acceptance/runs/078-detector-route-rule-run-6.md @@ -174,4 +174,4 @@ wrote and is routed. ## Answer sheets -Posted on the pull request: LINK PLACEHOLDER. +Posted on the pull request: [answer sheets](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6125343069); [command, models, marker instruction and agent definition](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6125342588); [the prompt](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/23#issuecomment-6125343418). From 0da928d9830d12101122821650daacd4e050a6b7 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:43:19 +0100 Subject: [PATCH 29/34] ROUTED: run 81 names the toolchain 6.1 Project record sentence again; 6.2 and 6.3 speak only of exceptions Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 51 ++++++++++++++++++++++---------------------- 1 file changed, 26 insertions(+), 25 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index f322fc5..fe7d05c 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,31 +15,32 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | -| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind | run 76 | `open` | -| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | -| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | +| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81 | run 76 | `open` | +| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed From ae6faf2b34d4f8e21d146fd62cef9445102d00fc Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:48:34 +0100 Subject: [PATCH 30/34] ROUTED: run 82 names the toolchain 6.2 last paragraph again Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index fe7d05c..c16596a 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -29,7 +29,7 @@ untouched text of the change those runs tested, and are listed here so that they | `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | | `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | | `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | | `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | From a095fa7ad823b41b0abe3f932d8484dee9a3fd10 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:53:10 +0100 Subject: [PATCH 31/34] ROUTED: toolchain 6.2's first paragraph against clause 6.4's defaults, from run 83 Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 1 + 1 file changed, 1 insertion(+) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index c16596a..5ae6e16 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -17,6 +17,7 @@ untouched text of the change those runs tested, and are listed here so that they | Document | Clause | Passage | Raised by | Status | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults | run 83 | `open` | | `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | | `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81 | run 76 | `open` | | `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | From 852dcd80f61a418609cd36375ca7631d3648a92e Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:58:23 +0100 Subject: [PATCH 32/34] ROUTED: run 84 names the toolchain Project record entry against clauses 6.1, 6.2 and 6.4 Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 54 ++++++++++++++++++++++---------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 5ae6e16..1423a48 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,33 +15,33 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults | run 83 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | -| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81 | run 76 | `open` | -| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | -| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults | run 83 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | +| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81, and by two readers of run 84 as the Project record entry against clauses 6.1, 6.2 and 6.4 | run 76 | `open` | +| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed From 568e323782847e3f03baeded0303a0979d7d41da Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 15:02:37 +0100 Subject: [PATCH 33/34] ROUTED: run 85 names the toolchain 6.2 first paragraph against clause 6.4 again Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 1423a48..fe7182a 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -17,7 +17,7 @@ untouched text of the change those runs tested, and are listed here so that they | Document | Clause | Passage | Raised by | Status | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- | -------------------------------------------------- | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults | run 83 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults; named again by two readers of run 85 | run 83 | `open` | | `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | | `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81, and by two readers of run 84 as the Project record entry against clauses 6.1, 6.2 and 6.4 | run 76 | `open` | | `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | From 40f7895867706c8bfdc7f5cb27a9ff1507ba2a8c Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 15:08:03 +0100 Subject: [PATCH 34/34] ROUTED: run 86's findings in untouched toolchain text, clause 6.4 against 6.2 new Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 55 ++++++++++++++++++++++---------------------- 1 file changed, 28 insertions(+), 27 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index fe7182a..e53b671 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,33 +15,34 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- | -------------------------------------------------- | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults; named again by two readers of run 85 | run 83 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | -| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81, and by two readers of run 84 as the Project record entry against clauses 6.1, 6.2 and 6.4 | run 76 | `open` | -| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | -| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project), the heading and first paragraph | "SHOULD state its own defaults for anything the method leaves to the project" against clause [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one)'s "MUST NOT supply a default" for a justification | run 86 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults; named again by two readers of run 85 and two of run 86 | run 83 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | +| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81, and by two readers of run 84 as the Project record entry against clauses 6.1, 6.2 and 6.4; and by two readers of run 86, the entry and clause 6.1's first paragraph together | run 76 | `open` | +| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed