From 4383fb8a45812f01f141dc695445ce74f8bead6d Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 11:34:10 +0100 Subject: [PATCH 01/21] Acceptance process: the test proves the clarity of the change, with keys that ask what the question asks Keys list the required elements per part of each question, questions ask for exactly those, a confusion finding is settled run by run and blocks only in text the change wrote or altered (others go to acceptance/ROUTED.md), single misses count against the document only where the key names the reading, and open marking points are settled in the rules before the next run rather than invented per run. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 252 ++++++++++++++++++++++-------- CONTRIBUTING.md | 32 ++-- acceptance/CHANGES.md | 36 +++++ acceptance/README.md | 51 ++++-- acceptance/ROUTED.md | 32 ++++ acceptance/detector/KEY.md | 129 +++++++++------ acceptance/detector/QUESTIONS.md | 9 +- acceptance/method/KEY.md | 107 +++++++------ acceptance/method/QUESTIONS.md | 3 +- acceptance/runs/README.md | 25 +-- acceptance/toolchain/KEY.md | 116 ++++++++------ acceptance/toolchain/QUESTIONS.md | 3 +- 12 files changed, 546 insertions(+), 249 deletions(-) create mode 100644 acceptance/CHANGES.md create mode 100644 acceptance/ROUTED.md diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index 31c8df0..84ab39b 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -6,49 +6,76 @@ document that only a strong reader can follow is not a specification, and the re matter most, coding agents meeting the method in a failing pipeline, are often the weakest. The test is run with a low-strength model so that clarity is measured at the floor, not the ceiling. +The test proves one thing: that a capable reader, reading the document cold and once, takes from +it what the document exists to say, and is not confused by the text the change put there. The +keyed questions prove the document; the confusion question proves the change. Everything below +serves that, and a rule that stops it proving that is a defect in this file, fixed as +[Changes to this process](#changes-to-this-process) describes. + ## The cohort -- **At least five independent readers**, each a fresh instance of a low-strength model (the - smallest current model in its family, for example Haiku), with no shared context and no - access to the change history, the critique findings or each other. -- Each reader receives **only the changed document or documents** and the fixed question set - below. Nothing else is read first. +- **An odd number of independent readers, at least five**, each a fresh instance of a + low-strength model (the smallest current model in its family, for example Haiku), with no + shared context and no access to the change history, the critique findings or each other. + +- **Each reader is a separate process with no tools and no project context, only its packet**: + started from an empty directory outside any repository, with no user, project or local settings + files, no `CLAUDE.md`, skills, plugins, hooks or MCP servers, and no saved session, so that no + project instruction reaches it. The model tool's own fixed system prompt is not project + context. For example, with the packet on standard input: + + ``` + claude -p --safe-mode --model haiku --tools "" --setting-sources "" --strict-mcp-config --no-session-persistence --output-format json < packet.md + ``` + + A reader is set aside, and its answers do not count, when its answer text is empty, when its + process reports an error, or when the model version it reports is not the cohort's model. A run + with a reader set aside is void: it takes a run number and has a record, and a fresh cohort + runs. No reader is set aside for any other cause. + +- Each reader receives **only its packet**: the document's question set, the changed document + and the fixture the questions name, each in full in the reader's prompt, the question set + first. Each file is preceded by exactly one line, `===== FILE: =====`, where `` is + the name the question set gives the file (so `fixtures/remediation-ledger.md`, and + `QUESTIONS.md` for the question set itself). The question set carries the instruction to the + reader, and nothing else is added. Every reader in a run receives the same packet. + - Readers are run **after** every stronger review has been applied. They are the last gate, not the first draft. ## The question set -Written before the run, with an **answer key** for every keyed question drawn verbatim from the -document, so that marking is mechanical. Each reader answers all of them. +Written before the run, with an **answer key** for every keyed question, so that marking is +mechanical. The key states, for each keyed item, exactly what an answer must contain and what +makes it wrong; the question asks for exactly that and nothing the key does not expect. A key +asks for nothing the question does not. Each reader answers all of them. 1. **Restatement.** In their own words, what the document asks of its subject, in at most six bullets. Keyed against the list of MUST clauses: a bullet is correct when it names an obligation the document actually states. -2. **Load-bearing statements.** Three questions, fixed per document, each asking for one - sentence and a quotation: +2. **Load-bearing statements.** Three or more questions, fixed per document, each asking for a + short answer and the sentence or sentences of the document it rests on, quoted. Each has one + or more parts, and the key lists the element an answer must state for each part: - `SPEC.md`: what a practitioner does before fixing a defect, in order; who may baseline or - suppress; what "conforms" means for a remediation. + suppress, and whether an agent may; what "conforms" means for a remediation and what a + verdict rests on. - `DETECTOR-SPEC.md`: what "conforms" means and whether a tool that predates the document can - conform; what a detector must do about an inline suppression route; where a bundled rule's - documentation must live. - - `TOOLING-SPEC.md`: where conformance is measured; how a grade is assigned when parts come - from different sources; how many grades a project that ships tooling has. + conform; what a detector must do about an inline suppression route and whether one of those + things is enough; where a bundled rule's documentation must live, at what version, and + whether it must be reachable offline. + - `TOOLING-SPEC.md`: where conformance is measured and what the toolchain is; how a grade is + assigned when parts come from different sources; how many grades a project that ships + tooling has and whether one implies the other. 3. **Application.** A fixture case supplied with the questions, graded clause by clause with - Yes, Partial or No and one line of reasoning. Keyed against the intended verdicts; a divergence - counts against the document only when the reader's reasoning shows the clause was read - differently, not when the fixture was misread. + Yes, Partial or No and one line of reasoning. Keyed against the intended verdicts, with the + readings the key names as ones the text invites counting against the document. 4. **Confusion.** Any passage the reader found confusing on its one reading, quoted with its clause number, and nothing else: not what would make it clearer, not a proposed change. -The cohort settles one thing: is the document clear, coherent and comprehensive. A reader getting -everything right after one reading proves it; a reader getting confused fails it. The document -is designed by its editor to resolve the comprehension failures the cohort shows; the cohort is -the acceptance layer, not the designer. - ## What the cohort is not asked The cohort may point at what it found confusing. It is never asked to rate the document, to say @@ -58,61 +85,162 @@ not there to design the text, and a document shaped by what its weakest reader s a document designed by that reader. The signals this test takes from the cohort are a keyed answer missed, a wrong verdict reasoned from the document, and a passage pointed at; what to do about any of them is the editor's decision. An earlier version of this test asked the reader -what would make a clause clearer and for a clarity rating; both were removed on the author's +what would make a clause clearer and for a clarity rating; both were removed on the editor's instruction for that reason, and neither is reinstated. +## Marking + +The editor marks each answer sheet against the document's key, or delegates the marking to a +separate marker that has seen neither the drafting nor the review. A delegated marker receives +the document, its question set, fixture and key at the commit the readers read, this file, the +answer sheets, and the facts the record needs that the sheets cannot show. Its marks are final +unless the editor shows that a rule here was misapplied, and a re-mark says which rule and why. + +Every mark is given by a rule in this file or in the key, as follows. Where neither settles a +mark, the marker gives the reader the benefit of the doubt, records the point in the run record, +and the editor settles it by a change to this file or to the key before the next run. A run +record states no judgement that is not written here or in the key; a judgement an earlier run +recorded is not a rule. + +- **Keyed items.** Question 1 as a whole is one keyed item, and so is each item of question 2 + and each answer of question 3 for which the key gives an intended answer. +- **Reading an answer.** An answer is read for its sense, not its wording: a paraphrase, a + synonym, or a defined term used in its defined sense states what it means. A reader quotes the + document when it gives consecutive words of the document, in the document's order and of any + length, in quotation marks or a block quote, verbatim apart from letter case, quotation-mark or + apostrophe style, and markdown link or emphasis markup. +- **Question 1** is correct when at least four bullets each name an obligation the key lists and + no bullet asserts one the key marks wrong. +- **Question 2.** An answer is the reader's sentences and its quotation read together. It is + correct when it states every element the key lists as required for that item, in the + document's words or the reader's own, asserts nothing the key marks wrong, and quotes the + document. An element stated only in the quotation is stated. Anything in the key beyond the + required elements and the wrong assertions is explanation for the marker, and an answer is + never marked down for leaving it out. +- **Question 3.** Each item is correct when its grade is one the key accepts and its reasoning + mentions what the key says it must mention for that grade. A grade the key accepts without a + condition is correct whatever the reasoning. +- **A miss that counts against the document.** The key names, for question 1 and for each item + of question 3, the wrong readings the text invites. A miss of that kind blocks whatever the + count, since one reader in five taking that reading means one agent in five acting on it. Every + other single miss under questions 1 and 3 is the reader's and is tolerated by the threshold + below; a second reader making the same miss is a failed threshold, not a judgement. +- **An unanswered question.** Each keyed item left unanswered is a miss. Question 4 left + unanswered counts as no passage named. +- **Question 4.** A reader names a passage when it quotes words of the document or gives a clause + number with a description that identifies a sentence or paragraph. A remark that is not about + a passage of the document, such as that another document was not supplied, is recorded as a + note and is not a passage. Two readers name the same passage when the words or sentences they + name lie in the same paragraph; a heading, a list item, a table row and a terminology entry are + each a paragraph of their own. A passage named by two or more readers of the run is a finding. +- **Where a finding lies.** The record says, for each finding, whether it lies in text the change + wrote or altered: a sentence or paragraph the pull request adds, rewords or moves, read against + the branch it targets. A finding there blocks the change. A finding anywhere else is a finding + against the document, not against the change: it is recorded in + [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not + block. The change is judged on what it changed; the document is proved by the keyed questions, + which every run answers over the whole of it. + ## Pass criteria -The change passes when all of the following hold on a single run: +The change passes when all of the following hold on a single run. The thresholds apply to the +readers whose answers count; with five readers, "every reader" is five of five and "all but one" +is four of five. -| Criterion | Threshold | -| ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- | -| Load-bearing statements (question 2) | **5 of 5** readers correct on every one | -| Restatement and application (questions 1 and 3) | **At least 4 of 5** readers correct on each keyed item | -| A wrong answer traceable to the text | **None.** A miss whose reasoning quotes the document is a finding, whatever the count | -| A passage found confusing (question 4), by two or more | **None outstanding.** It is a finding and blocks until the text is changed and a fresh cohort no longer names it | -| Anything from one reader only | Recorded; applied at the editor's discretion; does not block | +| Criterion | Threshold | +| ------------------------------------------------------------------------------------- | -------------------------------------------------------------- | +| Load-bearing statements (question 2) | **Every** reader correct on every item | +| Restatement and application (questions 1 and 3) | **All but one** reader correct on each keyed item | +| A miss the key says counts against the document | **None**, whatever the count | +| A passage found confusing by two or more readers, in text the change wrote or altered | **None.** It is a finding and blocks until the text is changed | +| A passage found confusing by two or more readers, elsewhere | Recorded in `acceptance/ROUTED.md` and routed; does not block | +| Anything from one reader only | Recorded; applied at the editor's discretion; does not block | The asymmetry is deliberate. A load-bearing statement is the sentence the whole document exists to make, and one reader in five getting it wrong means one agent in five will act on the wrong reading; unanimity is the only acceptable result there. Everywhere else a single divergent reader -is more likely to be the reader than the text, so four of five is tolerated, but a miss that the -reader can justify from the document is always the document's fault. - -## The bar only moves up - -No criterion in this document is ever relaxed to let a change through. If the cohort is confused, -the test is working and the text is wrong; the fix is to the text, never to the threshold, the -question, the fixture or the key. A threshold may be raised, a keyed question may be added, and a -key may be corrected where it misquoted the document; none of those makes a failing run pass. The -one time a threshold was relaxed, after three runs, the relaxation was reversed on the author's -instruction and is recorded here so that it is not tried again. Removing the clarity rating and -the request for suggestions was not a relaxation: it changed what the cohort is asked, not how -strictly an answer is marked. +is more likely to be the reader than the text, so one such reader is tolerated, unless the key +says the text invites that reading. A finding is settled run by run: a passage two readers named +in one run has no standing in the next, which is judged on its own readers, and a passage the +fresh cohort names by fewer than two is clear. + +## Changing the test + +Nothing in this test is changed to let a run through. A threshold, a question, a key or a +fixture is changed only before a run's readers start, never changes a mark already given, and +is recorded with its reason: a threshold, a question or the rules here by a change to this +process, a key by step 4 of `CONTRIBUTING.md` when the text it quotes is reworded or when a +load-bearing obligation is added, and each in [acceptance/CHANGES.md](acceptance/CHANGES.md). A +question is reworded only so that it asks for what its key requires, and a key's required +elements are never more than its question asks for. A run made after such a change is a fresh +run under the new rules, not a re-mark of an old one. The one time a threshold was relaxed to +let a change through, after three runs, the relaxation was reversed on the editor's instruction +and is recorded here so that it is not tried again. Removing the clarity rating and the request +for suggestions was not that: it changed what the cohort is asked, not how strictly an answer is +marked. ## After a failure -Fix the text, never the questions, and run a fresh cohort. Readers are not reused across runs. -The number of runs it took is recorded; a document that needed three runs is telling its editor -something. +Fix the text the findings name and run a fresh cohort; readers are never reused. A keyed miss is +fixed in the text that carries the obligation, wherever the change left it. A finding on text the +change wrote or altered is fixed there. A finding elsewhere goes to `acceptance/ROUTED.md` and +is not fixed in this change. The number of runs it took is recorded; a document that needed three +runs is telling its editor something. ## When it runs -Once per pull request, on the final text, after the editor's rewrite and any key update. It is -the last gate before the merge to `next`, so that every change on `next` is one the cohort has -accepted, and the release branch needs no cohort of its own unless it changes specification text. -A wording change after the run means a fresh run; readers are never reused. +On the final text of each pull request, after the editor's rewrite and any key update, one cohort +per changed document. It is the last gate before the merge to `next`, so that every change on +`next` is one the cohort has accepted, and the release branch needs no cohort of its own unless it +changes specification text. A fresh cohort runs after every change to the pull request's own +text that follows a run, and after every void run; a text that failed is never run again +unchanged. A version number or date that `PUBLISHING.md` or step 5 of `CONTRIBUTING.md` has the +editor set is not such a change, and neither is a merge of `next` into the branch that brings in +text another cohort accepted and alters no sentence this change wrote. ## The record -Each run has a file under `acceptance/runs/`, named `NNN-.md` with `NNN` the next number in -the one sequence shared by the three documents, stating: the document, the pull request and the -commit read, the cohort size and model, the run number in that pull request, the marks per -reader per question, the findings applied, and whether it passed. The changelog line for the -change names the file. CI refuses a specification change whose pull request adds no run file; -it checks for at least one, and a pull request that changes two documents adds a run for each. - -Each published version's changelog entry carries the lines of the changes it contains, so a -version's cohort record is the set of runs those lines name. A version with no such line has not -been accepted and MUST NOT be described as published. The question sets and answer keys live -under `acceptance/` in this repository so that a run can be repeated and compared. +Each run has a file under `acceptance/runs/`, named `NNN--run-.md` with `NNN` the next +number in the one sequence shared by the three documents and `` the run number within the pull +request, stating: the document; the pull request and the commit read; the cohort size and model, +with the model version each reader reported; how the readers were isolated and the command they +were started with; any reader set aside and why; who marked and, for a delegated marker, how it +was started; the marks per reader per keyed item, with each miss and the reader's reasoning in +brief; every passage named under question 4, with the words quoted or the description given, its +clause and the readers naming it; each finding and whether it lies in text the change wrote or +altered; any point the rules did not settle; whether the run was made under a process change +before it merged and, if so, at which commit; the findings applied afterwards; the verdict; and +a link to the answer sheets. + +The answer sheets are not kept in this repository. Every reader's whole output as its process +returned it, the exact prompt every reader received, the command the readers were started with, +and the instruction a delegated marker was given are posted in one or more comments on the pull +request, linked from the record and never edited, hidden or deleted after posting. + +The changelog line for the change names the record. CI refuses a specification change whose pull +request adds no run file; it checks for at least one, and a pull request that changes two +documents adds a run for each. Each published version's changelog entry carries the lines of the +changes it contains, so a version's cohort record is the set of runs those lines name. A version +with no such line has not been accepted and MUST NOT be described as published. The question sets +and answer keys live under `acceptance/` in this repository so that a run can be repeated and +compared. + +`acceptance/ROUTED.md` lists every finding routed from a run, open until a change that fixes +its passage passes a cohort, or until the editor closes it with a recorded reason. The editor +reads that list before cutting a release. + +## Changes to this process + +The process files are this file, `acceptance/README.md`, `acceptance/runs/README.md`, +`acceptance/ROUTED.md`, `CONTRIBUTING.md` and the checks that enforce them: `tools/changes.py`, +the key-quotation checks in `spec-qa.py`, the CI workflow and the pull request template. A change +to any of them, or to a question set, is a change to this process. They are not specifications, +and a change to them is not tested by a cohort: the cohort measures whether a low-strength reader +can follow a specification, and the process is followed by the editor. Such a change names the +runs or the case that showed the need for it, and goes through steps 1, 3 and 6 of +`CONTRIBUTING.md`, with no scope, key or cohort step; its review reads every process file it +changes. It applies to every run whose readers start after it merges; a run made before then may +be made under it, following all of it at a commit named in the record before the readers start. +It is recorded by a line added at the top of [acceptance/CHANGES.md](acceptance/CHANGES.md), +naming its pull request, not in the changelog, which is for the three specifications. A change to +a key or a fixture follows `acceptance/README.md`. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fa1743e..013bb9d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -29,7 +29,9 @@ review. It is a patch version when released. SHOULD or MAY in a numbered section or appendix. It needs everything a clarity edit needs, plus the five tests in SCOPE.md answered in the pull request, and it is at least a minor version when released. `tools/changes.py` tells the two apart mechanically and CI runs it on every pull -request. +request. A change to the process files ACCEPTANCE.md names, this file among them, is neither: +ACCEPTANCE.md says how it is made, and it is recorded in `acceptance/CHANGES.md`, not in +CHANGELOG.md. ## What a pull request contains @@ -55,20 +57,26 @@ survives, the change is yours. the editor sets the pre-release header. 2. **Scope.** For an obligation, the editor confirms each SCOPE.md test. A failed test closes the pull request with a row in `DECLINED.md`. -3. **Review and rewrite.** An independent review of the diff against the document, then the - editor's rewrite, pushed to the pull request branch where the contributor allows it or to an - editor's branch that supersedes it. +3. **Review and rewrite.** An independent review of the diff against the whole document, by a + reviewer who did not draft the change, then the editor's rewrite, pushed to the pull request + branch where the contributor allows it or to an editor's branch that supersedes it. The pull + request names the reviewer and links or contains the review. 4. **Keys.** Where the change touches a sentence an acceptance key quotes, or adds a load-bearing - obligation, the editor updates the key before the cohort runs, never after. -5. **The cohort.** The acceptance test in ACCEPTANCE.md runs on the final text. Its record is - committed under `acceptance/runs/`, the `Unreleased` changelog line names it, and CI refuses - a specification change that lacks one. For an obligation, the editor sets the document's - header to `-dev, unpublished` in the same commit if it still names a published - version. A wording change after the run means a fresh run. + obligation, the editor updates the key before the cohort runs, never after. The key's + required elements stay what its question asks for; `acceptance/README.md` says what a key + update may and may not do. +5. **The cohort.** The acceptance test in ACCEPTANCE.md runs on the final text, one cohort per + changed document. Its record is committed under `acceptance/runs/`, the `Unreleased` + changelog line names it, and CI refuses a specification change that lacks one. For an + obligation, the editor sets the document's header to `-dev, unpublished` in the + same commit if it still names a published version. A change to the pull request's own text + after the run means a fresh run, as ACCEPTANCE.md says under "When it runs". A finding the + run routes to `acceptance/ROUTED.md` is not fixed in this pull request. 6. **Merge** into `next` by merge commit, and the branch is deleted. -Steps 2 to 5 are done by a person with a Claude Code session, not by CI, and they happen in -batches. A quiet pull request is not a declined one; a declined one says so. +Steps 2 to 5 are the editor's, not CI's, and they happen in batches. The editor may have an +agent draft, review or mark, as ACCEPTANCE.md allows for marking; every decision in these steps +stays the editor's. A quiet pull request is not a declined one; a declined one says so. ## Versions and releases diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md new file mode 100644 index 0000000..a076ff1 --- /dev/null +++ b/acceptance/CHANGES.md @@ -0,0 +1,36 @@ +# Changes to the acceptance process + +Changes to the process files that [ACCEPTANCE.md](../ACCEPTANCE.md) names, recorded here as it +describes: a line is added at the top for each change, naming its pull request. The process is +not a specification and is not versioned, and a change to it is not tested by a cohort. Changes +to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). + +- Pull request NNN: the test is made to prove what it exists to prove, the clarity of the + document to a cold reader, after thirty-four runs on two changes (19 to 30, 31 to 33 and 35 + to 52) in which keyed answers reached five of five and the runs still failed. Four defects in + the process were found. Keys required elements the question never asked for (detector 2b's + "need not remove or forbid", detector 2c's "at a version tracked together", and on pull + request 14 method 2d's order of the kinds), so readers who answered the question as put were + marked wrong: now each key lists, per item, the elements an answer must state, one for + each part of the question, and the questions are reworded to ask for exactly those; anything + else in the key only catches a wrong assertion. A passage two readers found confusing stayed + open whilst any single reader of a later cohort named any part of its paragraph, so cleared + passages kept coming back: now a finding is settled run by run, raised by two readers of one + run and clear when the next names it by fewer than two. A finding landed on text the change + never touched, so a change's cohort acted as a review of the whole document and could not + converge: now a finding in text the change wrote or altered blocks the change, and a finding + elsewhere is routed to `acceptance/ROUTED.md` and to a change of its own, whilst the keyed + questions still prove the whole document on every run. Markers invented judgements the rules + left open and later runs cited them as precedent: now an open point is marked for the reader + and settled in this file or the key before the next run, and a run record states no judgement + the rules do not. The rule that any miss whose reasoning quoted the document was a finding + whatever the count is replaced by the key naming the wrong readings the text invites, which + count against the document on one reader, every other single miss being tolerated by the + all-but-one threshold. Question 2's instruction asks for the sentence or sentences relied on, + since three keys rest on two. The cohort, the packet, set-aside readers, the posted sheets, + the fuller record and the route for a process change are kept from the earlier proposal in + pull request 13, which this change supersedes. Relative to the rules on `next`, the + traceable-miss row is narrower and the confusion row is applied to the change rather than + the document; both are recorded here as the reason the earlier rules could not converge, and + neither changes a mark already given. Prompted by runs 19 to 30 on pull request 14 and runs + 31 to 52 on the defined-terms change. diff --git a/acceptance/README.md b/acceptance/README.md index 4cfb6ec..49a097b 100644 --- a/acceptance/README.md +++ b/acceptance/README.md @@ -2,8 +2,10 @@ The question sets and answer keys that [ACCEPTANCE.md](../ACCEPTANCE.md) requires. One directory per specification document, each holding the fixed questions a cold reader answers and the key an -editor marks against. The fixture cases the application question uses live in `fixtures/`, and -the record of every run lives in `runs/`. +editor marks against. The fixture cases the application question uses live in `fixtures/`, the +record of every run lives in `runs/`, findings routed out of a run to a change of their own are +listed in [ROUTED.md](ROUTED.md), and changes to the acceptance process itself are recorded +in [CHANGES.md](CHANGES.md). | Document | Questions | Key | | ------------------ | ------------------------------------------------ | ------------------------------------ | @@ -14,24 +16,47 @@ the record of every run lives in `runs/`. ## How a run works 1. The editor identifies which documents the change touched. Only those are tested. -2. For each, five or more fresh low-strength readers each receive the document's current text, - its `QUESTIONS.md`, and the fixture the questions name. Nothing else: no key, no changelog, no - critique findings, no other reader's answers. -3. The editor marks each answer sheet against `KEY.md` and applies the pass criteria in - `ACCEPTANCE.md`. -4. The result is recorded as a file under [runs/](runs/), with cohort size, model, run number, - marks and the findings applied, and the change's `Unreleased` changelog line names that file. - A run that fails is followed by a change to the text, never to the questions, and a fresh - cohort. +2. For each, an odd number of fresh low-strength readers, at least five, each isolated as + `ACCEPTANCE.md` describes, receive the document's current text, its `QUESTIONS.md`, and the + fixture the questions name. Nothing else: no key, no changelog, no critique findings, no other + reader's answers. +3. The editor, or a separate marker the editor delegates to, marks each answer sheet against + `KEY.md` under the marking rules in `ACCEPTANCE.md` and applies its pass criteria. +4. The result is recorded as a file under [runs/](runs/), stating what `ACCEPTANCE.md` lists, + with the answer sheets posted on the pull request and linked from it, and the change's + `Unreleased` changelog line names that file. A run that fails is followed by a change to the + text the findings name and a fresh cohort; a finding on text the change did not write or + alter is added to [ROUTED.md](ROUTED.md) instead. + +## How a key is written + +A key says, for each keyed item, exactly what is marked and nothing a marker has to guess: + +- **Question 1**: the obligations a bullet may name, and the assertions that make a bullet wrong. +- **Question 2**: for each item, the elements an answer is required to state, numbered, one for + each part of the question; the assertions that make an answer wrong; and the sentence or + sentences of the document the answer rests on, quoted verbatim. The required elements are + what the question asks for, no more: an element the question does not ask for is not + required, however load-bearing, until the question is reworded to ask for it. +- **Question 3**: for each item, the grade or grades accepted, any condition on the reasoning, + and the wrong readings the text invites, which count against the document when one reader + takes them. + +Everything else in a key is explanation for the marker and never marks an answer down. ## Keeping the keys honest A key quotes the document. When a clause is reworded, the key line that quotes it is updated in the same commit, and the change is one the acceptance run itself then tests. A key that no longer -matches the document is a defect in this directory, not a reason to mark a reader wrong. +matches the document is a defect in this directory, not a reason to mark a reader wrong. A key +update under step 4 of `CONTRIBUTING.md` changes a quotation to follow the text, or adds a keyed +item for a new load-bearing obligation; it does not change what an existing item requires. A +change to what an item requires, or to a question, is a change to the process and is recorded in +[CHANGES.md](CHANGES.md) with its reason, before the readers of the next run start. Questions are fixed so that runs can be compared across versions. Adding a keyed question is -allowed when a new clause is load-bearing; removing or softening one to make a run pass is not. +allowed when a new clause is load-bearing; removing or softening one to make a run pass is not; +rewording one is allowed so that it asks for what its key requires, and for no other reason. The reader may quote a passage it found confusing; it is never asked to rate the document, say what would make a passage clearer, or propose a change: the cohort confirms comprehension and does not design the text. diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md new file mode 100644 index 0000000..bb82718 --- /dev/null +++ b/acceptance/ROUTED.md @@ -0,0 +1,32 @@ +# Passages routed out of a cohort + +A finding is a passage two or more readers of one cohort named as confusing, under question 4, +that lies outside the text the change being tested wrote or altered. `ACCEPTANCE.md` routes such +a finding here rather than blocking the change whose cohort raised it, so that each change is +judged on what it changed and the document is still made clearer, one change at a time. The +editor reads this list before cutting a release. + +A finding is open until a change that reworks its passage passes a cohort, when the line names +that run and moves below the rule, or until the editor closes it with a reason recorded here. A +finding raised again by a later run is noted on its line, not listed twice. + +## Open + +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | -------------------------------------- | +| `TOOLING-SPEC.md` | 9 and 9.1 | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 2, 4.2, 8.1 | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 9.2 | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 4.3 | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | run 33 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 4.1 | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | 5.3 | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | open | +| `DETECTOR-SPEC.md` | 7.1 | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | 8.1 | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | 5.3 | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | +| `SPEC.md` | 1 (Owner) | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | open | +| `SPEC.md` | 4 | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 (one reader, 29) | open; a method question for the editor | +| `SPEC.md` | 2 and 3.1 | "technique" used for a way of writing the Rule in section 2 and for a way of searching in clause 3.1 | runs 29, 38 | pull request 14 rewords both | + +## Closed + +None yet. diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index 132e6cb..b0a3f97 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -1,8 +1,9 @@ # Answer key for DETECTOR-SPEC.md -Marked against the document at the version named in the changelog entry that records the run. -Quotations below are from the document; when a clause is reworded, update the quotation here in -the same commit. +Marked against the document at the commit the readers read, with this key, its question set and +fixture at that same commit. Quotations below are from the document; when a clause is reworded, +update the quotation here in the same commit. `ACCEPTANCE.md` says how each item is marked; +`acceptance/README.md` says what each part of this key is for. ## 1. Restatement @@ -11,67 +12,93 @@ bullet asserting an obligation the document does not state, is a correct restate - Support bespoke rules written by the project ([4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it)). - Provide a harness that runs a single rule against supplied code ([4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code)). -- Allow a stable identifier per rule and print it, unaltered, with every finding ([4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding)). +- Allow a stable identifier per rule and print it, unaltered, with every finding ([4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding)); a + bullet that says each rule carries a stable identifier names this obligation. - Be invocable locally by the practitioner, over a subset down to one file, with the result in - the command's own output and never only through a hosted service or CI-only mode ([5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) to [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally)). + the command's own output and never only through a hosted service or CI-only mode ([5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) to [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally)); + a bullet that says every finding, or each finding, must appear in that output names [5.3](../../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran), + whose summary may leave findings out, and is not marked wrong for the plural. - Resolve a printed identifier to documentation; for bundled rules from the installed copy, offline, with the documentation shipped and versioned with the rule ([6.1](../../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) to [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together)). - Make any inline suppression route detectable or disableable ([7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable)). Wrong if it asserts: the detector must enforce a project's suppression policy; the detector must declare a version to conform; the detector must list a project's active defences or produce an -agent summary (those belong to the toolchain specification). +agent summary (those belong to the toolchain specification). A bullet that states what +conformance means, or restates a SHOULD as advice, asserts no obligation and is neither correct +nor wrong. + +Wrong readings the text invites, which count against the document on one reader: none named. ## 2. Load-bearing statements -- **2a.** Conformance is satisfying every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression), judged on evidence; a detector - that predates the document can conform, and the declaration is optional. Key quotation, section - 8: "A Detector Conforms if it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression)." Supporting, [8.1](../../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps): "A - maintainer MAY declare the version of this document the detector conforms to". An answer that - says a declaration is required, or that partial satisfaction is conformance, is wrong. -- **2b.** It must make the route either disableable by configuration or detectable by a rule the - project can write or a mechanical check it documents; it is not required to remove or forbid - it. Key quotation, [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable): "For each one it MUST do at least one of two things, and either alone - satisfies this clause: make the route disableable by configuration; or make the route - detectable, by a Rule the project can write in the Detector itself or by a mechanical check the - Detector documents". An answer that says the detector must forbid the route, or must require a - reason ([7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason) is SHOULD), is wrong. -- **2c.** With the rule, in the installed copy, at a version tracked together, and yes, offline. - Key quotations, [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access): "Resolution of a bundled rule's identifier MUST work from the installed - copy, without network access"; [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together): "A bundled rule's documentation MUST ship with the rule, at - a version tracked together". An answer naming the detector's website as an acceptable location - is wrong. +- **2a.** Required: (1) a detector conforms when it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression); + (2) yes, a detector that predates the document can conform, because conformance is judged on + the evidence and a declaration is not a condition of it. Wrong if it says a declaration is + required, or that partial satisfaction is conformance. Key quotation, section [8](../../DETECTOR-SPEC.md#8-conformance): "A Detector + Conforms if it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression)." Supporting, [8.1](../../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps): "A maintainer + MAY declare the version of this document the detector conforms to". "Judged on evidence", "the + declaration is optional" or "graded on evidence by anyone" each state the reason in element + (2); one of them is enough. +- **2b.** Required: (1) it must let the project switch the route off in its configuration, or + make the route detectable (by a rule the project can write, or by a mechanical check the + detector documents; naming either means of detection, or just "detectable", states this); + (2) yes, either one alone is enough. Wrong if it says the detector must remove or forbid the + route, or must require a reason ([7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason) is SHOULD). Key quotation, [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable): "For each one it + MUST do at least one of two things, and either alone satisfies this clause: make the route + disableable by configuration; or make the route detectable, by a Rule the project can write in + the Detector itself or by a mechanical check the Detector documents". An answer need not say + that the detector is not required to remove or forbid the route; that follows from element (2) + and the question does not ask it. +- **2c.** Required: (1) with the rule, in the installed copy; (2) at a version tracked together + with the rule; (3) yes, it must be reachable without network access. Wrong if it names the + detector's website, or any location reached over the network, as acceptable. Key quotations, + [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access): "Resolution of a bundled rule's identifier MUST work from the installed copy, without + network access"; [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together): "A bundled rule's documentation MUST ship with the rule, at a version + tracked together". "Shipped with the rule", "in the installed package" or "on disk with the + detector" each state element (1); "versioned with the rule" or "at the same version" states + element (2). ## 3. Application: Lintomatic 4.2 -| Clause | Intended verdict | Reasoning the fixture supports | -| ------ | ---------------- | -------------------------------------------------------------------------------------------------------------------- | -| [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it) | Yes | Project rules via the `Rule` interface, first class in every report | -| [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, with a distinguishing exit code | -| [4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding) | Yes | `LM-NNNN` fixed at first release; project names taken as given, not rewritten; printed on every finding line | -| [5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) | Yes | Local, no account, no server, no network | -| [5.2](../../DETECTOR-SPEC.md#52-the-detector-must-support-invocation-over-a-subset-at-minimum-a-single-file) | Yes | `check ` accepts a single file | -| [5.3](../../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | Yes | One line per finding on standard output whether or not the HTML report is enabled | -| [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally) | Yes | The hosted dashboard shows nothing the command line does not print | -| [6.1](../../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | Yes | `explain ` resolves a printed identifier; project rules resolve to their declared `doc` string | -| [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access) | Yes | `explain` reads the installed package; there is no website | -| [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) | No | Two of forty bundled rules print `No documentation declared`; every identifier a bundled rule can print MUST resolve | -| [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | Yes | The ignore comment is disableable with `ignore_comments = false` | - -Section [8](../../DETECTOR-SPEC.md#8-conformance) verdict: **does not conform**, because [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) fails and partial conformance MUST NOT be -described as conformance. A reader who grades [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) Partial on the grounds that thirty-eight resolve -has read the clause differently and that counts against the document; a reader who grades [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) Yes -because they missed the last paragraph of the fixture has misread the fixture, not the document. - -Clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation): **does not change the verdict**. It is a SHOULD, and Lintomatic's failure to check its -own identifiers in CI is a missed SHOULD, not a further MUST failure; the MUST that fails is [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together). - -A reader who marks [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) No because the comment takes no reason has confused [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) with [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason), which is -SHOULD; that counts against the document if the reasoning quotes [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) or [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason). +| Clause | Accepted grade | Condition on the reasoning, and what the fixture supports | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it) | Yes | Project rules via the `Rule` interface, first class in every report | +| [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, with a distinguishing exit code | +| [4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding) | Yes | `LM-NNNN` fixed at first release; project names taken as given, not rewritten; printed on every finding line | +| [5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) | Yes | Local, no account, no server, no network | +| [5.2](../../DETECTOR-SPEC.md#52-the-detector-must-support-invocation-over-a-subset-at-minimum-a-single-file) | Yes | `check ` accepts a single file | +| [5.3](../../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | Yes | One line per finding on standard output whether or not the HTML report is enabled; a reader who notes that the HTML report need not be named for that reason is right | +| [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally) | Yes | The hosted dashboard shows nothing the command line does not print | +| [6.1](../../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | Yes | `explain ` resolves a printed identifier; project rules resolve to their declared `doc` string; the two undocumented bundled rules are [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together)'s failure | +| [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access) | Yes | `explain` reads the installed package; there is no website | +| [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) | No | Two of forty bundled rules print `No documentation declared`; every identifier a bundled rule can print MUST resolve | +| [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | Yes | The ignore comment is disableable with `ignore_comments = false` | + +No grade above carries a condition on the reasoning: the accepted grade is correct whatever the +reasoning, and a reader who adds a caveat about what the fixture does not say and still gives +the accepted grade is correct. + +Section [8](../../DETECTOR-SPEC.md#8-conformance) verdict: **does not conform**; a reader is correct who says so and names [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), the +two undocumented bundled rules, or partial conformance not counting, as the reason. + +Clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation): **does not change the verdict**; a reader is correct who says so and gives as the +reason that it is a SHOULD, or that the MUST which fails is [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together). + +Wrong readings the text invites, which count against the document on one reader: + +- [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) graded Partial on the grounds that thirty-eight of forty resolve, which has read the + clause as admitting a proportion. +- [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) graded No because the comment takes no reason, where the reasoning quotes [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) or [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason), + which has read [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason)'s SHOULD into [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable). + +A reader who grades [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) Yes because they missed the last paragraph of the fixture has misread the +fixture, which is the reader's miss. ## 4. Confusion -No keyed answer. Record every passage quoted. Two or more readers quoting the same passage is a -finding that blocks until the text is changed and a fresh cohort no longer quotes it. What to -change is the editor's decision; the reader is not asked and any suggestion it volunteers is -discarded unread. +No keyed answer. Record every passage named, as `ACCEPTANCE.md` defines naming and the same +passage. A passage named by two or more readers of the run is a finding: it blocks the change +where it lies in text the change wrote or altered, and is routed to `acceptance/ROUTED.md` +otherwise. What to change is the editor's decision; the reader is not asked and any suggestion +it volunteers is discarded unread. diff --git a/acceptance/detector/QUESTIONS.md b/acceptance/detector/QUESTIONS.md index 58d06d1..2666b2f 100644 --- a/acceptance/detector/QUESTIONS.md +++ b/acceptance/detector/QUESTIONS.md @@ -10,14 +10,15 @@ In your own words and in at most six bullets, what does this document require of ## 2. Load-bearing statements -Answer each in one sentence, followed by the sentence of the document you are relying on, quoted. +Answer each part of each question in a sentence or two, then quote the sentence or sentences of +the document you are relying on. - 2a. What does it mean for a detector to conform, and can a detector that was written before this document existed, and has never heard of it, conform? - 2b. A detector ships an inline ignore comment. What must the detector do about that route for - it to conform? -- 2c. Where must the documentation for a bundled rule's identifier live, and must it be - reachable without network access? + it to conform, and is doing one of those things enough? +- 2c. Where must the documentation for a bundled rule's identifier live, at what version, and + must it be reachable without network access? ## 3. Application diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index 3ca1a80..182d6b2 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -1,8 +1,9 @@ # Answer key for SPEC.md -Marked against the document at the version named in the changelog entry that records the run. -Quotations below are from the document; when a clause is reworded, update the quotation here in -the same commit. +Marked against the document at the commit the readers read, with this key, its question set and +fixture at that same commit. Quotations below are from the document; when a clause is reworded, +update the quotation here in the same commit. `ACCEPTANCE.md` says how each item is marked; +`acceptance/README.md` says what each part of this key is for. ## 1. Restatement @@ -18,62 +19,80 @@ bullet asserting an obligation the document does not state, is a correct restate with the rule ([3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation)). - Record a defect whose fix is deferred where the project's other decisions are enumerable, rather than only mentioning it (section [2](../../SPEC.md#2-when-the-method-applies)). +- Take a baseline, a suppression or an instance left unfixed to the owner, never on the + practitioner's own authority (section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom)). Wrong if it asserts: the practitioner fixes the reported instance first and defends afterwards; a test may serve as the detector; an agent may baseline the remaining instances. +Wrong readings the text invites, which count against the document on one reader: none named. + ## 2. Load-bearing statements -- **2a.** Attribute the defect to a class, express it as a detector rule, prove the rule fires, - then sweep and fix every instance, in that order. Key quotations, section [3](../../SPEC.md#3-the-method): "Six clauses, in +- **2a.** Required: (1) the four steps before the fix: attribute the defect to a class, express + the class as a rule in a detector, prove the rule fires, and sweep the codebase; (2) in that + order, with the fix after the sweep. Wrong if it has the fix before the rule, or the sweep + before the proof, or a test as the detector. Key quotations, section [3](../../SPEC.md#3-the-method): "Six clauses, in order."; [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance): "The Practitioner MUST run the Rule across the entire codebase and record the total Instance count before fixing anything, and MUST then fix every Instance found rather - than only the one". An answer that has the fix before the rule, or the sweep before the proof, - is wrong. -- **2b.** The owner of the codebase, always a human; an agent may not. Key quotation, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): - "An Agent MUST NOT Baseline, suppress or knowingly leave an Instance unfixed on its own - authority, whatever the Instance count turns out to be." An answer that lets the practitioner - decide because the count was small, or because the project recorded no rule, is wrong (the - default owner paragraph in the terminology entry for Owner still requires a recorded decision). -- **2c.** All six clauses of section [3](../../SPEC.md#3-the-method) were followed for that defect, and the verdict rests on the - reviewer reproducing the red and green runs, not on the report. Key quotations, section [7](../../SPEC.md#7-conformance): "A - remediation Conforms if all six clauses of section [3](../../SPEC.md#3-the-method) were followed for that Defect, and the - verdict that they were rests on reproduction"; "A verdict on a remediation or a Defence MUST - rest on reproduction, not on the report". An answer that accepts the pull request description - as the verdict is wrong, and so is one that states the six clauses and says nothing of - reproduction. + than only the one". An answer that names the steps by their clause numbers in order states + both elements. +- **2b.** Required: (1) the owner of the codebase, the human who holds that authority; (2) no, + an agent may not do so on its own authority. Wrong if it lets the practitioner or an agent + decide because the count was small, because the project recorded no owner, or at all. Key + quotation, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): "An Agent MUST NOT Baseline, suppress or knowingly leave an Instance + unfixed on its own authority, whatever the Instance count turns out to be." "The owner" names + the defined term and states that the decision is a human's; an answer need not add "a human" + to be correct. The default owner paragraph in the terminology entry for Owner still requires a + recorded decision, so an answer naming whoever instructed the practitioner, for a project that + has recorded nobody, names the owner and is correct. +- **2c.** Required: (1) all six clauses of section [3](../../SPEC.md#3-the-method) were followed for that defect; (2) the verdict + rests on reproduction, the reviewer rerunning the defence, not on the report. Wrong if it + accepts the pull request description or the practitioner's report as the verdict, or says + that most clauses suffice. Key quotations, section [7](../../SPEC.md#7-conformance): "A remediation Conforms if all six clauses + of section [3](../../SPEC.md#3-the-method) were followed for that Defect, and the verdict that they were rests on + reproduction"; "A verdict on a remediation or a Defence MUST rest on reproduction, not on the + report". "Reproduction" or "reproducing the runs" states element (2); the red and green runs + need not be named. ## 3. Application: the ledger remediation -| Clause | Intended verdict | Reasoning the fixture supports | -| ----------------------------------------------------------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | Class, hazard and the next wider rule are named; Partial or No is accepted where the reader notes that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | -| [3.2](../../SPEC.md#32-build-the-net) | Yes | A rule in the project's static analyser, reading code, not a test | -| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | Commit B is red on the fixture and the originating instance and survives as its own commit | -| [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | Five findings recorded, but one instance was left unfixed behind an inline ignore on the agent's own authority; Partial accepted where the reasoning names that suppression | -| [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | The rule is permanent and blocking, but the suppression was not a recorded decision; [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | -| [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | The message is terse and carries an identifier, but the documentation lives on a wiki, not versioned with the rule; Partial accepted where the reasoning quotes the versioning sentence | +| Clause | Accepted grades | Condition on the reasoning, and what the fixture supports | +| ----------------------------------------------------------------------------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | No condition. Class, hazard and the next wider rule are named; Partial or No is the reading that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | +| [3.2](../../SPEC.md#32-build-the-net) | Yes | A rule in the project's static analyser, reading code, not a test | +| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | Commit B is red on the fixture and the originating instance and survives as its own commit | +| [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | The reasoning names the instance left unfixed behind an inline ignore on the agent's own authority; five findings were recorded | +| [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | The reasoning names the suppression no recorded decision covers; the rule is otherwise permanent and blocking. [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | +| [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | The reasoning names the documentation on a wiki, not versioned with the rule; the message is terse and carries an identifier | -Section [7](../../SPEC.md#7-conformance): the **remediation does not conform**, because [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) and [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) were not followed. The -**defence does not conform** either, because [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) fails; a reader who says the defence conforms -because the sweep is what failed has missed [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) and that counts against the document only if the -reasoning quotes the section [7](../../SPEC.md#7-conformance) definition of a conforming defence. +Section [7](../../SPEC.md#7-conformance): the **remediation does not conform**; a reader is correct who says so and names +[3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed. The **defence does not conform** either; a reader is +correct who says so and names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the reasons, or [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which +section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent. Section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): the **inline ignore on `Report::render()`** was the owner's decision, not the -practitioner's. An answer naming instead the decision not to build the wider rule is half right: -that is also listed in section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) as the owner's to take, and an answer naming both is correct. -An answer naming neither is wrong. +practitioner's. A reader who names it is correct, with or without the decision to leave the +wider rule unbuilt, which section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) also gives to the owner. A reader who names only the wider +rule, or neither, is wrong. + +Wrong readings the text invites, which count against the document on one reader: + +- [3.5](../../SPEC.md#35-enforce-permanently-and-block) graded Yes because the rule fails the build, which has not applied the clause's opening + sentence on unrecorded suppression. +- [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) graded Yes because the URL resolves, which has read the versioning sentence of [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) + differently. +- The defence graded as conforming because the sweep is what failed, where the reasoning quotes + section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence. -A reader who grades [3.5](../../SPEC.md#35-enforce-permanently-and-block) Yes because the rule fails the build has not applied the clause's opening -sentence on unrecorded suppression, and that counts against the document. A reader who grades -[3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) Yes because the URL resolves has read the versioning sentence of [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) -differently, which counts against the document; a reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the -fixture. A reader who attributes the suppression to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) rather than [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) has read the narrowing -passage of [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) as covering it, which is a fair reading and is not marked wrong. +A reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the fixture, which is the reader's miss. A reader who +attributes the suppression to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) rather than [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) has read the narrowing passage of [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) as +covering it, which is a fair reading and is not marked wrong. ## 4. Confusion -No keyed answer. Record every passage quoted. Two or more readers quoting the same passage is a -finding that blocks until the text is changed and a fresh cohort no longer quotes it. What to -change is the editor's decision; the reader is not asked and any suggestion it volunteers is -discarded unread. +No keyed answer. Record every passage named, as `ACCEPTANCE.md` defines naming and the same +passage. A passage named by two or more readers of the run is a finding: it blocks the change +where it lies in text the change wrote or altered, and is routed to `acceptance/ROUTED.md` +otherwise. What to change is the editor's decision; the reader is not asked and any suggestion +it volunteers is discarded unread. diff --git a/acceptance/method/QUESTIONS.md b/acceptance/method/QUESTIONS.md index 072a3c0..34fa82d 100644 --- a/acceptance/method/QUESTIONS.md +++ b/acceptance/method/QUESTIONS.md @@ -11,7 +11,8 @@ do when they are handed a defect? ## 2. Load-bearing statements -Answer each in one sentence, followed by the sentence of the document you are relying on, quoted. +Answer each part of each question in a sentence or two, then quote the sentence or sentences of +the document you are relying on. - 2a. What does a practitioner do before fixing the defect they were handed, and in what order? - 2b. Who may adopt a baseline or suppress an instance, and may an agent do so on its own diff --git a/acceptance/runs/README.md b/acceptance/runs/README.md index 4994f74..3f7ad64 100644 --- a/acceptance/runs/README.md +++ b/acceptance/runs/README.md @@ -1,15 +1,22 @@ # Acceptance run records -One file per cohort run, `NNN-.md`, numbered in the one sequence the three documents -share. Runs 1 to 15 predate this directory and are summarised in the changelog entries for -1.0.0 and 1.0.1 of the method specification, 1.0.0 of the detector specification and 0.2.0 of -the toolchain specification; numbering here continues from 16. +One file per cohort run, `NNN--run-.md`, numbered in the one sequence the three +documents share, with `` the run number within its pull request. Runs 1 to 15 predate this +directory and are summarised in the changelog entries for 1.0.0 and 1.0.1 of the method +specification, 1.0.0 of the detector specification and 0.2.0 of the toolchain specification; +numbering here continues from 16. -A record states, in this order: the document; the pull request and the commit the readers were -given; the cohort size and model; the run number within that pull request; the marks per reader -per question against the key; the passages quoted as confusing and by how many readers; the -findings applied to the text as a result; and the verdict against the criteria in -`ACCEPTANCE.md`. Answer sheets are not pasted in full; the marks are the record. +A record states what `ACCEPTANCE.md` lists under "The record", in this order: the document; the +pull request and the commit the readers were given; the cohort size and model, with the model +version each reader reported; how the readers were isolated and the command they were started +with; any reader set aside and why; who marked and, for a delegated marker, how it was started; +the marks per reader per keyed item, with each miss and the reader's reasoning in brief; every +passage named under question 4, with the words quoted or the description given, its clause and +the readers naming it; each finding and whether it lies in text the change wrote or altered; any +point the rules did not settle; whether the run was made under a process change before it merged +and, if so, at which commit; the findings applied afterwards; the verdict; and a link to the +answer sheets posted on the pull request. Earlier records show the style only; where one differs +from this order, this file wins. Answer sheets are not kept here; the marks are the record. The `Unreleased` changelog line for the change names the record, and `tools/changes.py` refuses a specification change whose pull request adds no file here. diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index 5a2602b..c11ae32 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -1,8 +1,9 @@ # Answer key for TOOLING-SPEC.md -Marked against the document at the version named in the changelog entry that records the run. -Quotations below are from the document; when a clause is reworded, update the quotation here in -the same commit. +Marked against the document at the commit the readers read, with this key, its question set and +fixture at that same commit. Quotations below are from the document; when a clause is reworded, +update the quotation here in the same commit. `ACCEPTANCE.md` says how each item is marked; +`acceptance/README.md` says what each part of this key is for. ## 1. Restatement @@ -25,61 +26,72 @@ bullet asserting an obligation the document does not state, is a correct restate defences on its own source ([8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation), [8.2](../../TOOLING-SPEC.md#82-a-shipped-toolchain-must-run-its-own-bundled-defences-on-its-own-source)). Wrong if it asserts: the toolchain must be a single product; a declaration is required to -conform; the agent summary (section [7](../../TOOLING-SPEC.md#7-agent-context)) is a MUST. +conform; the agent summary (section [7](../../TOOLING-SPEC.md#7-agent-context)) is a MUST. A bullet that calls agent support or the +self-audit a further condition "where it applies", or says a declaration is not a condition, +asserts no obligation and is neither correct nor wrong. + +Wrong readings the text invites, which count against the document on one reader: none named. ## 2. Load-bearing statements -- **2a.** At the project level; the toolchain is whatever the project has assembled from - third-party, first-party and project-level parts. Key quotation, section [9](../../TOOLING-SPEC.md#9-conformance): "A project's - Toolchain Conforms if every MUST in sections [4](../../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) to [6](../../TOOLING-SPEC.md#6-the-project-record) holds across the assembled parts, wherever - each part came from". An answer that says conformance is graded per tool, or that only a - packaged product can conform, is wrong. -- **2b.** One grade; which part satisfied each clause is evidence, not a grade of its own. Key - quotation, section [9](../../TOOLING-SPEC.md#9-conformance): "The project-level verdict is a single grade; which part of the Toolchain - satisfied each clause is evidence for that grade, not a second grade." -- **2c.** Two, graded and declared separately, and neither implies the other. Key quotation, - [9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately): "The two verdicts MUST be graded and declared separately, and neither implies the other." - An answer giving one grade, or saying the artefact grade follows from the project grade, is - wrong. +- **2a.** Required: (1) at the project level; (2) the toolchain is whatever the project has + assembled to run its checks through, from third-party, first-party and project-level parts. + Wrong if it says conformance is graded per tool, or that only a packaged product can conform. + Key quotation, section [9](../../TOOLING-SPEC.md#9-conformance): "A project's Toolchain Conforms if every MUST in sections [4](../../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) to [6](../../TOOLING-SPEC.md#6-the-project-record) + holds across the assembled parts, wherever each part came from". An answer that says everything the + project runs its checks through, or the assembled parts from wherever they came, or that lists + the three kinds of part, states element (2). +- **2b.** Required: (1) one grade; (2) which part satisfied each clause is evidence for that + grade, not a grade of its own. Wrong if it gives a grade per part. Key quotation, section [9](../../TOOLING-SPEC.md#9-conformance): + "The project-level verdict is a single grade; which part of the Toolchain satisfied each + clause is evidence for that grade, not a second grade." +- **2c.** Required: (1) two, graded and declared separately; (2) neither implies the other. + Wrong if it gives one grade, or says the artefact grade follows from the project grade or the + reverse. Key quotation, [9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately): "The two verdicts MUST be graded and declared separately, and + neither implies the other." ## 3. Application: the Zigpipe project -| Clause | Intended verdict | Reasoning the fixture supports | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap | -| [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)` | -| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | -| [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | -| [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | Lintomatic, then the tests, stopping on a Lintomatic failure | -| [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is accepted where the reasoning counts the two `(none)` routes, since 5.1 says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | -| [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration) | Yes | Derived from `lintomatic.toml`, not hand-maintained | -| [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | Yes | The `proj.*` rules appear in the same listing | -| [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) | Yes | `quality-exceptions.toml` is a defined path that `zig-qa run` loads | -| [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | A justification is required and an empty one rejected, but nothing rejects a justification that could be pasted onto any entry | -| [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | `zig-qa list --exceptions` uses the same format and operation as the rule listing | - -Either Partial or No is accepted for [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one); the reasoning must mention the generic-justification -requirement. A Yes on [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) with reasoning that quotes only the first sentence of the clause is a -finding against the document. - -Section [9](../../TOOLING-SPEC.md#9-conformance) verdict: **does not conform**, because [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) and [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) fail; partial conformance MUST NOT be -described as conformance. **Not with agent support**, because section [7](../../TOOLING-SPEC.md#7-agent-context) is not met (no summary). -**Section [8](../../TOOLING-SPEC.md#8-self-audit) does not apply**, because Zigpipe ships nothing to other projects; a reader who applies -[8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) to Zigpipe on the basis that Zigpipe's tooling includes a shipped package (zig-qa) has read -"where the Toolchain is one the project ships" (section [9](../../TOOLING-SPEC.md#9-conformance)) as reaching the consuming project, and that counts against the document. The -correct reading is that section [8](../../TOOLING-SPEC.md#8-self-audit) bears on zig-qa's own artefact grade, not on Zigpipe. - -Declaration: **does not affect the verdict**. Key quotation, [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance): "The declaration is optional." -A reader who says Zigpipe cannot be graded without one has read [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) wrongly. - -A reader who grades [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) Yes on the grounds that Lintomatic "mostly" conforms has read either [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) -or the detector document's section [8](../../DETECTOR-SPEC.md#8-conformance) differently; it counts against this document only if the -reasoning quotes [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification). A reader who grades [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) Yes because they missed the two undocumented rules -has misread the fixture. +| Clause | Accepted grades | Condition on the reasoning, and what the fixture supports | +| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap; a reader who notes that the detector specification was not supplied and grades from this document's summary of it is correct | +| [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)`. The two are blocking, since their firing stops `zig-qa run`; a reader who grades No on that condition is correct | +| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | +| [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | +| [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | Lintomatic, then the tests, stopping on a Lintomatic failure | +| [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | No condition. `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is the reading that counts the two `(none)` routes, since 5.1 says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | +| [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration) | Yes | Derived from `lintomatic.toml`, not hand-maintained | +| [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | Yes | The `proj.*` rules appear in the same listing | +| [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) | Yes or Partial | `quality-exceptions.toml` is a defined path that `zig-qa run` loads. Partial is accepted only where the reasoning says the fixture names a place for exceptions and none for calibrations or conventions, which the project record entry also covers | +| [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | The reasoning mentions that nothing rejects a justification that could be pasted onto any entry; a justification is required and an empty one rejected | +| [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | `zig-qa list --exceptions` uses the same format and operation as the rule listing | + +Section [9](../../TOOLING-SPEC.md#9-conformance) verdict: **does not conform**; a reader is correct who says so and names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or +[4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses. **Not with agent support**; a reader is correct who says so because +section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, there being no summary. **Section [8](../../TOOLING-SPEC.md#8-self-audit) does not apply**; a reader is correct +who says so because Zigpipe ships nothing to other projects, or because section [8](../../TOOLING-SPEC.md#8-self-audit) bears on +zig-qa's own artefact grade and not on Zigpipe. **The absence of a declaration does not affect +the verdict**; a reader is correct who says so. Key quotation, [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance): "The declaration is +optional." + +Wrong readings the text invites, which count against the document on one reader: + +- [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) graded Yes with reasoning that quotes only the first sentence of the clause, which has + not read the sentence on a generic justification. +- [8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) applied to Zigpipe on the basis that its tooling includes a shipped package, which has + read "where the Toolchain is one the project ships" (section [9](../../TOOLING-SPEC.md#9-conformance)) as reaching the consuming + project. +- [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) graded Yes on the grounds that Lintomatic "mostly" conforms, where the reasoning quotes + [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), which has read it as admitting a proportion. +- Zigpipe held ungradeable for want of a declaration, which has read [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) as a condition. + +A reader who grades [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) Yes because they missed the two undocumented rules has misread the +fixture, which is the reader's miss. ## 4. Confusion -No keyed answer. Record every passage quoted. Two or more readers quoting the same passage is a -finding that blocks until the text is changed and a fresh cohort no longer quotes it. What to -change is the editor's decision; the reader is not asked and any suggestion it volunteers is -discarded unread. +No keyed answer. Record every passage named, as `ACCEPTANCE.md` defines naming and the same +passage. A passage named by two or more readers of the run is a finding: it blocks the change +where it lies in text the change wrote or altered, and is routed to `acceptance/ROUTED.md` +otherwise. What to change is the editor's decision; the reader is not asked and any suggestion +it volunteers is discarded unread. diff --git a/acceptance/toolchain/QUESTIONS.md b/acceptance/toolchain/QUESTIONS.md index a4da266..35a2b91 100644 --- a/acceptance/toolchain/QUESTIONS.md +++ b/acceptance/toolchain/QUESTIONS.md @@ -11,7 +11,8 @@ toolchain? ## 2. Load-bearing statements -Answer each in one sentence, followed by the sentence of the document you are relying on, quoted. +Answer each part of each question in a sentence or two, then quote the sentence or sentences of +the document you are relying on. - 2a. At what level is toolchain conformance measured, and what is the toolchain? - 2b. A project's tooling is assembled from a third-party detector, a first-party package and From 36d87d18fe3b2aa586472a45b5888812a6831aae Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 11:43:12 +0100 Subject: [PATCH 02/21] Acceptance process: review 1 findings applied One route for each kind of change to the test (quotation and new item in the specification pull request, everything else a process change); ROUTED.md and runs/ are records, not process files; question 1 defined once; open points marked in a stated direction; quoting needs a six-word run; a finding lies in the change's text when any sentence of its paragraph changed; wrong readings stated as grade plus reasoning; keys ask only what their questions ask (detector 2a, toolchain 2c); the question 3 tables separate the condition from the explanation. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 123 ++++++++++++++++++------------ acceptance/CHANGES.md | 4 +- acceptance/README.md | 17 +++-- acceptance/ROUTED.md | 7 +- acceptance/detector/KEY.md | 17 ++--- acceptance/method/KEY.md | 37 ++++----- acceptance/toolchain/KEY.md | 49 ++++++------ acceptance/toolchain/QUESTIONS.md | 2 +- 8 files changed, 144 insertions(+), 112 deletions(-) diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index 84ab39b..c5bd8cd 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -51,8 +51,8 @@ makes it wrong; the question asks for exactly that and nothing the key does not asks for nothing the question does not. Each reader answers all of them. 1. **Restatement.** In their own words, what the document asks of its subject, in at most six - bullets. Keyed against the list of MUST clauses: a bullet is correct when it names an - obligation the document actually states. + bullets. Keyed against the key's list of the document's obligations: a bullet is correct when + it names one of them. 2. **Load-bearing statements.** Three or more questions, fixed per document, each asking for a short answer and the sentence or sentences of the document it rests on, quoted. Each has one @@ -97,34 +97,41 @@ answer sheets, and the facts the record needs that the sheets cannot show. Its m unless the editor shows that a rule here was misapplied, and a re-mark says which rule and why. Every mark is given by a rule in this file or in the key, as follows. Where neither settles a -mark, the marker gives the reader the benefit of the doubt, records the point in the run record, -and the editor settles it by a change to this file or to the key before the next run. A run -record states no judgement that is not written here or in the key; a judgement an earlier run -recorded is not a rule. +mark, the marker records the point in the run record and marks it as the open-point rule below +says, and the editor settles it in this file or in the key, as [Changing the test](#changing-the-test) +allows, before the next run. A run record states no judgement that is not written here or in +the key; a judgement an earlier run recorded is not a rule. - **Keyed items.** Question 1 as a whole is one keyed item, and so is each item of question 2 - and each answer of question 3 for which the key gives an intended answer. + and each answer of question 3 for which the key gives an accepted grade or answer. - **Reading an answer.** An answer is read for its sense, not its wording: a paraphrase, a synonym, or a defined term used in its defined sense states what it means. A reader quotes the - document when it gives consecutive words of the document, in the document's order and of any - length, in quotation marks or a block quote, verbatim apart from letter case, quotation-mark or - apostrophe style, and markdown link or emphasis markup. + document when it gives, in quotation marks or a block quote, a run of six or more consecutive + words of the document in the document's order, verbatim apart from letter case, quotation-mark + or apostrophe style, and markdown link or emphasis markup; a quotation that drops or changes a + word quotes the document by its longest such run. - **Question 1** is correct when at least four bullets each name an obligation the key lists and - no bullet asserts one the key marks wrong. + no bullet asserts one the key marks wrong. Several bullets that name different clauses of one + line of the key's list each count. A bullet asserting an obligation that is in neither list is + recorded as an open point for the key and is not counted either way. - **Question 2.** An answer is the reader's sentences and its quotation read together. It is correct when it states every element the key lists as required for that item, in the document's words or the reader's own, asserts nothing the key marks wrong, and quotes the document. An element stated only in the quotation is stated. Anything in the key beyond the required elements and the wrong assertions is explanation for the marker, and an answer is never marked down for leaving it out. -- **Question 3.** Each item is correct when its grade is one the key accepts and its reasoning - mentions what the key says it must mention for that grade. A grade the key accepts without a - condition is correct whatever the reasoning. +- **Question 3.** Each item is correct when its grade is one the key accepts and, where the key + sets a condition on the reasoning for that grade, the reasoning meets it. A grade the key + accepts with no condition is correct whatever the reasoning. A grade given on a stated + condition is read as the grade for the case the fixture presents, where the key says which + case that is; otherwise the item is unanswered. - **A miss that counts against the document.** The key names, for question 1 and for each item - of question 3, the wrong readings the text invites. A miss of that kind blocks whatever the - count, since one reader in five taking that reading means one agent in five acting on it. Every + of question 3, the wrong readings the text invites, each as a grade or answer together with + what the reasoning states or leaves out. A miss that matches one blocks whatever the count, + since one reader in five taking that reading means one agent in five acting on it, and it + blocks wherever the text lies, because the keyed questions are what prove the document. Every other single miss under questions 1 and 3 is the reader's and is tolerated by the threshold - below; a second reader making the same miss is a failed threshold, not a judgement. + below; a second miss on the same item fails the threshold, and no judgement is needed. - **An unanswered question.** Each keyed item left unanswered is a miss. Question 4 left unanswered counts as no passage named. - **Question 4.** A reader names a passage when it quotes words of the document or gives a clause @@ -134,12 +141,15 @@ recorded is not a rule. name lie in the same paragraph; a heading, a list item, a table row and a terminology entry are each a paragraph of their own. A passage named by two or more readers of the run is a finding. - **Where a finding lies.** The record says, for each finding, whether it lies in text the change - wrote or altered: a sentence or paragraph the pull request adds, rewords or moves, read against - the branch it targets. A finding there blocks the change. A finding anywhere else is a finding - against the document, not against the change: it is recorded in - [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not - block. The change is judged on what it changed; the document is proved by the keyed questions, - which every run answers over the whole of it. + wrote or altered: a paragraph in which the pull request, read against the branch it targets, + adds, rewords, moves or deletes a sentence, or changes a word, a term or a link. A finding + there blocks the change. A finding anywhere else is a finding against the document, not + against the change: it is recorded in [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a + change of its own, and does not block. The change is judged on what it changed; the document + is proved by the keyed questions, which every run answers over the whole of it. +- **Open points.** An open point in marking an answer is marked for the reader. An open point on + whether a reader named a passage, or whether two readers named the same one, counts the + passage. An open point on where a finding lies treats it as the change's own text. ## Pass criteria @@ -166,26 +176,37 @@ fresh cohort names by fewer than two is clear. ## Changing the test -Nothing in this test is changed to let a run through. A threshold, a question, a key or a -fixture is changed only before a run's readers start, never changes a mark already given, and -is recorded with its reason: a threshold, a question or the rules here by a change to this -process, a key by step 4 of `CONTRIBUTING.md` when the text it quotes is reworded or when a -load-bearing obligation is added, and each in [acceptance/CHANGES.md](acceptance/CHANGES.md). A -question is reworded only so that it asks for what its key requires, and a key's required +Nothing in this test is changed to let a particular run through. A threshold, a question, a +key, a fixture or a rule here is changed only before a run's readers start, never changes a mark +already given, and is changed in one of three ways, each recorded with its reason: + +- A key's quotation follows the text it quotes, and a key gains an item for a new load-bearing + obligation, with the question that asks for it, under step 4 of `CONTRIBUTING.md`, in the + specification pull request that makes the change; `acceptance/README.md` says what that may + and may not do. +- A threshold, a fixture, a rule here, what an existing key item requires, or the wording of a + question changes by a change to this process, with its own pull request. +- An open point a run recorded is settled by either route, whichever it needs. + +A question is reworded only so that it asks for what its key requires, and a key's required elements are never more than its question asks for. A run made after such a change is a fresh -run under the new rules, not a re-mark of an old one. The one time a threshold was relaxed to -let a change through, after three runs, the relaxation was reversed on the editor's instruction -and is recorded here so that it is not tried again. Removing the clarity rating and the request -for suggestions was not that: it changed what the cohort is asked, not how strictly an answer is -marked. +run under the new rules, not a re-mark of an old one. The test for a change to a rule is what +the rule measured: a rule that measured something other than whether a cold reader could follow +the text is a defect in this file and is corrected, and the correction says what the rule +measured instead; a rule that measured that, and found the text wanting, is not touched, and the +text is fixed. The one time a threshold was relaxed to let a change through, after three runs, +the relaxation was reversed on the editor's instruction and is recorded here so that it is not +tried again. Removing the clarity rating and the request for suggestions was not that: it +changed what the cohort is asked, not how strictly an answer is marked. ## After a failure -Fix the text the findings name and run a fresh cohort; readers are never reused. A keyed miss is -fixed in the text that carries the obligation, wherever the change left it. A finding on text the -change wrote or altered is fixed there. A finding elsewhere goes to `acceptance/ROUTED.md` and -is not fixed in this change. The number of runs it took is recorded; a document that needed three -runs is telling its editor something. +Fix the text the run's misses and findings name and run a fresh cohort; readers are never reused. +A keyed miss is fixed in the text that carries the obligation, wherever it lies, since the keyed +questions prove the document and a document one reader in five misreads is not merged to, whoever +wrote the sentence. A finding on text the change wrote or altered is fixed there. A finding +elsewhere goes to `acceptance/ROUTED.md` and is not fixed in this change. The number of runs it +took is recorded; a document that needed three runs is telling its editor something. ## When it runs @@ -217,8 +238,8 @@ returned it, the exact prompt every reader received, the command the readers wer and the instruction a delegated marker was given are posted in one or more comments on the pull request, linked from the record and never edited, hidden or deleted after posting. -The changelog line for the change names the record. CI refuses a specification change whose pull -request adds no run file; it checks for at least one, and a pull request that changes two +The changelog line for the change names every run made on it, the passing run last. CI refuses a +specification change whose pull request adds no run file; it checks for at least one, and a pull request that changes two documents adds a run for each. Each published version's changelog entry carries the lines of the changes it contains, so a version's cohort record is the set of runs those lines name. A version with no such line has not been accepted and MUST NOT be described as published. The question sets @@ -232,15 +253,17 @@ reads that list before cutting a release. ## Changes to this process The process files are this file, `acceptance/README.md`, `acceptance/runs/README.md`, -`acceptance/ROUTED.md`, `CONTRIBUTING.md` and the checks that enforce them: `tools/changes.py`, -the key-quotation checks in `spec-qa.py`, the CI workflow and the pull request template. A change -to any of them, or to a question set, is a change to this process. They are not specifications, -and a change to them is not tested by a cohort: the cohort measures whether a low-strength reader -can follow a specification, and the process is followed by the editor. Such a change names the -runs or the case that showed the need for it, and goes through steps 1, 3 and 6 of -`CONTRIBUTING.md`, with no scope, key or cohort step; its review reads every process file it +`CONTRIBUTING.md` and the checks that enforce them: `tools/changes.py`, the key-quotation checks +in `spec-qa.py`, the CI workflow and the pull request template. A change to any of them is a +change to this process, and so is a change to a threshold, a fixture, the wording of a question +or what a key item requires, as [Changing the test](#changing-the-test) says. They are not +specifications, and a change to them is not tested by a cohort: the cohort measures whether a +low-strength reader can follow a specification, and the process is followed by the editor. Such +a change names the runs or the case that showed the need for it, and goes through steps 1, 3 and +6 of `CONTRIBUTING.md`, with no scope, key or cohort step; its review reads every process file it changes. It applies to every run whose readers start after it merges; a run made before then may be made under it, following all of it at a commit named in the record before the readers start. It is recorded by a line added at the top of [acceptance/CHANGES.md](acceptance/CHANGES.md), -naming its pull request, not in the changelog, which is for the three specifications. A change to -a key or a fixture follows `acceptance/README.md`. +naming its pull request, not in the changelog, which is for the three specifications. +`acceptance/ROUTED.md` and `acceptance/runs/` are records, not process files: a specification +pull request writes to them. diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md index a076ff1..cc01848 100644 --- a/acceptance/CHANGES.md +++ b/acceptance/CHANGES.md @@ -6,7 +6,7 @@ not a specification and is not versioned, and a change to it is not tested by a to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). - Pull request NNN: the test is made to prove what it exists to prove, the clarity of the - document to a cold reader, after thirty-four runs on two changes (19 to 30, 31 to 33 and 35 + document to a cold reader, after thirty-three runs on two changes (19 to 30, 31 to 33 and 35 to 52) in which keyed answers reached five of five and the runs still failed. Four defects in the process were found. Keys required elements the question never asked for (detector 2b's "need not remove or forbid", detector 2c's "at a version tracked together", and on pull @@ -33,4 +33,4 @@ to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). traceable-miss row is narrower and the confusion row is applied to the change rather than the document; both are recorded here as the reason the earlier rules could not converge, and neither changes a mark already given. Prompted by runs 19 to 30 on pull request 14 and runs - 31 to 52 on the defined-terms change. + 31 to 33 and 35 to 52 on the defined-terms change. diff --git a/acceptance/README.md b/acceptance/README.md index 49a097b..dcf39c0 100644 --- a/acceptance/README.md +++ b/acceptance/README.md @@ -48,11 +48,18 @@ Everything else in a key is explanation for the marker and never marks an answer A key quotes the document. When a clause is reworded, the key line that quotes it is updated in the same commit, and the change is one the acceptance run itself then tests. A key that no longer -matches the document is a defect in this directory, not a reason to mark a reader wrong. A key -update under step 4 of `CONTRIBUTING.md` changes a quotation to follow the text, or adds a keyed -item for a new load-bearing obligation; it does not change what an existing item requires. A -change to what an item requires, or to a question, is a change to the process and is recorded in -[CHANGES.md](CHANGES.md) with its reason, before the readers of the next run start. +matches the document is a defect in this directory, not a reason to mark a reader wrong. + +A key, a question set or a fixture changes in one of two ways, as `ACCEPTANCE.md` says under +"Changing the test", and always before the readers of the next run start: + +- **In a specification pull request, under step 4 of `CONTRIBUTING.md`**: a quotation follows + the text it quotes, or a keyed item is added for a new load-bearing obligation together with + the question that asks for it. The pull request's own changelog line records it. Such an + update does not change what an existing item requires, nor an existing question's wording. +- **In a process change, with its own pull request**: what an existing item requires, the + wording of a question, a fixture, or the rules of marking. It is recorded in + [CHANGES.md](CHANGES.md) with its reason. Questions are fixed so that runs can be compared across versions. Adding a keyed question is allowed when a new clause is load-bearing; removing or softening one to make a run pass is not; diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index bb82718..eb35815 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -8,7 +8,9 @@ editor reads this list before cutting a release. A finding is open until a change that reworks its passage passes a cohort, when the line names that run and moves below the rule, or until the editor closes it with a reason recorded here. A -finding raised again by a later run is noted on its line, not listed twice. +finding raised again by a later run is noted on its line, not listed twice. The rows below were +raised by runs 31 to 52 under the earlier rules, on untouched text of the change those runs +tested, and are listed here so that they are not lost. ## Open @@ -24,8 +26,7 @@ finding raised again by a later run is noted on its line, not listed twice. | `DETECTOR-SPEC.md` | 8.1 | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | | `DETECTOR-SPEC.md` | 5.3 | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | | `SPEC.md` | 1 (Owner) | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | open | -| `SPEC.md` | 4 | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 (one reader, 29) | open; a method question for the editor | -| `SPEC.md` | 2 and 3.1 | "technique" used for a way of writing the Rule in section 2 and for a way of searching in clause 3.1 | runs 29, 38 | pull request 14 rewords both | +| `SPEC.md` | 4 | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | open; a method question for the editor | ## Closed diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index b0a3f97..2f6fd96 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -33,13 +33,12 @@ Wrong readings the text invites, which count against the document on one reader: ## 2. Load-bearing statements - **2a.** Required: (1) a detector conforms when it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression); - (2) yes, a detector that predates the document can conform, because conformance is judged on - the evidence and a declaration is not a condition of it. Wrong if it says a declaration is + (2) yes, a detector that predates the document can conform. Wrong if it says a declaration is required, or that partial satisfaction is conformance. Key quotation, section [8](../../DETECTOR-SPEC.md#8-conformance): "A Detector Conforms if it satisfies every MUST in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression)." Supporting, [8.1](../../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps): "A maintainer - MAY declare the version of this document the detector conforms to". "Judged on evidence", "the - declaration is optional" or "graded on evidence by anyone" each state the reason in element - (2); one of them is enough. + MAY declare the version of this document the detector conforms to". The reason, that + conformance is judged on evidence and a declaration is optional, is explanation: an answer + that gives it, or gives it instead of a plain yes, states element (2). - **2b.** Required: (1) it must let the project switch the route off in its configuration, or make the route detectable (by a rule the project can write, or by a mechanical check the detector documents; naming either means of detection, or just "detectable", states this); @@ -61,7 +60,7 @@ Wrong readings the text invites, which count against the document on one reader: ## 3. Application: Lintomatic 4.2 -| Clause | Accepted grade | Condition on the reasoning, and what the fixture supports | +| Clause | Accepted grade | What the fixture supports (no row carries a condition on the reasoning) | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it) | Yes | Project rules via the `Rule` interface, first class in every report | | [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, with a distinguishing exit code | @@ -87,9 +86,9 @@ reason that it is a SHOULD, or that the MUST which fails is [6.3](../../DETECTOR Wrong readings the text invites, which count against the document on one reader: -- [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) graded Partial on the grounds that thirty-eight of forty resolve, which has read the - clause as admitting a proportion. -- [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) graded No because the comment takes no reason, where the reasoning quotes [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) or [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason), +- [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) graded Partial with reasoning that says most, or thirty-eight of forty, bundled rules + resolve, which has read the clause as admitting a proportion. +- [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) graded No with reasoning that says the comment takes no reason and quotes [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) or [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason), which has read [7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason)'s SHOULD into [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable). A reader who grades [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) Yes because they missed the last paragraph of the fixture has misread the diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index 182d6b2..36bfafd 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -57,33 +57,34 @@ Wrong readings the text invites, which count against the document on one reader: ## 3. Application: the ledger remediation -| Clause | Accepted grades | Condition on the reasoning, and what the fixture supports | -| ----------------------------------------------------------------------------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | No condition. Class, hazard and the next wider rule are named; Partial or No is the reading that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | -| [3.2](../../SPEC.md#32-build-the-net) | Yes | A rule in the project's static analyser, reading code, not a test | -| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | Commit B is red on the fixture and the originating instance and survives as its own commit | -| [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | The reasoning names the instance left unfixed behind an inline ignore on the agent's own authority; five findings were recorded | -| [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | The reasoning names the suppression no recorded decision covers; the rule is otherwise permanent and blocking. [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | -| [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | The reasoning names the documentation on a wiki, not versioned with the rule; the message is terse and carries an identifier | +| Clause | Accepted grades | Condition on the reasoning | What the fixture supports | +| ----------------------------------------------------------------------------------------- | ------------------ | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | None | Class, hazard and the next wider rule are named; Partial or No is the reading that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | +| [3.2](../../SPEC.md#32-build-the-net) | Yes | None | A rule in the project's static analyser, reading code, not a test | +| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | None | Commit B is red on the fixture and the originating instance and survives as its own commit | +| [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | Mentions the instance left unfixed, the inline ignore, or `Report::render()` | Five findings were recorded, but one instance was left unfixed behind an inline ignore on the agent's own authority | +| [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | Mentions the suppression, the inline ignore, or that no recorded decision covers it | The rule is permanent and blocking, but the suppression was not a recorded decision. [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | +| [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | Mentions the wiki, or that the documentation is not versioned or co-located with the rule | The message is terse and carries an identifier, but the documentation lives on a wiki, not versioned with the rule | Section [7](../../SPEC.md#7-conformance): the **remediation does not conform**; a reader is correct who says so and names [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed. The **defence does not conform** either; a reader is correct who says so and names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the reasons, or [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which -section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent. +section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent. Other clauses named +alongside are not marked. Section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): the **inline ignore on `Report::render()`** was the owner's decision, not the -practitioner's. A reader who names it is correct, with or without the decision to leave the -wider rule unbuilt, which section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) also gives to the owner. A reader who names only the wider -rule, or neither, is wrong. +practitioner's. A reader who names it is correct; any other decision named alongside it is not +marked. A reader who names only the decision to leave the wider rule unbuilt, or neither, is +wrong. Wrong readings the text invites, which count against the document on one reader: -- [3.5](../../SPEC.md#35-enforce-permanently-and-block) graded Yes because the rule fails the build, which has not applied the clause's opening - sentence on unrecorded suppression. -- [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) graded Yes because the URL resolves, which has read the versioning sentence of [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) - differently. -- The defence graded as conforming because the sweep is what failed, where the reasoning quotes - section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence. +- [3.5](../../SPEC.md#35-enforce-permanently-and-block) graded Yes with reasoning that says the rule blocks, or is permanent, and does not mention + the suppression. +- [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) graded Yes with reasoning that says the identifier resolves, or the URL works, and does not + mention versioning or co-location. +- The defence graded as conforming with reasoning that says only the sweep failed and that + quotes section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence. A reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the fixture, which is the reader's miss. A reader who attributes the suppression to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) rather than [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) has read the narrowing passage of [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) as diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index c11ae32..af28752 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -45,26 +45,26 @@ Wrong readings the text invites, which count against the document on one reader: grade, not a grade of its own. Wrong if it gives a grade per part. Key quotation, section [9](../../TOOLING-SPEC.md#9-conformance): "The project-level verdict is a single grade; which part of the Toolchain satisfied each clause is evidence for that grade, not a second grade." -- **2c.** Required: (1) two, graded and declared separately; (2) neither implies the other. - Wrong if it gives one grade, or says the artefact grade follows from the project grade or the - reverse. Key quotation, [9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately): "The two verdicts MUST be graded and declared separately, and +- **2c.** Required: (1) two; (2) neither implies the other. Wrong if it gives one grade, or says + the artefact grade follows from the project grade or the reverse. That the two are graded and + declared separately is explanation; an answer that says it has stated element (1). Key quotation, [9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately): "The two verdicts MUST be graded and declared separately, and neither implies the other." ## 3. Application: the Zigpipe project -| Clause | Accepted grades | Condition on the reasoning, and what the fixture supports | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap; a reader who notes that the detector specification was not supplied and grades from this document's summary of it is correct | -| [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)`. The two are blocking, since their firing stops `zig-qa run`; a reader who grades No on that condition is correct | -| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | -| [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | -| [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | Lintomatic, then the tests, stopping on a Lintomatic failure | -| [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | No condition. `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is the reading that counts the two `(none)` routes, since 5.1 says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | -| [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration) | Yes | Derived from `lintomatic.toml`, not hand-maintained | -| [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | Yes | The `proj.*` rules appear in the same listing | -| [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) | Yes or Partial | `quality-exceptions.toml` is a defined path that `zig-qa run` loads. Partial is accepted only where the reasoning says the fixture names a place for exceptions and none for calibrations or conventions, which the project record entry also covers | -| [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | The reasoning mentions that nothing rejects a justification that could be pasted onto any entry; a justification is required and an empty one rejected | -| [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | `zig-qa list --exceptions` uses the same format and operation as the rule listing | +| Clause | Accepted grades | Condition on the reasoning | What the fixture supports | +| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | None | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap; a reader who notes that the detector specification was not supplied and grades from this document's summary of it is correct | +| [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | None. A No given on the condition that the two bundled rules are blocking is a No: their firing stops `zig-qa run`, so they are | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)` | +| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | None | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | +| [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | None | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | +| [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | None | Lintomatic, then the tests, stopping on a Lintomatic failure | +| [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | None | `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is the reading that counts the two `(none)` routes, since [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | +| [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration) | Yes | None | Derived from `lintomatic.toml`, not hand-maintained | +| [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | Yes | None | The `proj.*` rules appear in the same listing | +| [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) | Yes or Partial | For Partial: mentions that the fixture names a place for exceptions and none for calibrations or conventions | `quality-exceptions.toml` is a defined path that `zig-qa run` loads; the project record entry also covers calibrations and conventions | +| [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | Mentions that nothing rejects a generic justification, one that could be pasted onto any entry | A justification is required and an empty one rejected | +| [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | None | `zig-qa list --exceptions` uses the same format and operation as the rule listing | Section [9](../../TOOLING-SPEC.md#9-conformance) verdict: **does not conform**; a reader is correct who says so and names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses. **Not with agent support**; a reader is correct who says so because @@ -76,14 +76,15 @@ optional." Wrong readings the text invites, which count against the document on one reader: -- [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) graded Yes with reasoning that quotes only the first sentence of the clause, which has - not read the sentence on a generic justification. -- [8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) applied to Zigpipe on the basis that its tooling includes a shipped package, which has - read "where the Toolchain is one the project ships" (section [9](../../TOOLING-SPEC.md#9-conformance)) as reaching the consuming - project. -- [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) graded Yes on the grounds that Lintomatic "mostly" conforms, where the reasoning quotes - [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), which has read it as admitting a proportion. -- Zigpipe held ungradeable for want of a declaration, which has read [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) as a condition. +- [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) graded Yes with reasoning that says a justification is required or an empty one rejected + and does not mention a generic justification, which has not read the sentence on one. +- Section [8](../../TOOLING-SPEC.md#8-self-audit) said to apply to Zigpipe with reasoning that says its tooling includes a shipped + package, which has read "where the Toolchain is one the project ships" (section [9](../../TOOLING-SPEC.md#9-conformance)) as + reaching the consuming project. +- [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) graded Yes with reasoning that says Lintomatic mostly or largely conforms and quotes [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), + which has read it as admitting a proportion. +- The declaration said to affect the verdict, with reasoning that Zigpipe cannot be graded without + one, which has read [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) as a condition. A reader who grades [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) Yes because they missed the two undocumented rules has misread the fixture, which is the reader's miss. diff --git a/acceptance/toolchain/QUESTIONS.md b/acceptance/toolchain/QUESTIONS.md index 35a2b91..86de411 100644 --- a/acceptance/toolchain/QUESTIONS.md +++ b/acceptance/toolchain/QUESTIONS.md @@ -26,7 +26,7 @@ the document you are relying on. Grade the Zigpipe project's toolchain, as described in the fixture, against every MUST clause in sections [4](../../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) to [6](../../TOOLING-SPEC.md#6-the-project-record): [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record), [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes), [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure), [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them), [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration), [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones), [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences). For each give Yes, Partial or No and one line of reasoning. Then state whether the toolchain conforms under section -9, whether it conforms with agent support, and whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies to it. Finally, state +[9](../../TOOLING-SPEC.md#9-conformance), whether it conforms with agent support, and whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies to it. Finally, state whether the absence of a declaration affects the verdict. ## 4. Confusion From 8a63448820785166faeef81263fc1143b3028710 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 11:52:20 +0100 Subject: [PATCH 03/21] Acceptance process: review 2 findings applied Question 1 defined once (first six bullets, same clause counts once, a bullet on neither list not counted); the verdict items of question 3 ask for their reason and the keys state each as an accepted answer with a condition; a finding lies in the change's text also where the readers set it against that text, read against the merge base; the cohort's model is the version the editor names; a keyed miss whose fix would change an obligation is routed and waited for; the question set is the one list of load-bearing questions; ROUTED.md defines its status column and links its clauses. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 102 +++++++++++++++--------------- CONTRIBUTING.md | 6 +- acceptance/CHANGES.md | 11 ++-- acceptance/README.md | 9 +-- acceptance/ROUTED.md | 33 +++++----- acceptance/detector/KEY.md | 11 ++-- acceptance/detector/QUESTIONS.md | 2 +- acceptance/method/KEY.md | 30 ++++----- acceptance/method/QUESTIONS.md | 4 +- acceptance/toolchain/KEY.md | 22 ++++--- acceptance/toolchain/QUESTIONS.md | 4 +- 11 files changed, 123 insertions(+), 111 deletions(-) diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index c5bd8cd..39110a7 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -29,7 +29,8 @@ serves that, and a rule that stops it proving that is a defect in this file, fix ``` A reader is set aside, and its answers do not count, when its answer text is empty, when its - process reports an error, or when the model version it reports is not the cohort's model. A run + process reports an error, or when the model version it reports is not the cohort's model, the + one version the editor names for the run before its readers start. A run with a reader set aside is void: it takes a run number and has a record, and a fresh cohort runs. No reader is set aside for any other cause. @@ -54,24 +55,17 @@ asks for nothing the question does not. Each reader answers all of them. bullets. Keyed against the key's list of the document's obligations: a bullet is correct when it names one of them. -2. **Load-bearing statements.** Three or more questions, fixed per document, each asking for a - short answer and the sentence or sentences of the document it rests on, quoted. Each has one - or more parts, and the key lists the element an answer must state for each part: - - - `SPEC.md`: what a practitioner does before fixing a defect, in order; who may baseline or - suppress, and whether an agent may; what "conforms" means for a remediation and what a - verdict rests on. - - `DETECTOR-SPEC.md`: what "conforms" means and whether a tool that predates the document can - conform; what a detector must do about an inline suppression route and whether one of those - things is enough; where a bundled rule's documentation must live, at what version, and - whether it must be reachable offline. - - `TOOLING-SPEC.md`: where conformance is measured and what the toolchain is; how a grade is - assigned when parts come from different sources; how many grades a project that ships - tooling has and whether one implies the other. +2. **Load-bearing statements.** Three or more questions, fixed per document and listed in its + question set under `acceptance/`, each asking for a short answer and the sentence or + sentences of the document it rests on, quoted. Each has one or more parts, and the key lists + the element an answer must state for each part. An obligation is load-bearing, and gets a + question, when a reader who misreads it would act against the method; the editor decides, and + the pull request that adds the question says why. 3. **Application.** A fixture case supplied with the questions, graded clause by clause with - Yes, Partial or No and one line of reasoning. Keyed against the intended verdicts, with the - readings the key names as ones the text invites counting against the document. + Yes, Partial or No and one line of reasoning, then the verdicts the question set asks for, + each with its reason. Keyed against the accepted grades and answers, with the readings the + key names as ones the text invites counting against the document. 4. **Confusion.** Any passage the reader found confusing on its one reading, quoted with its clause number, and nothing else: not what would make it clearer, not a proposed change. @@ -98,9 +92,10 @@ unless the editor shows that a rule here was misapplied, and a re-mark says whic Every mark is given by a rule in this file or in the key, as follows. Where neither settles a mark, the marker records the point in the run record and marks it as the open-point rule below -says, and the editor settles it in this file or in the key, as [Changing the test](#changing-the-test) -allows, before the next run. A run record states no judgement that is not written here or in -the key; a judgement an earlier run recorded is not a rule. +says, and the editor settles it in this file or in the key, as +[Changing the test](#changing-the-test) allows, before the next run. A run record states no +judgement that is not written here or in the key; a judgement an earlier run recorded is not a +rule. - **Keyed items.** Question 1 as a whole is one keyed item, and so is each item of question 2 and each answer of question 3 for which the key gives an accepted grade or answer. @@ -111,20 +106,21 @@ the key; a judgement an earlier run recorded is not a rule. or apostrophe style, and markdown link or emphasis markup; a quotation that drops or changes a word quotes the document by its longest such run. - **Question 1** is correct when at least four bullets each name an obligation the key lists and - no bullet asserts one the key marks wrong. Several bullets that name different clauses of one - line of the key's list each count. A bullet asserting an obligation that is in neither list is - recorded as an open point for the key and is not counted either way. + no bullet asserts one the key marks wrong. Only the first six bullets are marked. Several + bullets that name different clauses of one line of the key's list each count; two that name + the same clause count once. A bullet asserting an obligation that is in neither list is not + counted either way, and is recorded as an open point for the key. - **Question 2.** An answer is the reader's sentences and its quotation read together. It is correct when it states every element the key lists as required for that item, in the document's words or the reader's own, asserts nothing the key marks wrong, and quotes the document. An element stated only in the quotation is stated. Anything in the key beyond the required elements and the wrong assertions is explanation for the marker, and an answer is never marked down for leaving it out. -- **Question 3.** Each item is correct when its grade is one the key accepts and, where the key - sets a condition on the reasoning for that grade, the reasoning meets it. A grade the key - accepts with no condition is correct whatever the reasoning. A grade given on a stated - condition is read as the grade for the case the fixture presents, where the key says which - case that is; otherwise the item is unanswered. +- **Question 3.** Each item is correct when its grade or answer is one the key accepts and, + where the key sets a condition on the reasoning for it, the reasoning meets it. A grade or + answer the key accepts with no condition is correct whatever the reasoning. A grade given on a + stated condition is read as the grade for the case the fixture presents, where the key says + which case that is; otherwise the item is unanswered. - **A miss that counts against the document.** The key names, for question 1 and for each item of question 3, the wrong readings the text invites, each as a grade or answer together with what the reasoning states or leaves out. A miss that matches one blocks whatever the count, @@ -134,22 +130,26 @@ the key; a judgement an earlier run recorded is not a rule. below; a second miss on the same item fails the threshold, and no judgement is needed. - **An unanswered question.** Each keyed item left unanswered is a miss. Question 4 left unanswered counts as no passage named. -- **Question 4.** A reader names a passage when it quotes words of the document or gives a clause - number with a description that identifies a sentence or paragraph. A remark that is not about - a passage of the document, such as that another document was not supplied, is recorded as a - note and is not a passage. Two readers name the same passage when the words or sentences they - name lie in the same paragraph; a heading, a list item, a table row and a terminology entry are - each a paragraph of their own. A passage named by two or more readers of the run is a finding. +- **Question 4.** A reader names a passage when it quotes the document, as defined above, or + gives a clause number with a description or a shorter quotation that identifies a sentence or + paragraph. A quotation whose words occur in more than one paragraph, given with no clause + number, names the paragraph where they first occur. A remark that is not about a passage of + the document, such as that another document was not supplied, is recorded as a note and is + not a passage. Two readers name the same passage when the words or sentences they name lie in + the same paragraph; a heading, a list item, a table row and a terminology entry are each a + paragraph of their own. A passage named by two or more readers of the run is a finding. - **Where a finding lies.** The record says, for each finding, whether it lies in text the change - wrote or altered: a paragraph in which the pull request, read against the branch it targets, - adds, rewords, moves or deletes a sentence, or changes a word, a term or a link. A finding + wrote or altered: a paragraph in which the pull request, read against its merge base with the + branch it targets, adds, rewords, moves or deletes a sentence, or changes a word, a term or a + link; or a paragraph the readers naming the finding set against such a paragraph. A finding there blocks the change. A finding anywhere else is a finding against the document, not against the change: it is recorded in [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not block. The change is judged on what it changed; the document is proved by the keyed questions, which every run answers over the whole of it. -- **Open points.** An open point in marking an answer is marked for the reader. An open point on - whether a reader named a passage, or whether two readers named the same one, counts the - passage. An open point on where a finding lies treats it as the change's own text. +- **Open points.** An open point in marking an answer is marked for the reader, except that a + question 1 bullet on neither list is not counted, as above. An open point on whether a reader + named a passage, or whether two readers named the same one, counts the passage. An open point + on where a finding lies treats it as the change's own text. ## Pass criteria @@ -164,7 +164,7 @@ is four of five. | A miss the key says counts against the document | **None**, whatever the count | | A passage found confusing by two or more readers, in text the change wrote or altered | **None.** It is a finding and blocks until the text is changed | | A passage found confusing by two or more readers, elsewhere | Recorded in `acceptance/ROUTED.md` and routed; does not block | -| Anything from one reader only | Recorded; applied at the editor's discretion; does not block | +| Anything else from one reader only | Recorded; applied at the editor's discretion; does not block | The asymmetry is deliberate. A load-bearing statement is the sentence the whole document exists to make, and one reader in five getting it wrong means one agent in five will act on the wrong @@ -202,11 +202,12 @@ changed what the cohort is asked, not how strictly an answer is marked. ## After a failure Fix the text the run's misses and findings name and run a fresh cohort; readers are never reused. -A keyed miss is fixed in the text that carries the obligation, wherever it lies, since the keyed -questions prove the document and a document one reader in five misreads is not merged to, whoever -wrote the sentence. A finding on text the change wrote or altered is fixed there. A finding -elsewhere goes to `acceptance/ROUTED.md` and is not fixed in this change. The number of runs it -took is recorded; a document that needed three runs is telling its editor something. +A failed keyed item is fixed in the text that carries the obligation, wherever it lies, since the +keyed questions prove the document as a whole, whoever wrote the sentence; where that fix would +change an obligation, it is routed to a change of its own, and this change waits for it. A +finding on text the change wrote or altered is fixed there. A finding elsewhere goes to +`acceptance/ROUTED.md` and is not fixed in this change. The number of runs it took is recorded; +a document that needed three runs is telling its editor something. ## When it runs @@ -239,11 +240,12 @@ and the instruction a delegated marker was given are posted in one or more comme request, linked from the record and never edited, hidden or deleted after posting. The changelog line for the change names every run made on it, the passing run last. CI refuses a -specification change whose pull request adds no run file; it checks for at least one, and a pull request that changes two -documents adds a run for each. Each published version's changelog entry carries the lines of the -changes it contains, so a version's cohort record is the set of runs those lines name. A version -with no such line has not been accepted and MUST NOT be described as published. The question sets -and answer keys live under `acceptance/` in this repository so that a run can be repeated and +specification change whose pull request adds no run file; it checks for at least one, and a pull +request that changes two documents adds a run for each. Each published version's changelog entry +carries the lines of the changes it contains, so a version's cohort record is the set of runs +those lines name. A version with no such line has not been accepted and MUST NOT be described as +published. The question sets and answer keys live under `acceptance/` in this repository so that +a run can be repeated and compared. `acceptance/ROUTED.md` lists every finding routed from a run, open until a change that fixes diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 013bb9d..733e05b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -29,9 +29,9 @@ review. It is a patch version when released. SHOULD or MAY in a numbered section or appendix. It needs everything a clarity edit needs, plus the five tests in SCOPE.md answered in the pull request, and it is at least a minor version when released. `tools/changes.py` tells the two apart mechanically and CI runs it on every pull -request. A change to the process files ACCEPTANCE.md names, this file among them, is neither: -ACCEPTANCE.md says how it is made, and it is recorded in `acceptance/CHANGES.md`, not in -CHANGELOG.md. +request. A change to the acceptance process, as ACCEPTANCE.md defines one under "Changes to +this process", is neither: ACCEPTANCE.md says how it is made, and it is recorded in +`acceptance/CHANGES.md`, not in CHANGELOG.md. ## What a pull request contains diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md index cc01848..29a20db 100644 --- a/acceptance/CHANGES.md +++ b/acceptance/CHANGES.md @@ -1,13 +1,14 @@ # Changes to the acceptance process -Changes to the process files that [ACCEPTANCE.md](../ACCEPTANCE.md) names, recorded here as it -describes: a line is added at the top for each change, naming its pull request. The process is -not a specification and is not versioned, and a change to it is not tested by a cohort. Changes -to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). +Changes to the acceptance process, as [ACCEPTANCE.md](../ACCEPTANCE.md) defines one under +"Changes to this process", recorded here as it describes: a line is added at the top for each +change, naming its pull request. The process is not a specification and is not versioned, and a +change to it is not tested by a cohort. Changes to the three specifications are recorded in +[CHANGELOG.md](../CHANGELOG.md). - Pull request NNN: the test is made to prove what it exists to prove, the clarity of the document to a cold reader, after thirty-three runs on two changes (19 to 30, 31 to 33 and 35 - to 52) in which keyed answers reached five of five and the runs still failed. Four defects in + to 52) in which keyed answers reached five of five and the runs still failed. Five defects in the process were found. Keys required elements the question never asked for (detector 2b's "need not remove or forbid", detector 2c's "at a version tracked together", and on pull request 14 method 2d's order of the kinds), so readers who answered the question as put were diff --git a/acceptance/README.md b/acceptance/README.md index dcf39c0..aaeba1f 100644 --- a/acceptance/README.md +++ b/acceptance/README.md @@ -25,8 +25,8 @@ in [CHANGES.md](CHANGES.md). 4. The result is recorded as a file under [runs/](runs/), stating what `ACCEPTANCE.md` lists, with the answer sheets posted on the pull request and linked from it, and the change's `Unreleased` changelog line names that file. A run that fails is followed by a change to the - text the findings name and a fresh cohort; a finding on text the change did not write or - alter is added to [ROUTED.md](ROUTED.md) instead. + text its misses and findings name and a fresh cohort; a finding on text the change did not + write or alter is added to [ROUTED.md](ROUTED.md) instead. ## How a key is written @@ -50,8 +50,9 @@ A key quotes the document. When a clause is reworded, the key line that quotes i the same commit, and the change is one the acceptance run itself then tests. A key that no longer matches the document is a defect in this directory, not a reason to mark a reader wrong. -A key, a question set or a fixture changes in one of two ways, as `ACCEPTANCE.md` says under -"Changing the test", and always before the readers of the next run start: +A key, a question set or a fixture changes by one of two routes, as `ACCEPTANCE.md` says under +"Changing the test", always before the readers of the next run start, and an open point a run +recorded is settled by whichever route it needs: - **In a specification pull request, under step 4 of `CONTRIBUTING.md`**: a quotation follows the text it quotes, or a keyed item is added for a new load-bearing obligation together with diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index eb35815..60fc616 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -8,25 +8,26 @@ editor reads this list before cutting a release. A finding is open until a change that reworks its passage passes a cohort, when the line names that run and moves below the rule, or until the editor closes it with a reason recorded here. A -finding raised again by a later run is noted on its line, not listed twice. The rows below were -raised by runs 31 to 52 under the earlier rules, on untouched text of the change those runs -tested, and are listed here so that they are not lost. +finding raised again by a later run is noted on its line, not listed twice. The status column +holds `open`, or the `editor/` branch or pull request number of the change that is +reworking the passage. The rows below were raised by runs 31 to 52 under the earlier rules, on +untouched text of the change those runs tested, and are listed here so that they are not lost. ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | -------------------------------------- | -| `TOOLING-SPEC.md` | 9 and 9.1 | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 2, 4.2, 8.1 | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 9.2 | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 4.3 | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | run 33 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 4.1 | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | 5.3 | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | open | -| `DETECTOR-SPEC.md` | 7.1 | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | 8.1 | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | 5.3 | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | -| `SPEC.md` | 1 (Owner) | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | open | -| `SPEC.md` | 4 | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | open; a method question for the editor | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | ---------------------------------------- | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | run 33 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | ## Closed diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index 2f6fd96..aaddb69 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -8,7 +8,7 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting an obligation the document does not state, is a correct restatement. +bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. - Support bespoke rules written by the project ([4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it)). - Provide a harness that runs a single rule against supplied code ([4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code)). @@ -78,11 +78,12 @@ No grade above carries a condition on the reasoning: the accepted grade is corre reasoning, and a reader who adds a caveat about what the fixture does not say and still gives the accepted grade is correct. -Section [8](../../DETECTOR-SPEC.md#8-conformance) verdict: **does not conform**; a reader is correct who says so and names [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), the -two undocumented bundled rules, or partial conformance not counting, as the reason. +The two answers the question then asks for, each with the condition its reason must meet: -Clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation): **does not change the verdict**; a reader is correct who says so and gives as the -reason that it is a SHOULD, or that the MUST which fails is [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together). +- **Conformance under section [8](../../DETECTOR-SPEC.md#8-conformance).** Accepted answer: does not conform. Condition: the reason + names [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), the two undocumented bundled rules, or partial conformance not counting. +- **Whether clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation) changes the verdict.** Accepted answer: it does not. Condition: the reason + says that it is a SHOULD, or that the MUST which fails is [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together). Wrong readings the text invites, which count against the document on one reader: diff --git a/acceptance/detector/QUESTIONS.md b/acceptance/detector/QUESTIONS.md index 2666b2f..5c288f8 100644 --- a/acceptance/detector/QUESTIONS.md +++ b/acceptance/detector/QUESTIONS.md @@ -25,7 +25,7 @@ the document you are relying on. Grade Lintomatic 4.2, as described in the fixture, against every MUST clause in sections [4](../../DETECTOR-SPEC.md#4-authoring-rules) to [7](../../DETECTOR-SPEC.md#7-suppression): [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it), [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), [4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding), [5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure), [5.2](../../DETECTOR-SPEC.md#52-the-detector-must-support-invocation-over-a-subset-at-minimum-a-single-file), [5.3](../../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran), [5.4](../../DETECTOR-SPEC.md#54-a-finding-must-not-be-reportable-only-through-a-hosted-service-licence-tier-or-ci-only-mode-the-practitioner-cannot-invoke-locally), [6.1](../../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation), [6.2](../../DETECTOR-SPEC.md#62-resolution-of-a-bundled-rules-identifier-must-work-from-the-installed-copy-without-network-access), [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together), [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable). For each give Yes, Partial or No and one line of reasoning. Then state whether Lintomatic conforms under section [8](../../DETECTOR-SPEC.md#8-conformance), and why. Finally, -state whether clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation) changes the verdict. +state whether clause [6.4](../../DETECTOR-SPEC.md#64-the-detector-should-fail-its-own-release-if-a-bundled-rule-lacks-resolvable-documentation) changes the verdict, and why. ## 4. Confusion diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index 36bfafd..b756f47 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -8,7 +8,7 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting an obligation the document does not state, is a correct restatement. +bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. - Attribute the defect to a class, bounded on both sides, with the hazard stated ([3.1](../../SPEC.md#31-attribute-the-defect-to-a-class)). - Express the class as a rule in a detector that reads code, never a test ([3.2](../../SPEC.md#32-build-the-net)). @@ -43,7 +43,7 @@ Wrong readings the text invites, which count against the document on one reader: quotation, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): "An Agent MUST NOT Baseline, suppress or knowingly leave an Instance unfixed on its own authority, whatever the Instance count turns out to be." "The owner" names the defined term and states that the decision is a human's; an answer need not add "a human" - to be correct. The default owner paragraph in the terminology entry for Owner still requires a + to be correct. The default-owner sentences of the terminology entry for Owner still require a recorded decision, so an answer naming whoever instructed the practitioner, for a project that has recorded nobody, names the owner and is correct. - **2c.** Required: (1) all six clauses of section [3](../../SPEC.md#3-the-method) were followed for that defect; (2) the verdict @@ -61,21 +61,22 @@ Wrong readings the text invites, which count against the document on one reader: | ----------------------------------------------------------------------------------------- | ------------------ | ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) | Yes, Partial or No | None | Class, hazard and the next wider rule are named; Partial or No is the reading that no independent search technique is recorded, which step 2 and the record [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class) requires both call for | | [3.2](../../SPEC.md#32-build-the-net) | Yes | None | A rule in the project's static analyser, reading code, not a test | -| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes | None | Commit B is red on the fixture and the originating instance and survives as its own commit | +| [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) | Yes, Partial or No | For Partial or No: attributes the inline ignore to this clause, a fair reading of Part B | Commit B is red on the fixture and the originating instance and survives as its own commit | | [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) | No or Partial | Mentions the instance left unfixed, the inline ignore, or `Report::render()` | Five findings were recorded, but one instance was left unfixed behind an inline ignore on the agent's own authority | | [3.5](../../SPEC.md#35-enforce-permanently-and-block) | Partial or No | Mentions the suppression, the inline ignore, or that no recorded decision covers it | The rule is permanent and blocking, but the suppression was not a recorded decision. [3.5](../../SPEC.md#35-enforce-permanently-and-block): "neither does a Rule whose green run depends on a Suppression that no recorded decision covers" | | [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) | No or Partial | Mentions the wiki, or that the documentation is not versioned or co-located with the rule | The message is terse and carries an identifier, but the documentation lives on a wiki, not versioned with the rule | -Section [7](../../SPEC.md#7-conformance): the **remediation does not conform**; a reader is correct who says so and names -[3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed. The **defence does not conform** either; a reader is -correct who says so and names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the reasons, or [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which -section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent. Other clauses named -alongside are not marked. +The three answers the question then asks for, each with the condition its reason must meet: -Section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom): the **inline ignore on `Report::render()`** was the owner's decision, not the -practitioner's. A reader who names it is correct; any other decision named alongside it is not -marked. A reader who names only the decision to leave the wider rule unbuilt, or neither, is -wrong. +- **The remediation under section [7](../../SPEC.md#7-conformance).** Accepted answer: does not conform. Condition: the reason + names [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed; other clauses named alongside are not marked. +- **The defence alone.** Accepted answer: does not conform. Condition: the reason names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation), or + [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also + requires to be absent; other clauses named alongside are not marked. +- **The decision section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) says was not the practitioner's.** Accepted answer: the inline ignore on + `Report::render()`, which was the owner's decision. Condition: none; any other decision named + alongside it is not marked. An answer naming only the decision to leave the wider rule unbuilt, + or naming none, is wrong. Wrong readings the text invites, which count against the document on one reader: @@ -86,9 +87,8 @@ Wrong readings the text invites, which count against the document on one reader: - The defence graded as conforming with reasoning that says only the sweep failed and that quotes section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence. -A reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the fixture, which is the reader's miss. A reader who -attributes the suppression to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) rather than [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) has read the narrowing passage of [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) as -covering it, which is a fair reading and is not marked wrong. +A reader who grades [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) Yes has misread the fixture, which is the reader's miss, whether or not +the inline ignore was attributed to [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire) instead. ## 4. Confusion diff --git a/acceptance/method/QUESTIONS.md b/acceptance/method/QUESTIONS.md index 34fa82d..9a2f393 100644 --- a/acceptance/method/QUESTIONS.md +++ b/acceptance/method/QUESTIONS.md @@ -24,8 +24,8 @@ the document you are relying on. Grade the ledger remediation, as described in the fixture, against each of the six clauses of section [3](../../SPEC.md#3-the-method): [3.1](../../SPEC.md#31-attribute-the-defect-to-a-class), [3.2](../../SPEC.md#32-build-the-net), [3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire), [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), [3.5](../../SPEC.md#35-enforce-permanently-and-block), [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation). For each give Yes, Partial or No and one line of -reasoning. Then state whether the remediation conforms under section [7](../../SPEC.md#7-conformance), whether the defence -alone conforms, and which decision in the fixture, if any, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) says was not the +reasoning. Then state whether the remediation conforms under section [7](../../SPEC.md#7-conformance) and why, whether the defence +alone conforms and why, and which decision in the fixture, if any, section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom) says was not the practitioner's to take. ## 4. Confusion diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index af28752..7353852 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -8,7 +8,7 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting an obligation the document does not state, is a correct restatement. +bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. - Route defences only through detectors that conform to the detector specification, wrapping permitted ([4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification)). @@ -66,13 +66,19 @@ Wrong readings the text invites, which count against the document on one reader: | [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | Partial or No | Mentions that nothing rejects a generic justification, one that could be pasted onto any entry | A justification is required and an empty one rejected | | [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Yes | None | `zig-qa list --exceptions` uses the same format and operation as the rule listing | -Section [9](../../TOOLING-SPEC.md#9-conformance) verdict: **does not conform**; a reader is correct who says so and names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or -[4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses. **Not with agent support**; a reader is correct who says so because -section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, there being no summary. **Section [8](../../TOOLING-SPEC.md#8-self-audit) does not apply**; a reader is correct -who says so because Zigpipe ships nothing to other projects, or because section [8](../../TOOLING-SPEC.md#8-self-audit) bears on -zig-qa's own artefact grade and not on Zigpipe. **The absence of a declaration does not affect -the verdict**; a reader is correct who says so. Key quotation, [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance): "The declaration is -optional." +The four answers the question then asks for, each with the condition its reason must meet: + +- **Conformance under section [9](../../TOOLING-SPEC.md#9-conformance).** Accepted answer: does not conform. Condition: the reason + names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses; other clauses named alongside are not marked. +- **Conformance with agent support.** Accepted answer: no. Condition: the reason says that + section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, or that there is no summary. +- **Whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies.** Accepted answer: it does not. Condition: the reason says that + Zigpipe ships nothing to other projects, or that section [8](../../TOOLING-SPEC.md#8-self-audit) bears on zig-qa's own artefact + grade and not on Zigpipe. +- **Whether the absence of a declaration affects the verdict.** Accepted answer: it does not. + Condition: the reason says that the declaration is optional, or not a condition of + conformance, or that a verdict rests on the evidence. Key quotation, [9.2](../../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance): "The declaration is + optional." Wrong readings the text invites, which count against the document on one reader: diff --git a/acceptance/toolchain/QUESTIONS.md b/acceptance/toolchain/QUESTIONS.md index 86de411..d4e5cd7 100644 --- a/acceptance/toolchain/QUESTIONS.md +++ b/acceptance/toolchain/QUESTIONS.md @@ -26,8 +26,8 @@ the document you are relying on. Grade the Zigpipe project's toolchain, as described in the fixture, against every MUST clause in sections [4](../../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) to [6](../../TOOLING-SPEC.md#6-the-project-record): [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record), [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes), [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure), [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them), [5.2](../../TOOLING-SPEC.md#52-the-listing-must-be-derived-from-the-active-configuration), [5.3](../../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones), [6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences). For each give Yes, Partial or No and one line of reasoning. Then state whether the toolchain conforms under section -[9](../../TOOLING-SPEC.md#9-conformance), whether it conforms with agent support, and whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies to it. Finally, state -whether the absence of a declaration affects the verdict. +[9](../../TOOLING-SPEC.md#9-conformance) and why, whether it conforms with agent support and why, and whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies to it +and why. Finally, state whether the absence of a declaration affects the verdict, and why. ## 4. Confusion From 4cc568884c13ed12c7b31a56f41459e26519de30 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 11:58:31 +0100 Subject: [PATCH 04/21] Acceptance process: review 3 findings applied Two routes for changing the test, with a changed or removed obligation's key item following the text in the specification pull request; an open point holds up only its own pull request; a fresh run follows a change to the document, not to the record; detector 2b names both things; the section 7 and section 9 reasons accept every failing clause; the restatement rule reads plainly; ROUTED.md defines a finding as ACCEPTANCE.md does. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 30 ++++++++++++++++-------------- CONTRIBUTING.md | 4 ++-- acceptance/README.md | 20 +++++++++++++------- acceptance/ROUTED.md | 6 +++--- acceptance/detector/KEY.md | 13 +++++++------ acceptance/method/KEY.md | 7 ++++--- acceptance/toolchain/KEY.md | 10 ++++++---- 7 files changed, 51 insertions(+), 39 deletions(-) diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index 39110a7..d1bd6af 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -141,8 +141,8 @@ rule. - **Where a finding lies.** The record says, for each finding, whether it lies in text the change wrote or altered: a paragraph in which the pull request, read against its merge base with the branch it targets, adds, rewords, moves or deletes a sentence, or changes a word, a term or a - link; or a paragraph the readers naming the finding set against such a paragraph. A finding - there blocks the change. A finding anywhere else is a finding against the document, not + link; or a paragraph that any reader naming the finding sets against such a paragraph. A + finding there blocks the change. A finding anywhere else is a finding against the document, not against the change: it is recorded in [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not block. The change is judged on what it changed; the document is proved by the keyed questions, which every run answers over the whole of it. @@ -178,15 +178,18 @@ fresh cohort names by fewer than two is clear. Nothing in this test is changed to let a particular run through. A threshold, a question, a key, a fixture or a rule here is changed only before a run's readers start, never changes a mark -already given, and is changed in one of three ways, each recorded with its reason: +already given, and is changed by one of two routes, each recorded with its reason: -- A key's quotation follows the text it quotes, and a key gains an item for a new load-bearing - obligation, with the question that asks for it, under step 4 of `CONTRIBUTING.md`, in the - specification pull request that makes the change; `acceptance/README.md` says what that may - and may not do. -- A threshold, a fixture, a rule here, what an existing key item requires, or the wording of a - question changes by a change to this process, with its own pull request. -- An open point a run recorded is settled by either route, whichever it needs. +- In the specification pull request that changes the text, under step 4 of `CONTRIBUTING.md`: + a key's quotation follows the text it quotes; a key gains an item, with the question that asks + for it, for a new load-bearing obligation; and where the change alters or removes the + obligation an existing item tests, the item and its question follow the text, since that is + the text changing and not the test. `acceptance/README.md` says what that may and may not do. +- By a change to this process, with its own pull request: a threshold, a fixture, a rule here, + what an existing key item requires of an unchanged obligation, or the wording of a question. + +An open point a run recorded is settled by whichever route it needs, before the next run of the +pull request whose run recorded it; a run of another pull request is not held up by it. A question is reworded only so that it asks for what its key requires, and a key's required elements are never more than its question asks for. A run made after such a change is a fresh @@ -214,9 +217,9 @@ a document that needed three runs is telling its editor something. On the final text of each pull request, after the editor's rewrite and any key update, one cohort per changed document. It is the last gate before the merge to `next`, so that every change on `next` is one the cohort has accepted, and the release branch needs no cohort of its own unless it -changes specification text. A fresh cohort runs after every change to the pull request's own +changes specification text. A fresh cohort runs after every change to the changed document's text that follows a run, and after every void run; a text that failed is never run again -unchanged. A version number or date that `PUBLISHING.md` or step 5 of `CONTRIBUTING.md` has the +unchanged, and a commit that adds the run record or the changelog line changes no document. A version number or date that `PUBLISHING.md` or step 5 of `CONTRIBUTING.md` has the editor set is not such a change, and neither is a merge of `next` into the branch that brings in text another cohort accepted and alters no sentence this change wrote. @@ -245,8 +248,7 @@ request that changes two documents adds a run for each. Each published version's carries the lines of the changes it contains, so a version's cohort record is the set of runs those lines name. A version with no such line has not been accepted and MUST NOT be described as published. The question sets and answer keys live under `acceptance/` in this repository so that -a run can be repeated and -compared. +a run can be repeated and compared. `acceptance/ROUTED.md` lists every finding routed from a run, open until a change that fixes its passage passes a cohort, or until the editor closes it with a recorded reason. The editor diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 733e05b..ea5a525 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -69,8 +69,8 @@ survives, the change is yours. changed document. Its record is committed under `acceptance/runs/`, the `Unreleased` changelog line names it, and CI refuses a specification change that lacks one. For an obligation, the editor sets the document's header to `-dev, unpublished` in the - same commit if it still names a published version. A change to the pull request's own text - after the run means a fresh run, as ACCEPTANCE.md says under "When it runs". A finding the + same commit if it still names a published version. A change to the document's text after the + run means a fresh run, as ACCEPTANCE.md says under "When it runs". A finding the run routes to `acceptance/ROUTED.md` is not fixed in this pull request. 6. **Merge** into `next` by merge commit, and the branch is deleted. diff --git a/acceptance/README.md b/acceptance/README.md index aaeba1f..ec5c3b2 100644 --- a/acceptance/README.md +++ b/acceptance/README.md @@ -32,12 +32,15 @@ in [CHANGES.md](CHANGES.md). A key says, for each keyed item, exactly what is marked and nothing a marker has to guess: -- **Question 1**: the obligations a bullet may name, and the assertions that make a bullet wrong. +- **Question 1**: the obligations a bullet may name, the assertions that make a bullet wrong, + and the wrong readings the text invites, if any. - **Question 2**: for each item, the elements an answer is required to state, numbered, one for each part of the question; the assertions that make an answer wrong; and the sentence or - sentences of the document the answer rests on, quoted verbatim. The required elements are - what the question asks for, no more: an element the question does not ask for is not - required, however load-bearing, until the question is reworded to ask for it. + sentences of the document the answer rests on, quoted verbatim, which show the marker where + the elements come from and which `spec-qa.py` checks against the document; a reader's + quotation need not be that one. The required elements are what the question asks for, no + more: an element the question does not ask for is not required, however load-bearing, until + the question is reworded to ask for it. - **Question 3**: for each item, the grade or grades accepted, any condition on the reasoning, and the wrong readings the text invites, which count against the document when one reader takes them. @@ -55,9 +58,12 @@ A key, a question set or a fixture changes by one of two routes, as `ACCEPTANCE. recorded is settled by whichever route it needs: - **In a specification pull request, under step 4 of `CONTRIBUTING.md`**: a quotation follows - the text it quotes, or a keyed item is added for a new load-bearing obligation together with - the question that asks for it. The pull request's own changelog line records it. Such an - update does not change what an existing item requires, nor an existing question's wording. + the text it quotes; a keyed item is added for a new load-bearing obligation together with the + question that asks for it; or, where the change alters or removes the obligation an existing + item tests, that item and its question follow the text, which is the text changing and not + the test. The pull request's own changelog line records it. Such an update does not change + what an item requires of an obligation the change leaves as it was, nor the wording of a + question whose obligation is unchanged. - **In a process change, with its own pull request**: what an existing item requires, the wording of a question, a fixture, or the rules of marking. It is recorded in [CHANGES.md](CHANGES.md) with its reason. diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 60fc616..24beef9 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -1,8 +1,8 @@ # Passages routed out of a cohort -A finding is a passage two or more readers of one cohort named as confusing, under question 4, -that lies outside the text the change being tested wrote or altered. `ACCEPTANCE.md` routes such -a finding here rather than blocking the change whose cohort raised it, so that each change is +A finding is a passage two or more readers of one cohort named as confusing under question 4. +`ACCEPTANCE.md` routes here a finding that lies outside the text the change being tested wrote +or altered, rather than blocking the change whose cohort raised it, so that each change is judged on what it changed and the document is still made clearer, one change at a time. The editor reads this list before cutting a release. diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index aaddb69..5b4d4aa 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -7,8 +7,8 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement -A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. +A bullet is correct when it names one of these obligations. Four or more correct bullets, with +no bullet making an assertion listed below as wrong, is a correct restatement. - Support bespoke rules written by the project ([4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it)). - Provide a harness that runs a single rule against supplied code ([4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code)). @@ -39,10 +39,11 @@ Wrong readings the text invites, which count against the document on one reader: MAY declare the version of this document the detector conforms to". The reason, that conformance is judged on evidence and a declaration is optional, is explanation: an answer that gives it, or gives it instead of a plain yes, states element (2). -- **2b.** Required: (1) it must let the project switch the route off in its configuration, or - make the route detectable (by a rule the project can write, or by a mechanical check the - detector documents; naming either means of detection, or just "detectable", states this); - (2) yes, either one alone is enough. Wrong if it says the detector must remove or forbid the +- **2b.** Required: (1) the two things, both named: let the project switch the route off in its + configuration, or make the route detectable (by a rule the project can write, or by a + mechanical check the detector documents; naming either means of detection, or just + "detectable", states that alternative); an answer naming only one of the two things misses + this element; (2) yes, either one alone is enough. Wrong if it says the detector must remove or forbid the route, or must require a reason ([7.2](../../DETECTOR-SPEC.md#72-an-inline-suppression-route-should-require-a-written-reason) is SHOULD). Key quotation, [7.1](../../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable): "For each one it MUST do at least one of two things, and either alone satisfies this clause: make the route disableable by configuration; or make the route detectable, by a Rule the project can write in diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index b756f47..a990c7b 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -7,8 +7,8 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement -A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. +A bullet is correct when it names one of these obligations. Four or more correct bullets, with +no bullet making an assertion listed below as wrong, is a correct restatement. - Attribute the defect to a class, bounded on both sides, with the hazard stated ([3.1](../../SPEC.md#31-attribute-the-defect-to-a-class)). - Express the class as a rule in a detector that reads code, never a test ([3.2](../../SPEC.md#32-build-the-net)). @@ -69,7 +69,8 @@ Wrong readings the text invites, which count against the document on one reader: The three answers the question then asks for, each with the condition its reason must meet: - **The remediation under section [7](../../SPEC.md#7-conformance).** Accepted answer: does not conform. Condition: the reason - names [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed; other clauses named alongside are not marked. + names [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), [3.5](../../SPEC.md#35-enforce-permanently-and-block) or [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation) among the clauses not followed; other clauses named alongside are not + marked. - **The defence alone.** Accepted answer: does not conform. Condition: the reason names [3.6](../../SPEC.md#36-make-the-failure-message-terse-and-point-it-at-real-documentation), or [3.5](../../SPEC.md#35-enforce-permanently-and-block)'s unrecorded suppression, which section [7](../../SPEC.md#7-conformance)'s definition of a conforming defence also requires to be absent; other clauses named alongside are not marked. diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index 7353852..2d2bea3 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -7,8 +7,8 @@ update the quotation here in the same commit. `ACCEPTANCE.md` says how each item ## 1. Restatement -A bullet is correct when it names one of these obligations. Four or more correct bullets, and no -bullet asserting one the list of wrong assertions below marks wrong, is a correct restatement. +A bullet is correct when it names one of these obligations. Four or more correct bullets, with +no bullet making an assertion listed below as wrong, is a correct restatement. - Route defences only through detectors that conform to the detector specification, wrapping permitted ([4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification)). @@ -69,9 +69,11 @@ Wrong readings the text invites, which count against the document on one reader: The four answers the question then asks for, each with the condition its reason must meet: - **Conformance under section [9](../../TOOLING-SPEC.md#9-conformance).** Accepted answer: does not conform. Condition: the reason - names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) or [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) among the failing clauses; other clauses named alongside are not marked. + names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) or [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) among the failing clauses; other clauses named alongside are not + marked. - **Conformance with agent support.** Accepted answer: no. Condition: the reason says that - section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, or that there is no summary. + section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, or that there is no summary, or that the toolchain does not conform + under section [9](../../TOOLING-SPEC.md#9-conformance) at all. - **Whether section [8](../../TOOLING-SPEC.md#8-self-audit) applies.** Accepted answer: it does not. Condition: the reason says that Zigpipe ships nothing to other projects, or that section [8](../../TOOLING-SPEC.md#8-self-audit) bears on zig-qa's own artefact grade and not on Zigpipe. From c31d49a0ac8e66931784ac09ef342d78347b0957 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 12:00:29 +0100 Subject: [PATCH 05/21] Acceptance process: CHANGES.md names pull request 22 Co-Authored-By: Claude Fable 5.1 --- acceptance/CHANGES.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md index 29a20db..daa3d44 100644 --- a/acceptance/CHANGES.md +++ b/acceptance/CHANGES.md @@ -6,7 +6,8 @@ change, naming its pull request. The process is not a specification and is not v change to it is not tested by a cohort. Changes to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). -- Pull request NNN: the test is made to prove what it exists to prove, the clarity of the +- [Pull request #22](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/22): + the test is made to prove what it exists to prove, the clarity of the document to a cold reader, after thirty-three runs on two changes (19 to 30, 31 to 33 and 35 to 52) in which keyed answers reached five of five and the runs still failed. Five defects in the process were found. Keys required elements the question never asked for (detector 2b's From e6bdf402e051155cf4b7752f62ad4dd8a5ef41a1 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 12:57:49 +0100 Subject: [PATCH 06/21] Keys and routed findings: open points from runs 53 to 59 The method key lists fixing the original defect with a reproducing test among the obligations, says a bullet on the verdict resting on reproduction describes the reviewer, and says that an answer to 2a naming no step does not state the steps. The toolchain key lists the governing principle, says a bullet on how conformance is measured is neither correct nor wrong, and accepts a section 9 reason that points to the grades given. acceptance/ROUTED.md gains the findings runs 53 to 59 routed, and the runs that raised existing rows again. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 33 +++++++++++++++++++-------------- acceptance/method/KEY.md | 9 +++++++-- acceptance/toolchain/KEY.md | 12 +++++++++--- 3 files changed, 35 insertions(+), 19 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 24beef9..901caed 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -10,24 +10,29 @@ A finding is open until a change that reworks its passage passes a cohort, when that run and moves below the rule, or until the editor closes it with a reason recorded here. A finding raised again by a later run is noted on its line, not listed twice. The status column holds `open`, or the `editor/` branch or pull request number of the change that is -reworking the passage. The rows below were raised by runs 31 to 52 under the earlier rules, on +reworking the passage. The rows raised by runs 31 to 52 were raised under the earlier rules, on untouched text of the change those runs tested, and are listed here so that they are not lost. ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | ---------------------------------------- | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | run 33 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- | ---------------------------------------- | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59 | `open` | ## Closed diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index a990c7b..86cf8ff 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -21,9 +21,13 @@ no bullet making an assertion listed below as wrong, is a correct restatement. than only mentioning it (section [2](../../SPEC.md#2-when-the-method-applies)). - Take a baseline, a suppression or an instance left unfixed to the owner, never on the practitioner's own authority (section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom)). +- Fix the original defect conventionally, with a test that reproduces it, once the defence is in + place (sections [2](../../SPEC.md#2-when-the-method-applies) and [6](../../SPEC.md#6-relationship-to-test-driven-development)). Wrong if it asserts: the practitioner fixes the reported instance first and defends afterwards; -a test may serve as the detector; an agent may baseline the remaining instances. +a test may serve as the detector; an agent may baseline the remaining instances. A bullet that +says the verdict on a remediation rests on reproduction (section [7](../../SPEC.md#7-conformance)) describes the reviewer, +not the practitioner, and is neither correct nor wrong. Wrong readings the text invites, which count against the document on one reader: none named. @@ -36,7 +40,8 @@ Wrong readings the text invites, which count against the document on one reader: order."; [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance): "The Practitioner MUST run the Rule across the entire codebase and record the total Instance count before fixing anything, and MUST then fix every Instance found rather than only the one". An answer that names the steps by their clause numbers in order states - both elements. + both elements; one that says only that section [3](../../SPEC.md#3-the-method)'s six clauses are followed in order, + naming no step, does not state element (1). - **2b.** Required: (1) the owner of the codebase, the human who holds that authority; (2) no, an agent may not do so on its own authority. Wrong if it lets the practitioner or an agent decide because the count was small, because the project recorded no owner, or at all. Key diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index 2d2bea3..e0388d4 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -24,11 +24,16 @@ no bullet making an assertion listed below as wrong, is a correct restatement. same way as the defences ([6.1](../../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself) to [6.3](../../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences)). - A shipped toolchain also fails its release on undocumented bundled defences and runs its own defences on its own source ([8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation), [8.2](../../TOOLING-SPEC.md#82-a-shipped-toolchain-must-run-its-own-bundled-defences-on-its-own-source)). +- Make everything the method requires of a practitioner possible without the project building + the mechanism first (section [1](../../TOOLING-SPEC.md#1-what-this-document-is-for), the governing principle). Wrong if it asserts: the toolchain must be a single product; a declaration is required to conform; the agent summary (section [7](../../TOOLING-SPEC.md#7-agent-context)) is a MUST. A bullet that calls agent support or the self-audit a further condition "where it applies", or says a declaration is not a condition, -asserts no obligation and is neither correct nor wrong. +asserts no obligation and is neither correct nor wrong. So is a bullet that states how +conformance is measured rather than what is required: the toolchain judged as one assembled +whole at the project level (sections [1](../../TOOLING-SPEC.md#1-what-this-document-is-for) and [9](../../TOOLING-SPEC.md#9-conformance)), the project-level verdict a single grade +(section [9](../../TOOLING-SPEC.md#9-conformance)), or a shipped artefact graded separately ([9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately)). Wrong readings the text invites, which count against the document on one reader: none named. @@ -69,8 +74,9 @@ Wrong readings the text invites, which count against the document on one reader: The four answers the question then asks for, each with the condition its reason must meet: - **Conformance under section [9](../../TOOLING-SPEC.md#9-conformance).** Accepted answer: does not conform. Condition: the reason - names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) or [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) among the failing clauses; other clauses named alongside are not - marked. + names [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification), [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) or [6.2](../../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) among the failing clauses, or points to the grades the reader gave above, as "the + clauses graded No", where those grades have one of the three at No or Partial; other clauses + named alongside are not marked. - **Conformance with agent support.** Accepted answer: no. Condition: the reason says that section [7](../../TOOLING-SPEC.md#7-agent-context) is not met, or that there is no summary, or that the toolchain does not conform under section [9](../../TOOLING-SPEC.md#9-conformance) at all. From 4081aa45e771f14fb832e708ba612c88f5008d8e Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:16:08 +0100 Subject: [PATCH 07/21] Toolchain key and routed findings: open points from runs 61 and 62 The toolchain key counts a bullet on not describing partial conformance as conformance, and says a bullet calling section 7 recommended or a SHOULD asserts no obligation. acceptance/ROUTED.md gains clause 3.4's "tremendous" sentence from run 61 and notes that run on the two rows it raised again. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 5 +++-- acceptance/toolchain/KEY.md | 6 ++++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 901caed..76acd8f 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -31,8 +31,9 @@ untouched text of the change those runs tested, and are listed here so that they | `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56 | `open` | | `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | | `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | ## Closed diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index e0388d4..aadab7a 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -26,11 +26,13 @@ no bullet making an assertion listed below as wrong, is a correct restatement. defences on its own source ([8.1](../../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation), [8.2](../../TOOLING-SPEC.md#82-a-shipped-toolchain-must-run-its-own-bundled-defences-on-its-own-source)). - Make everything the method requires of a practitioner possible without the project building the mechanism first (section [1](../../TOOLING-SPEC.md#1-what-this-document-is-for), the governing principle). +- Not describe partial conformance as conformance (section [9](../../TOOLING-SPEC.md#9-conformance)). Wrong if it asserts: the toolchain must be a single product; a declaration is required to conform; the agent summary (section [7](../../TOOLING-SPEC.md#7-agent-context)) is a MUST. A bullet that calls agent support or the -self-audit a further condition "where it applies", or says a declaration is not a condition, -asserts no obligation and is neither correct nor wrong. So is a bullet that states how +self-audit a further condition "where it applies", says that section [7](../../TOOLING-SPEC.md#7-agent-context) is recommended or a +SHOULD, or says a declaration is not a condition, asserts no obligation and is neither correct +nor wrong. So is a bullet that states how conformance is measured rather than what is required: the toolchain judged as one assembled whole at the project level (sections [1](../../TOOLING-SPEC.md#1-what-this-document-is-for) and [9](../../TOOLING-SPEC.md#9-conformance)), the project-level verdict a single grade (section [9](../../TOOLING-SPEC.md#9-conformance)), or a shipped artefact graded separately ([9.1](../../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately)). From 99074928ac750fd2098b309725424dd221403539 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:29:03 +0100 Subject: [PATCH 08/21] Routed findings from runs 63 and 64 acceptance/ROUTED.md gains the detector's clause 6.1 paragraph on a project's own rule, from run 63, and notes run 64 on the four method rows it raised again. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 76acd8f..55462f4 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,25 +15,26 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- | ---------------------------------------- | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------------------------------- | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | run 63 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59, 64 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61, 64 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | ## Closed From 5852a8562abd0828b9519726ddcce3490d13b388 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:40:51 +0100 Subject: [PATCH 09/21] Routed findings: four detector rows closed by run 66, two toolchain rows from run 67 acceptance/ROUTED.md gains a Closed table, with the detector's clauses 7.1, 8.1, 5.3 and 6.1 rows closed by editor/detector-route-rule's run 66, notes run 66 on the clause 6.1 row on a project's own rule, and gains the toolchain's clause 6.2 heading and documented-check paragraph from run 67. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 47 ++++++++++++++++++++++++-------------------- 1 file changed, 26 insertions(+), 21 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 55462f4..f995c21 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,27 +15,32 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------------------------------- | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | run 63 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59, 64 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61, 64 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61, 64 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------------------------------- | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | run 67 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59, 64 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61, 64 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | ## Closed -None yet. +A row moves here when a change that reworks its passage passes a cohort, naming that run, or +when the editor closes it with a reason. + +| Document | Clause | Passage | Raised by | Closed by | +| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ------------------------------------ | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule`, run 66 | From f3ec69ebb2b2fc10fc592765ba20848acbdcb00a Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 13:47:43 +0100 Subject: [PATCH 10/21] Routed findings: rows from runs 19 to 30, two closed by defined-terms run 64 acceptance/ROUTED.md gains the passages runs 19 to 30 on the semantic-detectors change named in text that change no longer touches (the Narrowing entry, section 4's lead-in to the owner's decisions, clause 3.1's paragraph on independent techniques and the section 3 table's 3.1 row), notes those runs on the rows they raised again, and closes the method's two-bounds paragraph and bundled-defence paragraph by editor/defined-terms's run 64. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index f995c21..c6c3d28 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -10,7 +10,7 @@ A finding is open until a change that reworks its passage passes a cohort, when that run and moves below the rule, or until the editor closes it with a reason recorded here. A finding raised again by a later run is noted on its line, not listed twice. The status column holds `open`, or the `editor/` branch or pull request number of the change that is -reworking the passage. The rows raised by runs 31 to 52 were raised under the earlier rules, on +reworking the passage. The rows raised by runs 19 to 52 were raised under the earlier rules, on untouched text of the change those runs tested, and are listed here so that they are not lost. ## Open @@ -25,22 +25,28 @@ untouched text of the change those runs tested, and are listed here so that they | `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | | `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | run 67 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, and the bracketed verb "Suppress" among nouns | runs 41, 47, 50, 56, 64 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | | `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 56, 59, 64 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 53, 59, 61, 64 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 53, 59, 61, 64 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | run 61 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed A row moves here when a change that reworks its passage passes a cohort, naming that run, or when the editor closes it with a reason. -| Document | Clause | Passage | Raised by | Closed by | -| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ------------------------------------ | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 39, 42 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | run 54 | `editor/detector-route-rule`, run 66 | +| Document | Clause | Passage | Raised by | Closed by | +| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------ | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 20, 22, 24, 30, 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 20, 30, 39, 42 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | runs 20, 54 | `editor/detector-route-rule`, run 66 | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on the two bounds | "Both bounds measure breadth within one level; the level itself is a separate question, answered against the report." | runs 19, 56 | `editor/defined-terms`, run 64 | +| `SPEC.md` | [3.5](../SPEC.md#35-enforce-permanently-and-block), the paragraph on a bundled defence | "For a Bundled defence, the project whose entry point demonstrates it is any project the Toolchain is genuinely installed into", and its substitutes sentence | runs 25, 29, 53, 59 | `editor/defined-terms`, run 64 | From bc6ae14586c1a392c1808989114ebc71a46b9d27 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:01:43 +0100 Subject: [PATCH 11/21] Method key: clause 3.3's narrowing sentence; routed findings from runs 68 to 71 The method key lists the sentence behind a narrowing, and the owner's decision where it cannot be written, among the obligations, after three readers of run 70 restated it. acceptance/ROUTED.md gains Appendix A's closing paragraph from run 70 and the detector's clause 5.3 from run 69, and notes runs 68, 70 and 71 on the rows they raised again. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 60 +++++++++++++++++++++------------------- acceptance/method/KEY.md | 2 ++ 2 files changed, 33 insertions(+), 29 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index c6c3d28..eced9c7 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,38 +15,40 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- | ---------------------------------------- | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | run 67 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | run 50 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------- | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | run 69 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed A row moves here when a change that reworks its passage passes a cohort, naming that run, or when the editor closes it with a reason. -| Document | Clause | Passage | Raised by | Closed by | -| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------ | -| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 20, 22, 24, 30, 36, 42, 45, 48, 51, 54 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 20, 30, 39, 42 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | runs 20, 54 | `editor/detector-route-rule`, run 66 | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on the two bounds | "Both bounds measure breadth within one level; the level itself is a separate question, answered against the report." | runs 19, 56 | `editor/defined-terms`, run 64 | -| `SPEC.md` | [3.5](../SPEC.md#35-enforce-permanently-and-block), the paragraph on a bundled defence | "For a Bundled defence, the project whose entry point demonstrates it is any project the Toolchain is genuinely installed into", and its substitutes sentence | runs 25, 29, 53, 59 | `editor/defined-terms`, run 64 | +| Document | Clause | Passage | Raised by | Closed by | +| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------ | +| `DETECTOR-SPEC.md` | [7.1](../DETECTOR-SPEC.md#71-a-detector-may-offer-an-inline-suppression-route-but-must-make-it-detectable-or-disableable) | "by a Rule the project can write in the Detector itself", read as the project editing the detector's own source, against clause 4.1's bespoke rules | runs 20, 22, 24, 30, 36, 42, 45, 48, 51, 54, and 71 on a branch without the rework | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [8.1](../DETECTOR-SPEC.md#81-a-maintainer-may-declare-the-version-of-this-document-the-detector-conforms-to-and-the-declaration-records-known-gaps) | "MUST record that gap alongside the version it declares" against "The declaration is optional." | runs 20, 30, 39, 42, and 71 on a branch without the rework | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "the location of the remainder": what the remainder is when the summary prints every finding | run 32 | `editor/detector-route-rule`, run 66 | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a Bundled rule" paragraph's sentence on a Message that carries its own documentation path, against the clause's mechanism keyed on the Identifier | runs 20, 54 | `editor/detector-route-rule`, run 66 | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on the two bounds | "Both bounds measure breadth within one level; the level itself is a separate question, answered against the report." | runs 19, 56, and 70 on a branch without the rework | `editor/defined-terms`, run 64 | +| `SPEC.md` | [3.5](../SPEC.md#35-enforce-permanently-and-block), the paragraph on a bundled defence | "For a Bundled defence, the project whose entry point demonstrates it is any project the Toolchain is genuinely installed into", and its substitutes sentence | runs 25, 29, 53, 59 | `editor/defined-terms`, run 64 | diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index 86cf8ff..e2ffd60 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -23,6 +23,8 @@ no bullet making an assertion listed below as wrong, is a correct restatement. practitioner's own authority (section [4](../../SPEC.md#4-authority-which-decisions-belong-to-whom)). - Fix the original defect conventionally, with a test that reproduces it, once the defence is in place (sections [2](../../SPEC.md#2-when-the-method-applies) and [6](../../SPEC.md#6-relationship-to-test-driven-development)). +- Write down, for any code the rule is narrowed to exclude, why the hazard cannot arise there, + and take the exclusion to the owner where that sentence cannot be written ([3.3](../../SPEC.md#33-prove-the-net-by-making-the-rule-fire), Part B). Wrong if it asserts: the practitioner fixes the reported instance first and defends afterwards; a test may serve as the detector; an agent may baseline the remaining instances. A bullet that From 13b497ba3ddc0eba771e3c516bc89e4ffe14cb2d Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:18:48 +0100 Subject: [PATCH 12/21] Toolchain key: the 4.3 row says which case the fixture presents; ROUTED 5.3 row adds run 75 Run 72's open point: three readers graded clause 4.3 on the condition that the ignore comment is Lintomatic's only suppression route, and the key did not say which case the fixture presents. The 4.3 row now says so, and such a grade is the grade for this case. Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 2 +- acceptance/toolchain/KEY.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index eced9c7..68a0be9 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -18,7 +18,7 @@ untouched text of the change those runs tested, and are listed here so that they | Document | Clause | Passage | Raised by | Status | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------- | | `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | run 69 | `editor/detector-route-rule` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | | `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | diff --git a/acceptance/toolchain/KEY.md b/acceptance/toolchain/KEY.md index aadab7a..e098c90 100644 --- a/acceptance/toolchain/KEY.md +++ b/acceptance/toolchain/KEY.md @@ -63,7 +63,7 @@ Wrong readings the text invites, which count against the document on one reader: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | No | None | Lintomatic fails detector [6.3](../../DETECTOR-SPEC.md#63-a-bundled-rules-documentation-must-ship-with-the-rule-at-a-version-tracked-together) on two undocumented bundled rules and nothing in the wrapping closes that gap; a reader who notes that the detector specification was not supplied and grades from this document's summary of it is correct | | [4.2](../../TOOLING-SPEC.md#42-the-toolchain-must-resolve-every-identifier-a-defence-it-routes-can-print-from-the-installed-copy-without-network-access) | No | None. A No given on the condition that the two bundled rules are blocking is a No: their firing stops `zig-qa run`, so they are | `zq.*` and `proj.*` resolve offline, but the two enabled bundled rules resolve to `(none)` | -| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | None | Ignore comments disabled project-wide; exceptions only through the record the toolchain reads | +| [4.3](../../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | Yes | None | The fixture names one suppression route of Lintomatic's, the inline ignore comment, disabled project-wide, with exceptions only through the record the toolchain reads; a grade given on the condition that the comment is Lintomatic's only route is the grade for this case | | [4.4](../../TOOLING-SPEC.md#44-the-toolchains-own-invocation-must-satisfy-the-detector-specifications-reporting-clauses-for-every-defence-it-routes) | Yes | None | `zig-qa run` is local, prints Lintomatic's output unchanged, one line per finding | | [4.5](../../TOOLING-SPEC.md#45-the-toolchains-entry-point-must-run-detectors-before-runners-and-must-stop-on-a-detector-failure) | Yes | None | Lintomatic, then the tests, stopping on a Lintomatic failure | | [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) | Yes, Partial or No | None | `zig-qa list` prints identifier, summary and documentation path without running the rules; Partial or No is the reading that counts the two `(none)` routes, since [5.1](../../TOOLING-SPEC.md#51-the-toolchain-must-be-able-to-list-the-defences-active-in-a-project-without-triggering-them) says the listing MUST provide the route and [4.1](../../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) point 2 says a gap may be named again by a later clause | From 389ef3f967fe910150b14f77623c1150423210a5 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:35:36 +0100 Subject: [PATCH 13/21] Acceptance: a missing-document remark is a note however it quotes; a parted paragraph is the change's text Settles the open points of runs 76, 77, 78 and 80. ACCEPTANCE.md says a remark that another document was not supplied, or that a term is defined only there, is a note however much of the document it quotes to locate the reference; and that a paragraph the pull request parts from what it refers to, by inserting a paragraph between them or moving either, is the change's text. The Lintomatic fixture says `check --only` prints its findings as `check` does, and the detector key's 4.2 row says so and which case a conditional grade is for. ROUTED gains the toolchain 6.1 Project record sentence (run 76) and the detector 4.2 sentence on a detector failing 4.1 (run 78, taken by editor/defined-terms); the 4.3 row links the section, whose heading the toolchain change rewrites. Co-Authored-By: Claude Fable 5.1 --- ACCEPTANCE.md | 10 ++++--- acceptance/ROUTED.md | 48 ++++++++++++++++--------------- acceptance/detector/KEY.md | 2 +- acceptance/fixtures/lintomatic.md | 3 +- 4 files changed, 34 insertions(+), 29 deletions(-) diff --git a/ACCEPTANCE.md b/ACCEPTANCE.md index d1bd6af..77eac8f 100644 --- a/ACCEPTANCE.md +++ b/ACCEPTANCE.md @@ -134,15 +134,17 @@ rule. gives a clause number with a description or a shorter quotation that identifies a sentence or paragraph. A quotation whose words occur in more than one paragraph, given with no clause number, names the paragraph where they first occur. A remark that is not about a passage of - the document, such as that another document was not supplied, is recorded as a note and is - not a passage. Two readers name the same passage when the words or sentences they name lie in + the document, such as that another document was not supplied or that a term is defined only + in another document, is recorded as a note and is not a passage, however much of the document + it quotes to locate the reference. Two readers name the same passage when the words or sentences they name lie in the same paragraph; a heading, a list item, a table row and a terminology entry are each a paragraph of their own. A passage named by two or more readers of the run is a finding. - **Where a finding lies.** The record says, for each finding, whether it lies in text the change wrote or altered: a paragraph in which the pull request, read against its merge base with the branch it targets, adds, rewords, moves or deletes a sentence, or changes a word, a term or a - link; or a paragraph that any reader naming the finding sets against such a paragraph. A - finding there blocks the change. A finding anywhere else is a finding against the document, not + link; a paragraph the pull request parts from what it refers to, by inserting a paragraph + between them or moving either; or a paragraph that any reader naming the finding sets against + such a paragraph. A finding there blocks the change. A finding anywhere else is a finding against the document, not against the change: it is recorded in [acceptance/ROUTED.md](acceptance/ROUTED.md), routed to a change of its own, and does not block. The change is judged on what it changed; the document is proved by the keyed questions, which every run answers over the whole of it. diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 68a0be9..f322fc5 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,29 +15,31 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ---------------------------------------- | -| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#43-the-toolchain-must-forbid-through-a-defence-of-its-own-every-suppression-route-that-bypasses-the-project-record) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind | run 76 | `open` | +| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed diff --git a/acceptance/detector/KEY.md b/acceptance/detector/KEY.md index 5b4d4aa..dd559a8 100644 --- a/acceptance/detector/KEY.md +++ b/acceptance/detector/KEY.md @@ -64,7 +64,7 @@ Wrong readings the text invites, which count against the document on one reader: | Clause | Accepted grade | What the fixture supports (no row carries a condition on the reasoning) | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [4.1](../../DETECTOR-SPEC.md#41-the-detector-must-support-bespoke-rules-written-by-the-project-that-runs-it) | Yes | Project rules via the `Rule` interface, first class in every report | -| [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, with a distinguishing exit code | +| [4.2](../../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code) | Yes | `check --only ` runs one rule, bundled or project, prints its findings as `check` does, and exits with a distinguishing code; the fixture says all three, so a grade given on the condition that `--only` prints as `check` does is the grade for this case | | [4.3](../../DETECTOR-SPEC.md#43-the-detector-must-allow-a-rule-to-carry-a-stable-identifier-and-must-print-it-with-every-finding) | Yes | `LM-NNNN` fixed at first release; project names taken as given, not rewritten; printed on every finding line | | [5.1](../../DETECTOR-SPEC.md#51-the-detector-must-be-invocable-by-the-practitioner-locally-with-no-infrastructure) | Yes | Local, no account, no server, no network | | [5.2](../../DETECTOR-SPEC.md#52-the-detector-must-support-invocation-over-a-subset-at-minimum-a-single-file) | Yes | `check ` accepts a single file | diff --git a/acceptance/fixtures/lintomatic.md b/acceptance/fixtures/lintomatic.md index f168d56..9d334fc 100644 --- a/acceptance/fixtures/lintomatic.md +++ b/acceptance/fixtures/lintomatic.md @@ -21,7 +21,8 @@ src/db.zig:88:3 proj.no-raw-sql Query built by string concatenation Fuller detail, including a snippet, is written to `.lintomatic/report.html`. The one-line form is printed whether or not the HTML report is enabled. -`lintomatic check --only ` runs a single rule. It accepts any rule name, +`lintomatic check --only ` runs a single rule and prints its findings as +`check` does. It accepts any rule name, bundled or project, and exits 1 when the rule fires, 0 when it does not, and 2 when the rule name is unknown. From 0da928d9830d12101122821650daacd4e050a6b7 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:43:19 +0100 Subject: [PATCH 14/21] ROUTED: run 81 names the toolchain 6.1 Project record sentence again; 6.2 and 6.3 speak only of exceptions Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 51 ++++++++++++++++++++++---------------------- 1 file changed, 26 insertions(+), 25 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index f322fc5..fe7d05c 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,31 +15,32 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | -| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind | run 76 | `open` | -| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | -| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | +| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81 | run 76 | `open` | +| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed From ae6faf2b34d4f8e21d146fd62cef9445102d00fc Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:48:34 +0100 Subject: [PATCH 15/21] ROUTED: run 82 names the toolchain 6.2 last paragraph again Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index fe7d05c..c16596a 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -29,7 +29,7 @@ untouched text of the change those runs tested, and are listed here so that they | `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | | `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | | `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | | `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | | `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | From a095fa7ad823b41b0abe3f932d8484dee9a3fd10 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:53:10 +0100 Subject: [PATCH 16/21] ROUTED: toolchain 6.2's first paragraph against clause 6.4's defaults, from run 83 Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 1 + 1 file changed, 1 insertion(+) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index c16596a..5ae6e16 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -17,6 +17,7 @@ untouched text of the change those runs tested, and are listed here so that they | Document | Clause | Passage | Raised by | Status | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults | run 83 | `open` | | `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | | `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81 | run 76 | `open` | | `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | From 852dcd80f61a418609cd36375ca7631d3648a92e Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 14:58:23 +0100 Subject: [PATCH 17/21] ROUTED: run 84 names the toolchain Project record entry against clauses 6.1, 6.2 and 6.4 Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 54 ++++++++++++++++++++++---------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 5ae6e16..1423a48 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,33 +15,33 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults | run 83 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | -| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81 | run 76 | `open` | -| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | -| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults | run 83 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | +| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81, and by two readers of run 84 as the Project record entry against clauses 6.1, 6.2 and 6.4 | run 76 | `open` | +| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed From 568e323782847e3f03baeded0303a0979d7d41da Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 15:02:37 +0100 Subject: [PATCH 18/21] ROUTED: run 85 names the toolchain 6.2 first paragraph against clause 6.4 again Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index 1423a48..fe7182a 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -17,7 +17,7 @@ untouched text of the change those runs tested, and are listed here so that they | Document | Clause | Passage | Raised by | Status | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- | -------------------------------------------------- | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults | run 83 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults; named again by two readers of run 85 | run 83 | `open` | | `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | | `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81, and by two readers of run 84 as the Project record entry against clauses 6.1, 6.2 and 6.4 | run 76 | `open` | | `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | From 40f7895867706c8bfdc7f5cb27a9ff1507ba2a8c Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 15:08:03 +0100 Subject: [PATCH 19/21] ROUTED: run 86's findings in untouched toolchain text, clause 6.4 against 6.2 new Co-Authored-By: Claude Fable 5.1 --- acceptance/ROUTED.md | 55 ++++++++++++++++++++++---------------------- 1 file changed, 28 insertions(+), 27 deletions(-) diff --git a/acceptance/ROUTED.md b/acceptance/ROUTED.md index fe7182a..e53b671 100644 --- a/acceptance/ROUTED.md +++ b/acceptance/ROUTED.md @@ -15,33 +15,34 @@ untouched text of the change those runs tested, and are listed here so that they ## Open -| Document | Clause | Passage | Raised by | Status | -| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- | -------------------------------------------------- | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults; named again by two readers of run 85 | run 83 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | -| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81, and by two readers of run 84 as the Project record entry against clauses 6.1, 6.2 and 6.4 | run 76 | `open` | -| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | -| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | -| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | -| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | -| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | -| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | -| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | -| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | -| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | -| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | -| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | +| Document | Clause | Passage | Raised by | Status | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------- | +| `TOOLING-SPEC.md` | [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project), the heading and first paragraph | "SHOULD state its own defaults for anything the method leaves to the project" against clause [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one)'s "MUST NOT supply a default" for a justification | run 86 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one), the first paragraph | "It MUST NOT supply a default, and MUST reject an Exception that omits one" against clause [6.4](../TOOLING-SPEC.md#64-the-toolchain-should-state-its-own-defaults-for-anything-the-method-leaves-to-the-project)'s defaults; named again by two readers of run 85 and two of run 86 | run 83 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) and [6.3](../TOOLING-SPEC.md#63-the-project-record-must-be-enumerable-by-the-same-means-as-the-defences) | Both clauses speak only of exceptions, where section 2's Project record entry and clause 6.1 cover calibrations and conventions too | run 81 | `open` | +| `TOOLING-SPEC.md` | [6.1](../TOOLING-SPEC.md#61-the-toolchain-must-define-a-location-for-the-project-record-and-must-read-it-itself), the first paragraph | "MUST define where the Project record lives" against section 2's Project record entry (calibrations, exceptions and conventions): one location or one per kind; named again by four readers of run 81, and by two readers of run 84 as the Project record entry against clauses 6.1, 6.2 and 6.4; and by two readers of run 86, the entry and clause 6.1's first paragraph together | run 76 | `open` | +| `DETECTOR-SPEC.md` | [4.2](../DETECTOR-SPEC.md#42-the-detector-must-provide-a-harness-that-runs-a-single-rule-against-supplied-code), the first paragraph | "a Detector failing 4.1 is graded here on its Bundled rules": whether it limits or waives anything | run 78 | `editor/defined-terms`, which removes the sentence | +| `SPEC.md` | [Appendix A](../SPEC.md#appendix-a-instructing-an-agent), the closing paragraph | "Only then fix the original Defect in the normal way, with a test that reproduces it" | run 70 | `open` | +| `DETECTOR-SPEC.md` | [5.3](../DETECTOR-SPEC.md#53-the-result-must-reach-the-practitioner-in-the-output-of-the-command-they-ran) | "MUST carry both a usable summary and the location of the remainder" against "the output need not name it" | runs 69, 75 | `editor/detector-route-rule` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) and [9.1](../TOOLING-SPEC.md#91-a-project-that-ships-a-detector-or-a-toolchain-has-two-levels-of-conformance-graded-separately) | Section 9 counts section 8 in a shipping project's project grade; clause 9.1 says section 8 bears on the artefact grade only | runs 43, 46, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [2](../TOOLING-SPEC.md#2-terminology), [4.2](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through), [8.1](../TOOLING-SPEC.md#81-a-shipped-toolchain-must-fail-its-own-release-if-a-bundled-defence-lacks-resolvable-documentation) | The Defence row, "with its documentation, in force and Blocking", against clauses 4.2 and 8.1, which exist to catch a Blocking rule whose documentation is missing | runs 33, 40, 43, 46, 49, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9.2](../TOOLING-SPEC.md#92-the-declaration-is-the-claim-and-the-known-gap-record-not-a-condition-of-conformance) | "MUST record that gap alongside the version" against "The declaration is optional." | runs 33, 52, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.3](../TOOLING-SPEC.md#4-the-detectors-a-toolchain-routes-defences-through) | The heading, "through a defence of its own", against the body's "MUST disable ... or MUST run a Blocking Defence" | runs 33, 55 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [4.1](../TOOLING-SPEC.md#41-every-detector-the-toolchain-routes-a-defence-through-must-conform-to-the-detector-specification) | "That document's MUSTs, in brief, are six", where the detector specification's sections 4 to 7 carry eleven | runs 40, 46 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [9](../TOOLING-SPEC.md#9-conformance) | "A Toolchain Conforms with Agent support if, in addition, it satisfies section 7", where section 7 holds only SHOULDs | run 57 | `editor/toolchain-contradictions` | +| `TOOLING-SPEC.md` | [5.3](../TOOLING-SPEC.md#53-a-projects-own-defences-must-appear-in-the-listing-alongside-bundled-ones) | "Where such a Defence is not expressible in the Toolchain's own Detectors, its entry MAY be sourced from wherever it is enabled": what "such a Defence" and "its entry" refer to | run 52 | `open` | +| `TOOLING-SPEC.md` | [6.2](../TOOLING-SPEC.md#62-every-exception-in-the-project-record-must-carry-a-written-justification-that-names-the-hazard-and-the-scope-and-the-toolchain-must-reject-a-generic-one) | The heading, "the toolchain MUST reject a generic one", and the paragraph "The Toolchain MUST reject a justification that could be pasted onto any Exception unchanged, by a check it documents": whether the check must test that the hazard and the scope are named | runs 67, 68, 82 | `open` | +| `DETECTOR-SPEC.md` | [6.1](../DETECTOR-SPEC.md#61-the-detector-must-provide-a-mechanism-that-resolves-a-printed-identifier-to-its-documentation) | The "For a project's own Rule" paragraph: its URL sentence, "as method clause 3.6 allows", "a documentation path printed alongside a shorter Identifier", and "The lookup for such a Rule is an obligation on the project's assembled Toolchain" against section 1's governing principle | runs 28, 63, 66 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Owner entry | The default Owner sentences: whether whoever instructed the Practitioner may take section 4's decisions, "Always a human", and the bracketed verb "Suppress" among nouns | runs 23, 41, 47, 50, 56, 64 | `open` | +| `SPEC.md` | [1](../SPEC.md#1-terminology), the Narrowing entry | "Narrowing that excludes code which does carry the Hazard is a Suppression" | run 23 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom) | "the Rule MAY merge at full width with that case recorded as a known Instance awaiting the Owner" against "provided the Rule does not become Blocking until every Instance is fixed" | runs 50, 70 | `open`; a method question for the editor | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the lead-in to the owner's decisions | "Decisions that MUST go to whoever owns the codebase", against the Owner entry | run 21 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the lower-bound list item | "That rejection is tested as a Narrowing is under clause 3.3", and "A Rule drawn to the exact value or name found in the Defect is the same Narrowing however it is spelt" | runs 23, 56, 59, 64, 70 | `open` | +| `SPEC.md` | [3.1](../SPEC.md#31-attribute-the-defect-to-a-class), the paragraph on independent techniques | "Two techniques are independent when they would miss different things", and that a clean run of the Rule is never one of the two | run 25 | `open` | +| `SPEC.md` | [3](../SPEC.md#3-the-method), the table's 3.1 row | The fourth column, "Whether the next wider Rule is built, and whether the Rule stays narrower than the search showed" | run 21 | `open` | +| `SPEC.md` | [2](../SPEC.md#2-when-the-method-applies), "Attempt rather than pre-judge" | "naming at least two independent techniques tried at expressing the Rule, in the vocabulary clause 3.1 uses for a search" | runs 19, 29, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [4](../SPEC.md#4-authority-which-decisions-belong-to-whom), the list item on leaving the next wider Rule unbuilt | "The absence of an Instance is the Practitioner's reason not to widen unasked", read against clause 3.1's lower bound | runs 21, 53, 59, 61, 64, 70 | `open` | +| `SPEC.md` | [3.4](../SPEC.md#34-sweep-the-codebase-then-fix-every-instance), the "Fix them all." paragraph | "the scale at which that becomes true is tremendous" | runs 19, 61 | `open` | ## Closed From 4f2ebb58c8ff095d9746eca13d3a6a239cca1bcc Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 16:00:12 +0100 Subject: [PATCH 20/21] Method key 2a: an answer that leaves the sweep and the fix unordered does not state the order The question asks in what order, so element (2) is stated only where the answer puts the sweep before the fix of the defect handed. An answer saying the defect is fixed after some steps and, elsewhere, that every instance is swept and fixed, without saying which comes first, does not state it; a quotation such as clause 3.4's "before fixing anything" does; an answer whose own sentences put the fix before the sweep is wrong. Settles the open point run 89 recorded. A process change, recorded in acceptance/CHANGES.md; no cohort. Co-Authored-By: Claude Opus 5.5 --- acceptance/CHANGES.md | 8 ++++++++ acceptance/method/KEY.md | 15 ++++++++++++--- 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md index daa3d44..e268f6c 100644 --- a/acceptance/CHANGES.md +++ b/acceptance/CHANGES.md @@ -6,6 +6,14 @@ change, naming its pull request. The process is not a specification and is not v change to it is not tested by a cohort. Changes to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). +- Pull request PRNUM: the method key's 2a says how an answer is marked when it says that the + defect handed is fixed after some steps and, elsewhere, that every instance is swept and fixed, + without saying which comes first. The question asks in what order, so the order is the point of + element (2), and such an answer does not state it. The key now says that "the fix" in element + (2) is the fix of the defect handed, that a quotation such as clause 3.4's "before fixing anything" states it, and that an answer + putting the fix before the sweep is wrong whatever it quotes. Run 89, on the gap-terms change, + recorded the point as open and marked it for the reader under the open-point rule. This applies + to runs whose readers start after it merges, and changes no mark already given. - [Pull request #22](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/22): the test is made to prove what it exists to prove, the clarity of the document to a cold reader, after thirty-three runs on two changes (19 to 30, 31 to 33 and 35 diff --git a/acceptance/method/KEY.md b/acceptance/method/KEY.md index e2ffd60..961dc1e 100644 --- a/acceptance/method/KEY.md +++ b/acceptance/method/KEY.md @@ -37,13 +37,22 @@ Wrong readings the text invites, which count against the document on one reader: - **2a.** Required: (1) the four steps before the fix: attribute the defect to a class, express the class as a rule in a detector, prove the rule fires, and sweep the codebase; (2) in that - order, with the fix after the sweep. Wrong if it has the fix before the rule, or the sweep - before the proof, or a test as the detector. Key quotations, section [3](../../SPEC.md#3-the-method): "Six clauses, in + order, with the fix after the sweep. Wrong if it has the fix before the rule, the fix before + the sweep, or the sweep before the proof, or a test as the detector. Key quotations, section [3](../../SPEC.md#3-the-method): "Six clauses, in order."; [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance): "The Practitioner MUST run the Rule across the entire codebase and record the total Instance count before fixing anything, and MUST then fix every Instance found rather than only the one". An answer that names the steps by their clause numbers in order states both elements; one that says only that section [3](../../SPEC.md#3-the-method)'s six clauses are followed in order, - naming no step, does not state element (1). + naming no step, does not state element (1). In element (2), "the fix" is the fix of the + defect the practitioner was handed. The question asks in what order, so element (2) is + stated only where the answer's sentences or quotations put the sweep before that fix; naming + the steps by their clause numbers in order, as above, does, since clause [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance) sweeps before + fixing anything. "Sweep and fix every instance" does not by itself put the sweep before that + fix, so an answer that says in one sentence or part of a sentence that the defect is fixed after some + steps, and in another that every instance is swept and fixed, without saying which of the two + comes first, does not state element (2). A quotation that puts the sweep before any fix, such + as clause [3.4](../../SPEC.md#34-sweep-the-codebase-then-fix-every-instance)'s "before fixing anything", states it, except that an answer whose own sentences + put the fix before the sweep is wrong, whatever it quotes. - **2b.** Required: (1) the owner of the codebase, the human who holds that authority; (2) no, an agent may not do so on its own authority. Wrong if it lets the practitioner or an agent decide because the count was small, because the project recorded no owner, or at all. Key From ec4f3653396ff3bfda7f564c83ead81363806281 Mon Sep 17 00:00:00 2001 From: joseph Date: Mon, 12 Oct 2026 16:00:33 +0100 Subject: [PATCH 21/21] CHANGES: name pull request 26 Co-Authored-By: Claude Opus 5.5 --- acceptance/CHANGES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/acceptance/CHANGES.md b/acceptance/CHANGES.md index e268f6c..0fd0461 100644 --- a/acceptance/CHANGES.md +++ b/acceptance/CHANGES.md @@ -6,7 +6,7 @@ change, naming its pull request. The process is not a specification and is not v change to it is not tested by a cohort. Changes to the three specifications are recorded in [CHANGELOG.md](../CHANGELOG.md). -- Pull request PRNUM: the method key's 2a says how an answer is marked when it says that the +- [Pull request #26](https://github.com/Defence-Before-Fix/defence-before-fix.github.io/pull/26): the method key's 2a says how an answer is marked when it says that the defect handed is fixed after some steps and, elsewhere, that every instance is swept and fixed, without saying which comes first. The question asks in what order, so the order is the point of element (2), and such an answer does not state it. The key now says that "the fix" in element