Tighten cleaning handoff evidence format

[mcdigman]

Summary

• require contract-clean-consolidation action items to include line numbers and exact excerpts instead of prose-only target spans
• replace verbose basis/cleaner-instruction fields with compact issue gist, preservation check, and cleaner operation fields
• require contract-cleaner rejected/blocked ledger entries to include line numbers and short excerpts for human review
• prohibit freeform downstream-instructions sections from consolidation output

Verification

• checked staged diff only included the four prompt/skill files
• verified contract-clean-consolidation agent and skill copies remain identical
• verified contract-cleaner agent and skill copies remain identical

Prompt-only change; no tests run.

[mcdigman]

Review: Tighten cleaning handoff evidence format

Verdict: Approve with two should-fix integration gaps and a few minor notes. The change is well-targeted at the stated problem (verbose/repetitive consolidator reports; references to issues not in the report; text descriptions without line numbers) and stays almost entirely within scope. The four files are self-consistent — agent/skill copies are byte-identical, and no stale references to the renamed Basis/Cleaner instruction fields remain.

Does it fix the stated problem? Yes, on all three axes

• Verbosity/repetition: prose Basis + Cleaner instruction collapse into a 1–2 sentence Issue gist / preservation check + Cleaner operation; excerpts capped at 5–60 words / 2 lines with [...] elision.
• References to issues not in the report: the new Hardening rule that bare per SC003 cross-references are insufficient — every action/rejection/blocker must carry enough excerpt + neutral summary to be understood "without opening the original audit reports" — squarely removes the hallucination-invitation channel.
• Descriptions without line numbers: line numbers now mandatory on action items, rejections, blockers, and human-decision items, each anchored by an exact excerpt.

Quoting verbatim steering text into the action list is safe here: the action list and cleaning ledger are cleanup-phase process artifacts that are not passed to adversarial reviewers, and the new line/excerpt requirements land only on rejected/blocked ledger entries where the content is still present in the contract anyway — so no new leakage is created and the ledger's "not a vehicle for preserving removed content" guard is preserved.

Should-fix (integration)

1. The consolidator isn't guaranteed line-numbered input, yet line numbers are now mandatory. Input #1 to contract-clean-consolidation is still "The contract text", whereas the cleaner receives "The contract file (path provided by the orchestrator)". If the orchestrator pastes the contract inline, the consolidator (tools: Read only) has no reliable line numbers and must hand-count — the exact error-prone step the rule is meant to eliminate. Recommend aligning Input #1 to "the contract file (path provided by the orchestrator)" so it Reads a line-numbered file.

2. Action-list line numbers are pre-clean and will drift; the cleaner isn't warned. The cleaner executes deletions sequentially, so every action item after the first has stale line numbers. This is salvageable because the exact excerpt is the authoritative anchor and the ledger correctly records "current line number(s)" — but the cleaner prompt never says so. One sentence ("treat action-list line numbers as pre-clean references; the exact excerpt is authoritative when they disagree") would prevent a cleaner from trusting a drifted number.

Verifier priming — adequate now; recommend a shared format linter later

contract-clean-verifier was not touched and isn't broken by the new format (line numbers/excerpts are additive; it holds the pre-clean contract so the pre-clean line numbers resolve for it). The soft gap is that nothing algorithmically enforces that the consolidator actually produced a compliant report — mandatory line numbers present, no bare source-ID references, excerpt/issue-reference correspondence intact, no freeform-instruction section.

Rather than bolt prose checks onto the verifier, a better fix — out of scope for this PR — is a shell-script format linter with a firm pass/fail exit code, given to both the consolidator (self-check before emitting) and the verifier (gate on receipt). It could, via regexes and light parsing: assert every action/rejection/blocker carries line number(s) + excerpt; validate that cited line numbers actually correspond to the quoted excerpt in the referenced report/contract; and validate that prose issue references and severities match what the required .yaml metadata states. This is deliberately broader than this PR because ideally every agent in the pipeline gets its own compliant-review-format linter, so the whole family of report formats is enforced algorithmically to the extent possible, not just this handoff.

Scope / substantive-change notes

• Do not include a freeform downstream-instructions section is the one edit beyond pure formatting — it's an authority statement ("the cleaner's authority is limited to the structured action items and any resolved human decisions"). No downstream-instructions section is defined anywhere else, so it contradicts nothing; it's a defensive prohibition well-aligned with the anti-context-rot goal. Worth a conscious bless since it narrows the consolidator↔cleaner authority contract, but I'd keep it.
• Minor content loss: old Basis required stating "what substantive content survives and where it is preserved"; the new preservation check drops "and where." Still covered for condense by the separate canonical-statement rule; softer for remove/replace_with_neutral. Low impact.
• Mild tension: "prefer a compact table" alongside ~8 fields (several multi-sentence, plus a 60-word excerpt) won't render as a genuinely compact table. Not a defect — just don't expect it to bind; a structured list may stay more readable.

Prompt-only change; no functional risk. The PR's stated verification (staged diff limited to the four files; agent/skill copies identical) both check out.

[mcdigman]

Re-review of c48cb86 — findings resolved

Confirmed the latest commit closes all three items from my earlier review, does nothing beyond that, and introduces no new issues.

Earlier finding	Fix	Status
Should-fix #1 — consolidator Input #1 said "contract text", not guaranteeing line-numbered input	Changed to "The contract file (path provided by the orchestrator)", matching the cleaner	✅ Closed
Should-fix #2 — cleaner not warned action-list line numbers are pre-clean and drift	Added paragraph: action-list line numbers are pre-clean references, exact excerpt is authoritative when they disagree, ledger reports current line numbers	✅ Closed
Minor — preservation check had dropped "and where it is preserved"	Re-added "and where it is preserved" to the Issue gist / preservation check field	✅ Closed

Scope check: commit touches only the four in-scope files; each hunk maps 1:1 to a finding — no unrelated edits, reordering, or reword. Agent/skill copies remain byte-identical for both pairs. The new cleaner paragraph reinforces (does not contradict) the existing ledger-format requirement. The out-of-scope shared format-linter recommendation was correctly left for a future PR — no linter or verifier change introduced.

No new issues. Clean to merge from my review's standpoint.

[mcdigman]

Follow-up: A/B report assessment + a conflict-resolution design issue this surfaced

Assessed a candidate consolidator report produced at c48cb86 against the pre-PR baseline (reading only the two reports, i.e. the standpoint a downstream cleaner or a human blocker-decider actually occupies). Net: the new format is better on all three target axes — more readable, harder to hallucinate from, and decidable standalone — and it is longer mostly for the right reasons. But it exposes some residual verbosity traceable to prompt wording, one cross-cutting fix that belongs elsewhere, and a genuine modeling error in the conflict-resolution rule.

1. Readability — improved despite the length

• New report groups items by action type with uniform labeled fields and promotes blocked items to their own standalone section; the baseline was a flat stream hidden inside a <details> collapsible.
• The baseline's shortness was partly illusory: it leaked process narration ("I'll start by reading…"), carried visible mid-stream self-corrections ("Correction: …", "— actually, per protocol …"), and printed two contradictory count tables after an "inadvertently omitted" recovery. That is precisely the "confusing to humans / context-rot for agents" failure. The new report has none of it.

2. Hallucination hardening — substantively better

Every actionable, rejected, and blocked item now carries line numbers + a verbatim excerpt + a standalone neutral gist. I checked specifically for the two failure modes (bare "per SC003" cross-references, and description-without-text) and the new report has none — source IDs appear only alongside inline substance. The baseline anchored most spans by prose with no line numbers, the weaker form.

3. Decidable from the document alone — yes, for both consumers

Cleaner: each item has excerpt + lines + operation + postcondition, with exact replacement text for replace_with_neutral and canonical-location-with-lines for condense. Human: the two blocked items give exact excerpt + precise decision + outcomes + decision-type. Both are rulable without opening any other file.

Residual verbosity traceable to the consolidation prompt

1. Double-quoted spans. Long spans are quoted in a blockquote and then restated as opening/closing fragments in a parenthetical. The target-span rule reads as if the [...] opening/closing form is an addition to a full quote rather than an alternative. Fix: say explicitly use one representation, never both (keep only genuinely new info like an exclusion note).
2. Conflict Notes re-explain items already explained (they restate the corresponding action/blocked items). Scope the required-output wording to terse pointers rather than re-derivation.
3. Issue gist / preservation check balloons past its "one or two sentences" budget when preservation requires enumerating many canonical sites. Acknowledge the tension: allow a compact bullet list of canonical locations for multi-part removals, but cap the prose to two sentences.

One fix that belongs elsewhere, not in this prompt

The biggest readability win — no leaked narration / no in-place self-correction — is not guaranteed by this PR; it is likely partly the stochastic n=1 improvement. The prohibition ("emit only the structured report; do not narrate reading/thinking; revise self-corrections in place") is cross-cutting to every agent and should live in .claude/agent-shared/handoff-protocol.md, which also protects the cleaner, verifier, and the three audit passes. Items 1–3 above are also concrete targets for the previously-recommended shared format-linter.

---

4. Deeper issue this surfaced: the conflict-resolution rule is modeled wrong

Re-reading the Conflict Notes exposed a real design error, independent of length. The current rule ("adopt the smallest footprint — prefer condensation over full removal") keys off the recommended action verb, but the three passes have disjoint remits and each emits the most aggressive action its own scope authorizes. A verbosity pass is structurally forbidden from saying "strike this as steering garbage"; the strongest thing in its vocabulary is "condense, redundant." So steering=remove vs verbosity=condense on the same span is an artifact of scope partition, not a disagreement — they agree the bulk should go. Applying "smallest footprint" there is actively harmful: condensing steering content leaves steering content, which is exactly the surface-only outcome the clean-verifier is built to reject. The current rule can therefore manufacture edits the very next gate is designed to fail.

Fix 4a — Determination test: compare substance claims, not action verbs. All three passes already emit a substance claim (steering: "why removal would not delete substantive content"; verbosity: "what would be lost / 'nothing substantive'"; style-lint: "what substantive content may need preserving"). Use it:

• Concurrent (common, not a conflict): all findings assert nothing-substantive-lost / empty combined preserve-set → they agree; take the union action; fold into one action item with multiple Sources; no conflict note.
• Bounded-but-compatible: a finding names substantive content to keep, but the steering/verbosity/style effect can be removed while preserving that kernel → replace_with_neutral scoped to preserve it. Still not a conflict.
• True conflict (narrow residue that still blocks): one finding says "nothing substantive, remove" while another identifies substantive content that must survive in the same span and both cannot be honored; or two replace_with_neutral directions are mutually unsatisfiable.

This also shrinks Conflict Notes (Fix 2): most current "conflicts" reclassify as ordinary multi-source agreement — in the c48cb86 report, two of the three Conflict Notes are really "these do not conflict," written out as paragraphs.

Fix 4b — Replace "smallest footprint" with a lexicographic ordering. Footprint-minimization is the wrong top-level lever; keep it only as a tiebreaker among already-sufficient edits:

1. Substance-preservation floor (hard, dominant, unchanged): never delete the sole copy of a substantive requirement/criterion/authority/disambiguation; uncertainty → block.
2. Concern-completeness (new, dominates footprint): the edit must discharge every concern on the span. Closure differs by concern (steering = anchoring effect gone; verbosity = redundancy gone; style = risky shape gone). The action type is set by the strictest closure present, not by the smallest recommended action.
3. Minimal footprint (subordinate tiebreaker): among edits satisfying 1 and 2, pick the smallest.

Resulting action-type selection is deterministic:

Concerns on span	Substantive kernel?	Action
steering (± verbosity/style)	no	remove
steering (± verbosity/style)	yes, cleanly separable	replace_with_neutral preserving kernel
steering, kernel not cleanly separable	yes	block
verbosity only	no	remove or condense
verbosity only	yes (canonical elsewhere)	condense to cross-reference
style-lint only	—	replace_with_neutral / cite authority

Rule of thumb for the prompt: stacking concerns take the union of what each requires; condensation of steering content is never a valid steering fix. This is strictly more aligned with the clean-verifier than today's rule — worth stating explicitly so the two gates stop pulling against each other.

Where these live. 4a/4b are mostly a rewrite of the consolidation prompt's "Conflict resolution" bullet. One high-value upstream change belongs in .claude/agent-shared/conventions.md: give all three passes a shared discrete field (e.g. substantive_content_at_risk: none | <short list with location>) so the consolidator can mechanically compute the union of preserve-sets and detect true conflicts instead of inferring from prose — and so the format-linter can check it.

Caution to encode: this makes remove the default when concerns stack with no kernel present — more aggressive than today. The substance floor (tier 1) plus block-on-uncertainty are what keep that safe, so they must stay explicitly dominant; the reorder only moves footprint-minimization below concern-completeness, it does not touch the substance gate.

[mcdigman]

A/B result on the conflict-resolution protocol (report at 637f946)

Read the new consolidator report in full and cross-checked it against the c48cb86 baseline. The conflict change worked; a separate, older failure mode regressed back in. Nothing that regressed is traceable to the conflict edit — clean attribution for the experiment log.

1. Conflict issue — closed and correctly applied

The new protocol is visibly driving the reasoning, and the report explicitly separates bonafide conflicts from scope-differences:

• Malicious-Compliance (CA009): "not a bonafide conflict: the union of the actions is to condense … full removal would delete the substantive routing obligation that SL007 identifies." The condense is now justified by substance preservation (SL007's routing requirement), not "smaller footprint."
• ULP-boundary (CA007): union of {remove, remove, move-to-non-normative} → remove, since nothing substantive is unique to the paragraph. This is the class most at risk of being wrongly down-ruled to condense before; it now lands on the correct stricter closure.
• Witness (CA002): correctly does not force a resolution — VR010's own "escalate if necessary disambiguation" makes substance uncertain, so it blocks (the "no action both preserves substance and resolves concerns → human_decision_required" branch).
• Nested-scope cases (SC002⊃SL003; SC005≡SL002) correctly labeled "not a bonafide conflict."

No steering content is condensed-in-place anywhere. The change closed the issue and, in the ULP case, moves the outcome to the correct answer. Safe to keep.

2. Regressions — real, but on the output-hygiene axis, not the conflict axis

Same class c48cb86 had cured (and which I'd flagged as only partly fixed / possibly luck):

• Withdrawn items left inline. CA010 and CA020 are created, then "withdrawn"/"subsumed" with narrative in place, leaving non-contiguous IDs with holes.
• Internally inconsistent Summary. The replace_with_neutral row says count 5 but lists 6, "corrected" by a footnote instead of fixing the table; CA009 (an actionable condense) is miscategorized under "Subsumed/merged," so condense is undercounted and "Total actionable = 15" understates the real ~17. The metadata YAML (20 findings) is right; the prose Summary contradicts both body and YAML.
• Reasoning leaks into structured fields. CA013's target-span and CA014's gist contain live "checking:"/"Verify…" narration.

One separate item-level issue (not conflict-related): CA014 tells the cleaner to move a six-bullet block into Test Requirements and conditionally add an assertion "if not already" — beyond clean-condense and cleaner authority; the cleaner will likely block it or the verifier flag relocation/addition. The analogous item in c48cb86 was a simple "replace sub-list with a cross-reference."

Disposition shift (2→3 blocked; CA003 DI-2 prose) is within expected n=1 noise and its reasoning is sound (DI-2's binding weight is entangled with the still-blocked CA001 override → block-when-uncertain fires correctly).

3. Highest-leverage next change — and it is not "emit as an artifact"

The dominant remaining defect is output hygiene, now demonstrably not reliably absent. Root cause is single-pass, forward-only generation, not the output channel: c48cb86 (clean) and 637f946 (messy) were emitted the same way, so the channel isn't the variable. Switching to a file artifact does not fix it on its own — Write is also a single-pass dump; only a draft→re-read→Edit→finalize loop would, and that requires giving the deliberately read-only consolidator write tools (erodes its read-only guarantee, invites contract edits). Not worth it for a formatting problem.

Fix it with prompt rules that defer commitment so there is nothing to retract (works in a pure-generation, read-only agent):

1. Assign CA### IDs last, to the finalized set only — do disposition churn in reasoning, so there is no CA010 to withdraw and IDs stay contiguous.
2. Compute counts last, by counting the emitted list; state them once, reconciled — no correction footnotes.
3. Span/gist fields carry only final values; verification narration stays out of structured fields.
4. Subsumptions are a one-line ledger note, never a numbered item with a withdrawal story.

For determinism without changing the consolidator's tools, backstop with the previously-recommended format-linter (contiguous IDs, no withdrawn entries, Summary counts == enumerated items) at the verifier's intake or as a standalone lint step — this is its cleanest first spec.

Suggested sequencing for clean attribution: ship the commitment-deferral hygiene rules first (cheap, no tools change, keeps read-only), measure, and only reach for the linter if variance persists. Do not bundle the CA014 "don't instruct move/add beyond condense" tightening into the same commit — it's a separate axis (action-operation scoping) and would muddy attribution.

[mcdigman]

Model comparison — consolidator output at 9f4c314 (identical prompt)

Four runs of the same frozen prompt on the same inputs (PR #49 contract + the three audit reports). Objective, per-axis observations only; disposition-count differences are within expected n=1 stochasticity.

Runs: sonnet-4.6 (baseline the prompt was tuned against), opus-4.8 (no thinking), opus-4.8 (thinking on), gpt-5.5-codex.

Output hygiene (contiguous IDs, count reconciliation)

Run	Result
sonnet-4.6	Contiguous IDs; header/Summary/YAML counts reconcile
opus-4.8 (no thinking)	Contiguous IDs, but status header says "11 action items · 8 executable" while Summary/YAML say 18 total / 15 executable — header not recomputed
opus-4.8 (thinking)	Contiguous IDs; header (16 / 12 actionable / 6 rejected), Summary, and YAML all reconcile
gpt-5.5	Contiguous IDs; header (21), Summary, and YAML all reconcile

Handling of the two most-contested findings (SC002 DI-2 asserted prose; SC007/VR007 MC section)

Run	SC002 (DI-2)	SC007/VR007 (MC section)
sonnet-4.6	actioned (split condenses)	actioned (condense preserving routing obligation)
opus-4.8 (no thinking)	rejected	rejected (only the modal fix actioned)
opus-4.8 (thinking)	blocked → human decision	blocked → human decision
gpt-5.5	actioned (replace_with_neutral, full DI-2 rewrite)	actioned (replace_with_neutral, section rewrite)

All four block the possible_contract_defect (SC008) as required. Divergence is in whether the two contested steering spans are actioned (removed/neutralized now), blocked (routed to human), or rejected (left in place, dismissed).

Conflict-resolution / cross-item dependency handling

• opus-4.8 (thinking): identifies CA008↔CA016 as a bona-fide conflict (SL007 must preserve/strengthen the escalation clause that SC007/VR007 would delete) and gates the modal fix on the section decision; also gates a DI-3 condense on the DI-2 block.
• opus-4.8 (no thinking): tracks two dependencies (holds a deliverables bullet pending the override block; makes a witness parenthetical contingent on the witness block).
• sonnet-4.6: applies the bona-fide-vs-scope determination with union reasoning in conflict notes.
• gpt-5.5: blocks the same uncertain items but does not flag the DI-2 ↔ override-section dependency.

Verbosity-cleaning posture (rejections)

• Rejected-finding counts: sonnet 0 · gpt 0 · opus-no-thinking 2 · opus-thinking 6.
• opus-thinking's rejections argue several flagged repetitions are not cleanly-separable (e.g., the three anti-relocation passages each carry distinct content; VR006's remedy is a cross-section move beyond span-level cleanup). gpt and sonnet instead action those as condenses/cross-references.
• Net: more rejections → more low-severity redundancy left in place; fewer rejections → more redundancy removed but more reliance on the condense being safe.

Cleaner-executability of the emitted actions

• opus (thinking) and gpt give exact "replace with …" text for every neutral replacement.
• opus (no thinking) delegates some uniqueness judgment to the cleaner on two condense items ("delete bullets already stated elsewhere; retain if uncertain").
• gpt instructs a move/merge of two test lists between sections (CA007) — a restructure beyond contiguous-span cleanup.

Format / render-robustness

• opus (no thinking): single 7-column prose table; wraps/misaligns in width-limited markdown viewers.
• opus (thinking): labeled per-item blocks (**CAxxx · sources · section · lines · type** + bulleted fields); only the short rejected-findings table is tabular.
• sonnet: grouped action tables inside <details> collapsibles.
• gpt: flat labeled per-item list.

Metadata integrity

• opus (thinking): two input-attestation SHA-256 values in the YAML (steering_report_sha256, verbosity_report_sha256) do not match the values the other three runs report — hand-transcription corruption (inserted/dropped hex). The other three runs' input SHAs match each other.
• All four carry resolved_model: sonnet in the YAML regardless of the actual model — a stale template field in every run.
• All four preserved the injected style-lint SHA typo as data and treated the contract's reviewer-directed sections as data, per the hardening rules.

Raw disposition totals (actionable / blocked / rejected): sonnet 19/3/0 (24 items) · opus-no-thinking 15/3/2 (18) · opus-thinking 12/4/6 (16) · gpt 18/3/0 (21).