Filed so these survive #165 closing — @currentlycodinng flagged in the PR review that its five "Open decisions" read like separate tickets and would get lost once the PR merges. The doc content in #165 is accurate either way (independently verified against source in that review); none of these block it.
Each keeps the PR body's Lean. Two need a human call by nature (#1 is a code change, #3 is a judgement call); the other three are mechanical once someone says go.
1. mode sentinel in red_team() silently swallows mode="dynamic" on replay
previous_run= rejects eight data-selection args by name but not the ninth. mode defaults to Pipeline.DYNAMIC (runner.py:490), so Pipeline(mode) != Pipeline.DYNAMIC (runner.py:698) can't tell "passed dynamic" from "passed nothing"; the value is then overwritten by the stored pipeline. max_turns two lines away uses exactly the sentinel that would fix it. --mode dynamic on the CLI has the identical hole.
Lean: mode: Pipeline | str | None = None, resolve to DYNAMIC after the conflict check, same for cli.py; add a test (test_replay.py currently asserts only the eight that raise). Non-breaking. Documented in a !!! note for now because the routine may not touch src/.
2. Replay documented in two places that will drift
src/evaluatorq/redteam/README.md:101-128 already documents replay, and #165 had to backfill two facts from it (8+ char run-id prefix, newer-format rejection).
Lean: one source — move the shared fact-set into docs/_snippets/ and transclude into both, or cut the guide's mechanics to a link. Needs src/ scope.
3. Should replay be a peer bullet under ## Modes?
Its own first clause says it isn't reached by mode=, unlike the other three bullets. axes.md models replay as a mode value for coverage bookkeeping; #165 carried that into reader-facing prose.
Lean: kept the bullet (discoverability was why the gap ranked first, and every review persona found the section) — but this is a judgement call about how the repo models replay.
4. bt_sigma_aggregation is a false-positive Tier 1 gap
Flagged by docs-coverage purely on a symbol-name grep. It's internal (called only from build_report(aggregation='bt-sigma')); users reach the concept through report.bt_sigma, documented at pairwise-judging.md:168-245.
Lean: add a note to axes.md so a symbol with no prose but documented output stops surfacing, or coverage re-derives it every week.
5. Make the docs-autofill step-4 receipt a hard gate
A nonzero block exit currently lands in a table the routine then ignores. This run proved the risk: round-1 block01 genuinely failed (exit=127, eq not on PATH) and the stale receipt shipped into the critic prompts until two critics caught it. The skill also specifies how Python blocks execute but is silent on bash.
Lean: abort before step 5 on any nonzero exit; specify bash blocks run with the project venv on PATH; run the receipt against a fresh run store. Change lives in .claude/skills/docs-autofill/SKILL.md.
Filed so these survive #165 closing — @currentlycodinng flagged in the PR review that its five "Open decisions" read like separate tickets and would get lost once the PR merges. The doc content in #165 is accurate either way (independently verified against source in that review); none of these block it.
Each keeps the PR body's Lean. Two need a human call by nature (#1 is a code change, #3 is a judgement call); the other three are mechanical once someone says go.
1.
modesentinel inred_team()silently swallowsmode="dynamic"on replayprevious_run=rejects eight data-selection args by name but not the ninth.modedefaults toPipeline.DYNAMIC(runner.py:490), soPipeline(mode) != Pipeline.DYNAMIC(runner.py:698) can't tell "passed dynamic" from "passed nothing"; the value is then overwritten by the stored pipeline.max_turnstwo lines away uses exactly the sentinel that would fix it.--mode dynamicon the CLI has the identical hole.Lean:
mode: Pipeline | str | None = None, resolve toDYNAMICafter the conflict check, same forcli.py; add a test (test_replay.pycurrently asserts only the eight that raise). Non-breaking. Documented in a!!! notefor now because the routine may not touchsrc/.2. Replay documented in two places that will drift
src/evaluatorq/redteam/README.md:101-128already documents replay, and #165 had to backfill two facts from it (8+ char run-id prefix, newer-format rejection).Lean: one source — move the shared fact-set into
docs/_snippets/and transclude into both, or cut the guide's mechanics to a link. Needssrc/scope.3. Should
replaybe a peer bullet under## Modes?Its own first clause says it isn't reached by
mode=, unlike the other three bullets.axes.mdmodels replay as a mode value for coverage bookkeeping; #165 carried that into reader-facing prose.Lean: kept the bullet (discoverability was why the gap ranked first, and every review persona found the section) — but this is a judgement call about how the repo models replay.
4.
bt_sigma_aggregationis a false-positive Tier 1 gapFlagged by
docs-coveragepurely on a symbol-name grep. It's internal (called only frombuild_report(aggregation='bt-sigma')); users reach the concept throughreport.bt_sigma, documented atpairwise-judging.md:168-245.Lean: add a note to
axes.mdso a symbol with no prose but documented output stops surfacing, or coverage re-derives it every week.5. Make the docs-autofill step-4 receipt a hard gate
A nonzero block exit currently lands in a table the routine then ignores. This run proved the risk: round-1
block01genuinely failed (exit=127,eqnot onPATH) and the stale receipt shipped into the critic prompts until two critics caught it. The skill also specifies how Python blocks execute but is silent on bash.Lean: abort before step 5 on any nonzero exit; specify bash blocks run with the project venv on
PATH; run the receipt against a fresh run store. Change lives in.claude/skills/docs-autofill/SKILL.md.