Repository navigation
evaluation: noul_confidence must be |2p − 1| per the confidence page (same scale as Choice); document the Noul three-path pattern #140
Description
Context
From the Opus review of #136 (recipe 02, the first Noul recipe; comment 6071121590). src/jev_cookbook/evaluation.py and docs/evaluation.md define noul_confidence(p) = max(p, 1 − p) (range 0.5–1). The TypeSafe confidence page (https://docs.typesafe.ai/confidence.md, README S03) defines a Noul's confidence as |2p − 1| (range 0–1) and states it is the Choice formula (p_max − 1/n)/(1 − 1/n) applied with n = 2, so it sits on the same scale as Choice confidence. The toolkit helper therefore disagrees with the source, and a recipe that gates Nouls and Choices with one threshold, as the page recommends, gets wrong answers from it. docs/evaluation.md documents the Choice rescaling correctly on the same page.
Acceptance
noul_confidence(p)returns|2p − 1|per the confidence page (the oldmax(p, 1 − p)is removed or kept under a distinct, documented name such asnoul_majority);docs/evaluation.mdanddocs/glossary.mdstate the formula with the source link and note the shared scale with Choice.- Tests pin the formula at p ∈ {0, 0.25, 0.5, 0.75, 1} and the identity with
(p_max − 1/n)/(1 − 1/n)at n = 2; mutation: revert tomax(p, 1 − p)→ tests fail. - Every call site in
src/,recipes/_template,tools/new_recipe.pyand merged recipes is checked; any notebook whose outputs change is re-executed and committed byte-identically in the same PR (list them). - Add to
docs/evaluation.mda short "Noul three-path pattern" note: a business threshold on p plus a confidence gate|2p − 1| < c→ review outcome, composed fromthreshold_sweep/evaluate_threshold, since no helper selects both together (the Score recipes reported the same gap).
Required before the remaining Noul recipes (#6, #10, and later waves) are briefed. Foundation issue; allowed paths src/jev_cookbook/evaluation.py, tests/test_evaluation*.py, docs/evaluation.md, docs/glossary.md, plus re-executed notebooks only if their outputs change.
Activity
CompleteDotTech commentedon Oct 8, 2026
Governance G140-1 — source-list integration dependency (read-only reconciliation).
The primary’s same-worker #136 fix directive requires adding S03 to recipe 02’s README Sources. Current catalog rank 2 lists only S02/S04; current recipe-02 WIP README still lists S02/S04. The already merged #133 test
tests/test_recipe_sources.py::test_the_readme_sources_match_the_catalogcompares the exact source-ID sets for every published notebook. Adding the instructed S03 only inside recipe 02 will therefore fail that gate.#140 currently excludes catalog/recipes.json, and the existing recipe integration exception permits only generated root README regions. Neither currently authorizes the recipe builder to edit the catalog. Please record narrowly scoped ownership for the catalog source synchronization (either explicitly reviewed foundation scope here or a separate owned integration task) before #136’s final CI/review. Retain the source-consistency test and recipe-directory ownership. No competing repair or catalog edit was started by governance.
For #134, adding S03 is contingent on its actual corrected source list; do not assume a catalog change until that list is settled. The shared Noul-confidence correction and #136’s existing push hold remain owned by the primary and their current builders.
Source ownership reconciled: primary opened #142 as the narrowly scoped catalog source-sync foundation and assigned abc27e915711a31df (actual claude-sonnet-5 verified). G142-1 comment6071276395 records that active recipes02/03/04/05 ALL currently omit S03 and need matching same-worker README updates after delivery. This resolves missing scope/owner, not completion: independent final-head review, signed pinned merge and exact-main checks remain.
CompleteDotTech commentedon Oct 8, 2026
G140-2 — native dependency reconciliation for future Noul recipes.
#140 explicitly requires delivery before remaining Noul recipes (#6, #10 and later waves) are briefed. Governance verified and added the missing blocked-by #140 relationships on #6/#10, then audited all eighteen later-wave catalog recipes whose decision_types includes Noul: #15, #25, #26, #31, #35, #37, #39, #41, #42, #44, #48, #50, #53, #54, #56, #57, #58 and #60. Each was open and lacked this native blocker. All eighteen additions were read back; every pre-existing dependency was preserved.
This now covers all twenty FUTURE Noul recipes in the catalog. Existing recipe02 remains under the primary’s separately recorded #140 push hold; no new recipe02 dependency is claimed. #60 is only the Cookbook’s self-contained Python simulation recipe: Factorio sessions/worlds/controllers remain stopped and untouched.
No builder was dispatched, no source was edited and no dependency was removed. A relationship is not delivery evidence: retain genuine final signed-head independent approval, complete CI/current-main integration, primary pinned serial merge and exact-main verification before briefing/advancing affected work. Full per-issue before/POST/readback receipts are in the existing governance handoff directory.
Fix round 1: three-path Noul rule, simulated queue, honest selective …
Fix round 1 follow-up: a genuinely ambiguous, mixed-label cluster for…