[projection-agent] Publish NFugu JudgeEval codex fuse follow-up

[susyimes]

What changed

• Added signature-aware result reuse/upsert filtering for NFugu JudgeEval v0 so cached rows are not reused across incompatible candidate configurations.
• Added candidate-level codex_double reporting semantics while keeping condition-level summaries clear.
• Updated NFugu JudgeEval docs and tests for the CodexFuse/CodexDouble follow-up.

Why

This is a separate follow-up slice for the dirty D:\memsuOS worktree after the earlier docs-only and JudgeEval v0 PRs. Worker review found stale-cache semantics and misleading codex_double reporting risks, so the changes were refined before publication.

Validation

• python -m unittest tests.test_nfugu_judge_eval passed 15 tests.
• python -B scripts\run_nfugu_judge_eval.py --conditions single,codex_fuse --provider-preset mock --limit 2 --out-dir .memsuos\evals\codex-fuse-judgeeval-v0-mock-closeout passed.
• python -B scripts\run_nfugu_judge_eval.py --conditions single,codex_double --provider-preset mock --limit 2 --out-dir .memsuos\evals\codex-double-judgeeval-v0-mock-closeout passed.
• git diff --check passed with LF-to-CRLF warnings only.

Audit

• atprojection task: D:\atprojection-agent\staged\2026-07-02-1625-memsuos-codex-fuse-judgeeval.md
• worker session: 019f21f9-0f8a-75e1-a82c-a45a0c4f0612
• rollback: close this draft PR and delete branch codex/projection-agent-memsuos-codex-fuse-judgeeval; no default-branch merge has been performed.

[susyimes]

Audit result: blocked for merge.

PR #7 fixes stale result reuse and adds useful CodexDouble/CodexFuse coverage, and the local tests/smokes pass. However the CodexFuse summary/report semantics are still misleading when the conservative selector keeps C0.

Blocking issue

• In a low-confidence codex_fuse run, select_codex_fuse_final() correctly keeps winner_candidate_id=C0, but judge_conditions() / write_summary() report the unaccepted pool-judge preference as the row winner and as codex_fuse_metrics.chose_c1/chose_f1.
• Repro from an isolated audit checkout:
  • PYTHONPATH=src python -B scripts/run_nfugu_judge_eval.py --conditions single,codex_fuse --provider-preset mock --limit 2 --out-dir .memsuos/evals/codex-fuse-judgeeval-v0-mock-audit
• Evidence from generated results.jsonl:
  • bcb_hard_001: selection_winner=C0, reason monotonic guard kept baseline because judge did not strongly select C1 or F1, pool_confidence=0.1, judge_winner=C1, final_equal_single=true
  • bcb_hard_002: selection_winner=C0, same monotonic-guard reason, pool_confidence=0.1, judge_winner=F1, final_equal_single=true
• Evidence from generated judgeeval-summary.json:
  • judge_wins={"C0":0,"C1":1,"F1":1}
  • codex_fuse_metrics={"rows":2,"final_equal_single":2,"chose_f1":1,"chose_c1":1,"kept_c0":0}
• This contradicts the selector result and the doc statement that if the judge does not strongly select C1/F1, the final answer stays C0.

Likely source

• scripts/run_nfugu_judge_eval.py codex_fuse_candidate_pool_judge() prefers pool_judge.winner_candidate_id over selection.winner_candidate_id.
• write_summary() then uses row.judge.winner_condition for codex_fuse_metrics, so low-confidence pool preference is counted as an accepted win.

Expected fix

• Make report-level winner and codex_fuse_metrics.chose_* / kept_c0 reflect the accepted final selection (selection.winner_candidate_id).
• If the raw pool judge preference is still useful, record it separately, e.g. pool_judge_preference / pool_judge_wins, without counting it as selected output.
• Add a regression test where pool_judge.winner_candidate_id is C1 or F1 with confidence below threshold but selection.winner_candidate_id is C0; assert the report/summary winner is C0 and kept_c0 increments.

Audit validation already run

• git diff --check origin/main...HEAD: pass.
• PYTHONPATH=src python -m unittest tests.test_nfugu_judge_eval: pass, 15 tests.
• PYTHONPATH=src python -m unittest tests.test_nfugu tests.test_eval_scripts tests.test_agent_routing tests.test_intelligence_layer: pass, 79 tests.
• python -m py_compile scripts/run_nfugu_judge_eval.py: pass.
• gh pr checks 7 --repo susyimes/memsuOS: no checks reported.

I did not merge the PR because the generated eval summary can currently claim CodexFuse selected C1/F1 when the actual final selected answer was C0.

[Copilot]

Pull request overview

This PR updates the NFugu JudgeEval v0 harness to support CodexDouble/CodexFuse follow-up semantics, including signature-aware caching to prevent stale result reuse and clearer candidate-level reporting when the “condition winner” is actually a specific candidate (C0/C1/F1).

Changes:

• Added --conditions selection logic with codex_double / codex_fuse support and enforced single baseline ordering.
• Introduced run signatures (run_signature) and compatibility filtering for result reuse/upserts and summary/report generation.
• Added CodexDouble/CodexFuse generation + conservative selection flow, plus updated tests and docs.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
tests/test_nfugu_judge_eval.py	Adds unit tests covering condition selection ordering, provider defaults, run signature reuse, upsert filtering, and CodexDouble/CodexFuse reporting semantics.
scripts/run_nfugu_judge_eval.py	Implements condition selection, run signatures, compatible cache reuse, CodexDouble/CodexFuse generation & judging, and updated summary/report outputs.
docs/project-spec-index.md	Updates the index entry to reflect CodexFuse sanity-check availability while NFugu remains paused.
docs/nfugu-judge-eval-v0.md	Documents CodexFuse/CodexDouble semantics, caching/run-signature behavior, and example invocation commands.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.