Skip to content

workflow-fix #777: flip dual-DV continuous-companion polarity (fixed +/- margin PREFERRED, logp_pos_mean OPT-IN)#579

Closed
superkaiba wants to merge 2 commits into
mainfrom
issue-777
Closed

workflow-fix #777: flip dual-DV continuous-companion polarity (fixed +/- margin PREFERRED, logp_pos_mean OPT-IN)#579
superkaiba wants to merge 2 commits into
mainfrom
issue-777

Conversation

@superkaiba

Copy link
Copy Markdown
Owner

Closes task #777.

Summary

Corrects the dual-DV continuous-companion guidance in the project's
measurement-validity rules. The PREFERRED non-saturating continuous DV
is now the teacher-forced fixed positive-vs-negative completion margin
(fixed answer set across all contexts → no selection-on-outcome bias);
the length-normalized log-P of judged-positive on-policy completions
(logp_pos_mean) is flagged as an OPT-IN alternative that carries
selection-on-outcome bias and FAILED task #722's ρ(DV, rate) > 0
validation gate for 3 of 4 behaviors (ρ ≈ −0.3).

Evidence

Task #722's eval_results/issue_722/tf_margin/margin_chain.json:

  • margin ρ(DV, rate): refusal +0.56, sycophancy +0.40, broad_em +0.31 — all positive.
  • logp_pos_mean ρ(DV, rate) ≈ −0.3 for 3 of 4 behaviors — FAILED the same gate.

Diff

10 markdown files (0 .py, 0 configs), +65/-13 lines total across 2 commits:

  • d30d1fd600 — polarity flip in CLAUDE.md + .claude/rules/{marker-leakage-measurement,llm-judging}.md + 5 enforcer agent files (planner.md §6, critic.md Stats lens, analyzer.md step 3, interpretation-critic.md Lens 1, clean-result-critic.md dual-DV lens).
  • 883352fcc9 — folded-in Codex Minor CONCERN: bump 18→20 guideline count references to acknowledge the new § E2 continuous-companion-DV recipe alongside the 18 judging guidelines.

5 MUST-NOT-CHANGE invariants preserved

  1. On-policy judged RATE stays PRIMARY.
  2. ρ(DV, rate) > 0 validation gate intact.
  3. Task #558: persona-panel read on the #543 Phase-2 adapters #432issue-621: rank-1 LoRA read/write vector implants #456 teacher-forcing caveat intact.
  4. Graded-judge primary-for-ranking-targets extension intact.
  5. Marker three-space recipe (Log and analyze ALL THREE spaces) untouched.

Test plan

  • scripts/select_step9c_tests.py --base main touched-file scope: 2318 pass / 5 fail / 6 skip.
  • All 5 failures verified pre-existing on main (byte-identical evidence): 3× the #745 scripts/issue744_dump_and_stream.py:87 bare-dotenv lint (tracked separately), 2× HF-Hub network-state audit-claim tests.
  • 4 polarity-survivor greps empty in all 10 touched files (no old-wording survivors).
  • 4 invariant-preservation greps present.
  • Both code-reviewers PASS-class (Claude PASS, Codex CONCERNS→Minor 18→20 count folded in).

🤖 Generated with Claude Code

superkaiba and others added 2 commits June 30, 2026 16:45
…ed fixed +/- margin (preferred); logp_pos_mean is selection-confounded opt-in

Per #722: the teacher-forced fixed positive-vs-negative completion margin (FIXED answer pools across all contexts => no selection-on-outcome bias) PASSES the rho(DV, rate) > 0 validation gate (refusal +0.56, sycophancy +0.40, broad_em +0.31; all-positive). The judged-positive-conditional-mean log-P (logp_pos_mean) is a conditional mean over an outcome-selected subset (selection bias) and FAILS the validation for 3 of 4 behaviors (rho ~= -0.3). Flip the PREFERRED/OPT-IN polarity in CLAUDE.md § Measurement validity + 5 enforcer agent files; add the recipe + failure-mode subsection to .claude/rules/llm-judging.md; add a pure additive cross-reference note to .claude/rules/marker-leakage-measurement.md (the marker three-space recipe is NOT touched).

Invariants preserved: on-policy rate PRIMARY, rho(DV, rate) > 0 validation gate verbatim, #432->#456 teacher-forcing caution, graded-score-primary ranking/regression extension, marker three-space recipe.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…fter § E2 addition

Codex code-review CONCERN (minor): the new § E2 continuous-companion-DV
section in .claude/rules/llm-judging.md added numbered guidelines 19 and
20, but three cross-references still advertised "18 guidelines":
  - .claude/rules/llm-judging.md:8 ("full 18-guideline recipe")
  - CLAUDE.md:20 ("full 18-guideline set")
  - CLAUDE.md:20 ("the 18 judging guidelines")

Update each to acknowledge both halves: "20-guideline (the 18 judging
guidelines plus the § E2 continuous-companion-DV recipe)". Preserves
the original 18-count provenance while making the addendum visible. No
recipe lines touched.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
@superkaiba

Copy link
Copy Markdown
Owner Author

Content already on main via concurrent-session rebase-merge (see main commits 588370960f polarity flip + 02e8e01cd7 count fix). This PR's original commits d30d1fd600 + 883352fcc9 were rebase-rewritten during that merge; gh pr merge --rebase now returns "branch can't be rebased" because the commit content is semantically present on main. Closing rather than deleting the branch so the artifact-confirmed history is intact.

@superkaiba superkaiba closed this Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant