task #589: estimator-fragility sweep of #536 clustered leakage-line reads#466
Open
superkaiba wants to merge 33 commits into
Open
task #589: estimator-fragility sweep of #536 clustered leakage-line reads#466superkaiba wants to merge 33 commits into
superkaiba wants to merge 33 commits into
Conversation
…uf phase=done after 151s. [task.py]
…79 phase=done after 182s. [task.py]
… latent one [task.py]
…c7 replication N/A, [task.py]
… effort=high write=True pol [task.py]
… effort=high write=True pol [task.py]
… effort=high write=True pol [task.py]
…tq phase=done after 30s. [task.py]
…lore-persona- [task.py]
…nged from v1 (kill-c [task.py]
…wx phase=done after 60s. [task.py]
[task.py]
…onomous plan-gate (tas [task.py]
[task.py]
…a7 phase=done after 120s. [task.py]
…lore-persona- [task.py]
…ncertainty estimator [task.py]
[task.py]
[task.py]
…d: credential [task.py]
…onomous plan-gate (tas [task.py]
…2 + all reuse cod [task.py]
…ne reads Refit each of #536's four clustered headline tests (405-secondary, 490-distance-adjusted, 505-loo-null, 478-flatness-null) under both cluster-robust OLS and persona-RE MixedLM on the parent's identical persisted joins (raw + mean-centered). Reuses issue536_recompute_driver (join code of record) + issue536_mixedlm_refit (#478 cell verbatim); the ONE swept dimension is the uncertainty estimator. #478 is the only estimator-conditional flip (cluster-OLS significant vs MixedLM null on both joins); #505 MixedLM is reported FAILED (singular boundary, 18 clusters) per the hard rule, never falling back. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
csv.writer defaults to \r\n; pin lineterminator="\n" at both writer sites so the committed CSV stays LF-only under the repo's .gitattributes and re-runs are byte-identical. Regenerate the deliverables + figures. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…(defensive guard, per Codex code-review CONCERN) Closes the Codex code-review v1 minor finding: previously _fit_mixedlm set status='FAILED' only on non-PD Hessian or degenerate SE; a plain res.converged=False (e.g. LBFGS iteration-cap exit) fell through to status='OK'. Current sweep outputs are unchanged — all #505 FAILED cells already routed via the non-PD / degenerate-SE branch — but a future re-run could classify a non-converged cell as inconclusive while still plotting its p-value as usable. The added elif handles it explicitly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
#505 pooled The persona-VC MixedLM cell on #505 (vc_formula={'persona':'0+C(b)'}) is singular (non-PD Hessian) on both joins, leaving the #505 row a "third outcome" rather than a real estimator comparison. Add _fit_mixedlm_groups_only_reml (groups-only random intercept on j_i|seed, REML, no persona VC) and 2 new cells (estimator=mixedlm_alt_vc, _vc_spec=groups_only_reml) re-fit over the SAME persisted #536 joins (zero GPU, no new data). The MixedLM hard rule is unchanged: FAILED on non-convergence / singular boundary, never a fallback estimator. The existing 16 OLS / persona-VC cells stay byte-identical; the new optional _vc_spec column distinguishes persona_vc (16) from groups_only_reml (2). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… rows) CSV now 18 data rows (16 standard + 2 #505 mixedlm_alt_vc groups-only-REML), + trailing _vc_spec column; JSON 18 cells + alt-VC estimator_spec / note / row_meta field. The 2 alt-VC cells CONVERGE: raw beta=+0.831 p=0.0024 (significant), centered beta=+0.055 p=0.168 (null) — turning the singular persona-VC "unfittable" outcome into a real estimator read. Existing 16 cells byte-identical. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ter sign-flip)
Standalone methodology + statistical-knob + worked-examples reference for the estimator-fragility sweep of #536's four clustered leakage-line reads. No interpretation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
superkaiba
added a commit
that referenced
this pull request
Jun 14, 2026
… conflict family) Branch unsafe to blind-rebase via gh pr merge: many cross-task tasks/ renames (#581/#609/#632/#634/#589 itself moved folders during the branch's life) produced rename/rename conflicts on tasks/589/* paths and an add/add conflict on tasks/running/634/events.jsonl when the worktree merged origin/main. None of the branch's commits TOUCH tasks/ (task.py commits go to main); the conflicts are entirely from foreign churn, not branch deliverables. Cherry-picked just the branch's own added deliverable files for #589 (scripts/issue589_*, eval_results/issue_589/, figures/issue_589/, docs/methodology/issue_589.md). Shared src/, scripts/, and tests/ unchanged. PR #466 will be closed; the deliverables land on main here. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes task #589.