task #611: probe-split re-analysis (zero-GPU)#449
Draft
superkaiba wants to merge 4 commits into
Draft
Conversation
scripts/issue611_split_analysis.py — Phase 0 fail-loud validation (245-file enumeration, schema/marker/eos/gauge asserts, encoding-identical default-probe base identity, stats-key standalone-base cross-check, 1.0-nat rig check vs cross_eval delta_g) + Phase 1 paired contrasts d = system − role in Δlog P and EOS-margin space (per-seed-paired bootstrap N=10,000, rng seed 0), band accounting incl. own-slot Δmargin, base-prior decomposition, 0.5-nat saturation flags, and the exhaustive plan-§7 verdict classifier (DISCORDANT label, pinned quantifiers) → eval_results/issue_611/split_analysis.json (i611_split_analysis_v1). scripts/issue611_split_figures.py — hero split_verdict_grid + base_prior_ decomposition + own_slot_regime + exploratory dump (per-seed scatter, leakage allocation, logp/margin agreement) → figures/issue_611/. Computed headline verdict: concentration_confirmed (matches plan §2 preview). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ern stale-artifact-commit-metadata) Re-ran scripts/issue611_split_analysis.py + scripts/issue611_split_figures.py post-commit so split_analysis.json git_commit and figures/issue_611/*.meta.json commit record 7a817a6 (the commit containing the generating scripts) instead of the pre-implementation parent 16cb4be. Content verified identical except generated_at/git_commit (JSON) and created/commit (meta.json); rng seed 0 determinism holds (headline verdict unchanged: concentration_confirmed). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…arm labels (drop in-bar sys/role abbreviations, round-2 critique)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes task #611. Zero-GPU re-analysis of #533's logit captures: wrong-persona vs default-assistant leakage split, EOS-margin + unsaturated reads.