Task #605: matched-geometry bystander panels (two-term leakage gate test)#447
Draft
superkaiba wants to merge 19 commits into
Draft
Task #605: matched-geometry bystander panels (two-term leakage gate test)#447superkaiba wants to merge 19 commits into
superkaiba wants to merge 19 commits into
Conversation
…text module - Surgical checkout of run_experiment_541.py / run_experiment_500.py / issue541_personas.py / issue541_prescreen.py from origin/issue-541 at pinned SHA b1aad02 (plan 4.8 branch-port step); VM import smoke passes (run_experiment_541 imports cleanly against main's run_experiment_444 + library modules). - scripts/issue605_contexts.py: 120 marker candidates (content x affordance crossing, plan 4.1) + 20-pool pre-registered expansion round + 54 fact candidates (proximity x entity-familiarity, plan 4.5) + lints (glyph hygiene, dedupe, DV-contamination, demo/probe disjointness).
…2 more branch files - issue605_matched_panels.py: Phases 1/1.5/4/4.5 (vLLM gen, HF four-float corrected-slot priors, L21/L22 acts, pair table w/ centered-bank + negsim columns + legacy drift QA, tercile band edges + matched-stratum gates, rendered-string disjointness asserts, fact TF priors/FP screen/acts). - issue605_eval_marker.py: Phase 2 dispatcher (vLLM LoRA hot-swap gen, HF slot reads BOTH sides at identical slot in the #532 follow-up schema, A1 self-cell adapter-application smoke gate, bulk fail-loud HF upload, /issue Step 7 sentinel, [phase=done] reserved to top-level). - issue605_eval_fact.py: Phase 5 dispatcher (12-unit x 5-paraphrase headline rows, Messages-Batch 5-way Haiku judge w/ #541 verbatim prompt + 529-safe retries, TF scoring trained+base via engine-parameterized #541 metric, teacher-self adapter smoke, upload + sentinel). - issue605_analysis.py: registered statistics (pooled matched-sim partial w/ context-cluster bootstrap + Holm; grouped-CV delta-R2 w/ full-CV-per-resample cluster bootstrap), pinned Tobit/rank primary fits, sign-routed lattice, 11.5 diagnostics, figures, synthetic-stub CPU smoke path. - Partial-port reconciliation (pre-launch diff scan): main's issue444_persona_distance_topic.py lost _resolve_persona_prompt and main's personas.py lost the courthouse_architecture_historian teacher (both branch-only, never merged); ported both from b1aad02 (strict supersets of main's copies).
…lysis stub censoring - issue605_eval_fact.py: re-pin PROJECT_ROOT at the eval.exp444_judge_prompts import site (issue444_bystander_logprob re-inserts scripts/ at position 0, and scripts/eval.py then shadows the top-level eval/ package — caught by the stage-all dry-run smoke). - issue605_analysis.py: stub priors -20..-8 so the planted positive prior->shift effect is recoverable in BOTH spaces (the previous stub ceiling-censored most high-prior cells, nulling the raw log-prob partial while Tobit recovered it — faithful but a weaker smoke assertion).
- delta-CV-R^2 bootstrap groups folds by the ORIGINAL context id so duplicate resampled copies share a fold (cv-bootstrap-cluster-leakage blocker; null sim: old CI ~2.3x too narrow with inflated >0 mass) - failed Phase-1.5/4.5 selection gates now exit non-zero unless --allow-descope records the pre-registered descope-to-populated-bands path; both eval dispatchers refuse gate_pass=false panels without it (panel-gate-not-enforced) - marker analysis filters per-cell files to the selected (or descoped) panel and asserts per-source coverage; fact frame mirrors the gate/descope check - fact gauge assert adds modules_to_save; split-probe diagnostic uses the half-A per-q prior; H-level lattice routing documented in code + JSON; dead MAX_NEW_TOKENS twin removed Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
Deployment failed with the following error: Learn More: https://vercel.com/thomasjiralerspong-5386s-projects?upgradeToPro=build-rate-limit |
…ansion fact wiring 42 FACT_EXPANSION personas targeting the three Phase-4.5 gate-failure corners (mb mid/hi spread + same-12-contexts overlap, wfc mid/hi collinearity 0.62, mb/cah band_lo spread); thread --include-expansion through _resolve_fact_candidates/_run_fact/fact_select (mirrors the marker mechanism), record expansion_round in both selection JSONs, and resolve expansion labels in the Phase-5 eval persona pool. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…5 eval entrypoint (fail-loud defense-in-depth) Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
…rms in Phase-7 analysis A fact arm with gate_pass=false, zero passing bands, and no possible descope (the plan section-3 structural-alternative outcome; live case: marine_biologist after the pre-registered expansion round) is now DROPPED from the fact frame instead of blocking the whole battery: build_fact_frame derives arms_dropped_structural from the selection JSON, skips the arm's stale on-disk smoke cells, and analyze_fact records the structural finding + per-arm gate flags in analysis.json. Every other absence stays fail-loud (gate-fail with surviving bands but no descope; missing per-cell files or an entirely-missing surviving arm — the latter via a new assert). --stub-variant descoped exercises the path end-to-end on CPU. Analysis-only; no dispatcher edits. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…el selections) — JSONs only, npz on HF Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
… lattice, hero figures) Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
…lect-wide (40->100), --skip-completed enumeration, parent regression check - matched_panels: --panel-size + --frozen-selection (marker, select-only); _select_panel_frozen reuses parent edges/ranges/windows verbatim (no quantile recompute, no window slide); _prune_panel gains protected (parent 40 never pruned; superset asserted); fill scales linearly (25 at panel 100); wide record -> NEW marker_panel_selection_wide.json with frozen_from/panel_inherited/panel_new/realized_prior_distribution + prior-dist QA figure; disjointness re-run on full wide panel (source renders hoisted, identical comparisons) - eval_marker: --skip-completed flag (always-on per-cell existence skip; flag logs explicit skipped/pending enumeration; plan §5 launch spelling), threaded through --stage all dispatch - analysis: --marker-selection + --out; build_marker_frame takes selection override; FREE parent regression check (combined frame restricted to panel_inherited must reproduce parent pooled partials, |diff|<1e-6, fail-loud both values printed; verified bit-identical on the real 640-cell frame: +0.26298 logprob / +0.55031 EOS-margin) - commit realized wide selection record (gate_pass at fill 25, panel 100 = 40 inherited + 60 new, bands 92/95/77 in-window all gates PASS) + QA figure Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…res-dir guard Coverage assert now checks the registered 16 source cids (CONDITIONS_BY_ID, same list the eval dispatcher runs) instead of the observed frame, so a source with ZERO per_cell files fails loud naming the source; stub trees override explicitly (gated on --synthesize-stub, logged). Amendment runs (--marker-selection/--out) default figures to a dedicated subdir and refuse the parent figures/issue_605 (fixed filenames would overwrite committed hero figures). Parent 640-cell pooled partials verified bit-identical. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… selection record Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
…-frame analysis (1600 cells/100 contexts) + figures
… (findings-blind reference)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes task #605. Eval-only: matched-geometry panels testing whether the rank-1 leakage gate needs a behavior-overlap term beyond context similarity.