task #595: Prefix-carrier (Piggyback) binding strength as B→B′ leakage predictor#467
Draft
superkaiba wants to merge 22 commits into
Draft
task #595: Prefix-carrier (Piggyback) binding strength as B→B′ leakage predictor#467superkaiba wants to merge 22 commits into
superkaiba wants to merge 22 commits into
Conversation
notes: smoke IS the full run with one [task.py]
…tc phase=done after 212s. [task.py]
# Codex Code Review: Secre [task.py]
…entation v1 --> ## Implementation [task.py]
… FP, 2 SKIP) on plan [task.py]
…orated reconc [task.py]
[task.py]
…onomous plan-gate (tas [task.py]
## Reconciler Verdict — FA [task.py]
Post-hoc explanatory pass over #545's 19 frozen LoRA adapters (no training). - scripts/issue595_prefix_carrier.py: GPU driver. Phase 1 prefix-KV-shift (MSRD, TReFT eq.) per adapter -> raw all-L / layer-9 / gauge-normalized (divisor (alpha/sqrt(r))**2, the v3 squared-norm correction) predictor JSONs + per-layer profile. Phase 2 prefix-patch leakage recovery via a KV-cache substitution wrapper (HF generate; backend-parity HALT on bad_medical broad_em vs #545 L=0.1128). Phase 3 postfix + query controls. rsLoRA parity probe + per-recipe gauge-band assert gate Phase 1. --smoke = bad_medical, cap 4, full Phase 1->4 in-process serial (PASS_UNIFIED). - scripts/issue595_score_and_correlate.py: CPU off-pod Phase 4. Copies #545 predictors, runs score() with PFX in the held-out race, computes H1 Spearman rho trio (raw / L9 / gaugenorm_sq) with family-clustered bootstrap CIs. - scoring.py: 1-line groups-tuple extension ("PFX") admitting the prefix-binding family to the leave-family-out CV / quarantine race; idempotent on A/B/C/D. - tests: squared-gauge MSRD math, groups-extension idempotency (real #545 inputs), prefix-position pins, KV-cache patch-hook correctness. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reconciler FAIL fixes for the prefix-carrier Phase 2/3 patch path: - B1 (donor contamination): capture the base prefix KV under model.disable_adapter() on the PEFT-wrapped model instead of running a bare base(...) forward through the in-place-injected LoRA. Add detach_adapter() (base_model.unload()) for cross-row hygiene in both Phase 1 and Phase 2/3 — the round-1 `hasattr(model, "unload")` guard was always False (unload lives on base_model), so adapters silently stacked across rows and the donor was the trained, not base, prefix KV. - B2 (patch ordering): replace the post-attention cache rewrite with a full Qwen2Attention.forward override that splices base prefix K/V into key_states/value_states AFTER past_key_values.update() but BEFORE attention_interface() reads them, so the prefill attention output and the first-generated-token logits are computed against base prefix K/V (the Piggyback intervention), not just future decode reads. - Seed: thread DECODE_SEED=545 into every generate_patched call (matches #545 SamplingParams(seed=545)). - Cleanup: remove the unreachable "k" branch in _rate_from_summary. Tests (test_issue595_patch_hook_correctness.py rewritten): pin the prefill last-position logits CHANGE under a different-base prefix patch (FAILS under the round-1 post-attention rewrite, verified out-of-band), self-patch bit-identical regression guard, layer-0 localization + downstream propagation, donor-captured-under-disable_adapter vs active-adapter, and detach_adapter restores pristine base. 16/16 issue595 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…or issue503 judges (CBL2) + NaN->null gauge power (CBL3) CBL1 (phase1-rslora-parity-base-contamination): rsLoRA_parity_check injected the bad_medical LoRA into `base` in place and returned None; `del model` did NOT strip it, so the Phase 1 row loop read its first donor base KV off a contaminated base, invalidating every prefix-KV-shift predictor (the headline). Fix: rsLoRA_parity_check now detach_adapter()s and returns the LoRA-free base with a mandatory n_lora==0 assert; the run_phase1 call site re-binds `base = rsLoRA_parity_check(...)`. New test test_parity_probe_handoff_leaves_lorafree_base pins the handoff. CBL2 (issue503-judge-modules-uncommitted): the #545 judge harness (behavior_testbed_545/eval_battery.py:827/882/960, inherited by #595's judge path) imports explore_persona_space.experiments.issue503.{judges, advbench_judge}. Those modules live on issue-545/issue-503 but #545's Step 9b auto-merge never carried them to main, so a clean pod cloned from main crashes at the first judge call. Vendor the issue503/ tree from issue-545 onto issue-595 (lands on main as a side effect at #595's merge, unblocking every future task inheriting the #545 judge harness). CBL3 (layer9-gauge-power-nan-json): the L9 predictor variant wrote gauge_normalization_power=float("nan"), which json serializes to the non-standard `NaN` literal that strict parsers reject. Use None -> null for the descriptive (non-gauge-normalized) L9 variant; raw=0, gaugenorm_sq=2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…ge=45.25, inherited from #503 Bucket-D recipe)
…d5944) Phase 1 prefix-KV-shift: 3 PFX predictors (raw all-L, layer-9, gaugenorm_sq) + per_layer_profile.json (raw-alongside). Phase 2 prefix-patch recovery: PFX__patch_recovery.json (8 rows × 6 cols = 48 cells). Phase 3 controls: PFX_ctrl_postfix.json + PFX_ctrl_query.json (bad_medical seed-0). Phase 4 scoring + correlation: scoring_prefix/scoring_results.json + prefix_binding_correlation.json. Plus 104 #545 predictor JSONs copied for the predictor race. Pod: pod-595 (1xH100). Backend-parity asserted vs #545 vLLM L on bad_medical broad_em. Raw completions (18 files) excluded from git per Upload Policy — uploader at Step 8 pushes to HF data repo.
… + per-layer + gauge confound) + corrected H3 scoring repro
…-result artifact links
…se) + gauge-mediation + single-metric H3
… from figure text Plain-English chart titles / legends / axis labels per clean-result-critic Lens 2/3. Numbers unchanged (raw ρ=+0.53, gauge-corrected −0.01, held-out CV 0.108). Removed the per-fold B1/B5 detail and the on-figure PFX source path; provenance lives in the body Reproducibility section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
superkaiba
added a commit
that referenced
this pull request
Jun 14, 2026
experiments.issue503 was vendored on the issue-595 branch (PR #467) and never merged to main, so these 3 tests ModuleNotFoundError'd at collection on main, poisoning every Step 9c test-verdict pytest run fleet-wide (recurred on #630/#631, live on main 2026-06-13). Guard with pytest.importorskip so they skip-with-reason until the module lands, then run automatically. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes task #595.