task #657: behavior-direction alignment predictor (bake-off vs base prior)#482
Open
superkaiba wants to merge 91 commits into
Open
task #657: behavior-direction alignment predictor (bake-off vs base prior)#482superkaiba wants to merge 91 commits into
superkaiba wants to merge 91 commits into
Conversation
…kage bake-off) Training-free reuse-only analysis pipeline (plan v3): - issue623_extract_sycophancy_vector.py: additive --trait-name / --trait-description CLI args so the #623 Persona-Vectors recipe extracts refusal/marker/em directions (--trait-name=sycophancy reproduces #623 verbatim). - analysis/issue657_alignment_predictor.py: compute_alignment, prior_centroid_projection (M-Alts1 geometry-side prior control), partial_spearman (singly + doubly), lopo_heldout + grouped-CV ΔR², bootstrap_over_personas (held-out bystander = resampling unit), shuffled_direction_null, attenuation_sensitivity (M-Stats1 reliability band), and the #518/#605 cell-substrate loaders (marker join restricted to 24-panel-overlapping personas; wrong-marker-token fail-fast assert). - issue657_join_substrate.py (Phase 0), issue657_run_bake_off.py (Phase 3 unified in-process-serial dispatcher; smoke = sweep with one cell), issue657_plot.py (Phase 4). - issue657_extract.sh (Phase 1 pod dispatcher; [phase=...] + [phase=done] + sentinel). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…n + regression tests - bootstrap_over_personas_multi: collect held-out Spearman + grouped-CV ΔR² + doubly-partialled rho from ONE resampling pass (the held-out Spearman and ΔR² both come out of a single lopo_heldout call; computing them in separate bootstraps re-ran the expensive grouped-CV per statistic — one behavior's 3x B=10000 exceeded 5 min). _grouped_cv_delta_r2 now slices prebuilt numpy arrays in the fold loop instead of per-fold dict comprehensions. - bake-off rewired to the single-pass bootstrap; ~6 min/behavior at B=10000. - _load_da_base_rates warns LOUDLY when the #623 syc_i file is absent (the sparse worktree excludes eval_results/) so the sycophancy anchor never silently falls back to bystander_base_rate unnoticed. - tests/test_issue657_alignment_predictor.py: 10 CPU-only guards (marker-token fail-fast, doubly-partialled removes the covariate, prior_centroid weighting, bystander-resampling unit, 3-point reliability band, marker band-skip, additive --trait-name default reproduces #623). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…l EIV disattenuation) The round-1 band scaled prior*sqrt(R) then rank-residualized, which is rank-invariant, so all band points were byte-identical and the INDETERMINATE carve-out was unreachable (could falsely PASS a noise-inflated partial). Replace with classical errors-in-variables disattenuation on the rank-correlation (partial_spearman_eiv: residualize the exact prior_centroid_projection first, then the disattenuated bivariate partial formula with the noisy prior's cross-correlations scaled 1/sqrt(R)). Add delta_r2_align_beyond_prior_eiv (closed-form incremental R2 via the same disattenuation). attenuation_sensitivity now bootstraps both statistics' per-band-point CIs over the held-out bystander (B=10000 to match the headline gate; b=1 in smoke). Also: PRIMARY_H3_BEHAVIORS is now a literal frozenset (N1). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…d-2 regression tests B2: the H3 gate now reads the per-band-point bootstrap CIs (partial-rho AND delta_r2_align_beyond_prior CI lower bound > 0 across the whole reliability band == CONFIRMED; optimistic-end-only == INDETERMINATE), not point-estimate signs. _band_clears consumes the per-point CIs attenuation_sensitivity now produces; _h3_summary PASSes only on a CONFIRMED band. N2: a missing #623 syc_i.json HALTS a non-smoke production sycophancy run (the join-integrity kill criterion) unless --allow-syc-i-fallback is passed; smoke still warns-and-falls-back. Adds 10 regression tests: the EIV band moves the partial (not a no-op) and carries per-point CIs (B1), the H3 verdict is not PASS when a band point's CI straddles 0 (B2), PRIMARY_H3_BEHAVIORS is a frozenset excluding EM (N1), and the syc-i production halt vs opt-in fallback (N2). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…preflight F1 (log visibility through GCP instance DELETE): tee every per-phase output to the driver's own stdout (= <vm_scratch_dir>/logs/issue-657.log, the GCP handle log_path the poller tails LIVE before the EXIT-trap DELETE wipes the disk) AND the per-behavior file; fix the per-behavior mark so fail()'s tail names the right log; dump the tail to stdout in fail() + upload it to the HF data repo (_failure_logs/) as the durable cross-DELETE backstop. Round 1's `>file 2>&1` kept the refusal traceback off the poller-visible main log and DELETE erased it. F3 (fail in ~1s, not 30min): assert ANTHROPIC_API_KEY at the top of the dispatcher when the behavior_directions phase will run — the Claude artifact generator needs it, but the GCP backend lists it only in STARTUP_SECRET_ENV_KEYS (dropped when absent), not REQUIRED_LAUNCH_SECRET_KEYS, so a missing key crashed mid-extraction. Adds --print-trait-description <beh> so the CPU smoke reads the dispatcher's canonical trait strings (no drift between smoke and production path). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…r/em path Round 1 launched on GCP with the refusal/marker/em extraction path UNTESTED live (round-1 smoke only ran --trait-name=sycophancy). The Claude artifact-generation step (generate_artifacts) is a network/API call testable without a GPU; this smoke calls it for each of refusal/marker/em with the dispatcher's exact trait_description and asserts 5 instruction pairs, >= n_questions, and a non-empty placeholdered eval_prompt. Catches malformed descriptions (non-JSON), trait-name/description mismatches, and a missing ANTHROPIC_API_KEY. Always-on layer (no API) pins the dispatcher's three canonical descriptions; the live layer skips loud without the key so the keyless CI sweep stays green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…burst Launch attempt 2 crashed mid-refusal extraction on anthropic.RateLimitError (429, 1M output-tokens/min org cap) — the SDK default max_retries=2 exhausted on a transient burst. Three layered mitigations in generate_artifacts(): - R1: anthropic.Anthropic(max_retries=10, timeout=600.0) — SDK backoff rides through ~10min of short 429 bursts (respects retry-after). - R2: max_tokens 8192 -> 4096 — halves per-call output-token allocation vs the org cap (artifact responses run ~2000-3000 tokens; 4096 is ample headroom). - R3: outer 3-attempt retry with 60/120/180s sleeps for sustained 60s+ bursts wider than the SDK's ~10s-capped backoff; re-raises the captured RateLimitError if all attempts 429. Smoke 8/8 (5 always-on + 3 live generate_artifacts), alignment-predictor 20/20. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…efusal diff-mean + EM shift) NEW scripts/issue657_extract_diffmean_direction.py: the one genuinely new v6 code file. Two extraction modes under one CLI (--behavior): - refusal: Arditi diff-in-means (arXiv 2406.11717) over harmful AdvBench / harmless Alpaca banks at post-instruction-token positions (pure base-model read, PRIMARY). - em: #521 on_policy_em trained-shift re-extraction from the SINGLE on-HF issue_519/em_seed42 adapter (r=8/alpha=16, config-pinned fail-loud; em_turner_seed42 DROPPED) — L14 trained-base mean-response shift (trained-shift read, SECONDARY M-Alts2). Reuses #623's run_steering_probe verbatim for the K2 sanity gate (refusal -> held-out harmful+harmless probes; EM -> manifest questions under bare assistant); K2-fail does NOT raise (dispatcher routes the fallback). Writes the direction under both readout names per layer matching load_behavior_direction. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…_predictor.json + Phase-0 overlap gate Analysis library: - effective_primary_h3(): run-conditional primary set — EM always secondary (M-Alts2); marker demoted to secondary on the #521 trained-shift fallback (M-Alts3) or the Phase-0 panel-overlap exclusion. - marker_panel_overlap(): the FIRST Phase-0 deliverable — |EVAL_PERSONAS_24 ∩ #605-marker-panel| from the REAL substrate; overlap < 8 -> marker DV-(a)-only. - MARKER_MIN_OVERLAP single source of truth (imported by the bake-off). Bake-off: - reads marker_direction_kind (affordance|shift_fallback) + per-behavior k2_pass; threads effective_primary_h3 into is_primary_h3 + _h3_summary (reads the per-behavior flag, single source of truth; K2-failed direction excluded not nulled). - emits the §6.5 PRIMARY DELIVERABLE alignment_predictor.json (raw/singly/doubly- partialled rho, prior_centroid_projection, ΔR², 3-point reliability band, primary_vs_secondary, marker_direction_kind, marker_panel_overlap, per-behavior k2_pass, CIs) + per_behavior/*.json subdir + logs the overlap decision FIRST. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…m/marker shift modes + Phase-0 overlap issue657_extract.sh: per-behavior recipe routing (the pivot) — - refusal -> Arditi diff-mean extractor (NOT the #623 trait recipe) - em -> #521 on_policy_em shift re-extraction (config-pinned, fail-loud) - marker -> #623 affordance recipe; K2-fail -> #521 on_policy_marker shift fallback + marker_direction_kind=shift_fallback sentinel (M-Alts3) - --harmful/--harmless refusal-bank overrides; smoke flags thread through. issue657_extract_diffmean_direction.py: generalized the trained-shift path to a per-behavior adapter registry (em -> issue_519/em_seed42 dropout 0.05; marker -> issue_519/marker_seed42 dropout 0.0), GPU-aware (CPU smoke fallback), per-behavior K2 eval prompt (misalignment / marker-emission); marker fallback writes the shift_fallback sentinel. issue657_join_substrate.py: emits the marker panel-overlap as the FIRST Phase-0 deliverable + logs the primary-set-membership decision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ractor + adapter-config pin Analysis-lib tests: effective_primary_h3 (affordance primary / shift_fallback + overlap-exclusion demote / EM never primary), _h3_summary reads the per-behavior effective-primary flag (K2-fail excluded not nulled; secondary never gates primary), marker_panel_overlap fail-soft. Updated two band-verdict fixtures to the v6 contract (per-behavior is_primary_h3 flag). Extract-smoke tests: the NEW diff-mean extractor imports + exposes both shift modes with distinct adapter pins; CLI rejects unknown behavior; assert_shift_adapter_config fails loud on a turner-like (r=32/alpha=256) mismatch + missing config (the §4 pin); load_instruction_bank fails loud on an empty bank. The live Claude artifact test is now marker-only (refusal/em use the diff-mean extractor, not generate_artifacts). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-bank swap
The v6 EM-shift smoke crashed when the adapter-applied (PEFT-wrapped) model
went through the same activation-hook path as the base model:
`PeftModel.model.layers` -> AttributeError ('Qwen2ForCausalLM' has no
attribute 'layers') because the LoraModel nests one extra .base_model.model
level. Add `_decoder_layers(model)` that walks the .model/.base_model chain
until an object exposes .layers, so the hook resolves identically for the base
read and the adapter-applied read. Route both capture paths
(capture_post_instruction_activations + capture_mean_response_activations)
through it; add a CPU regression test that PEFT-wraps a tiny model and asserts
the same decoder-layer count resolves.
Also swap the default --harmful bank from walledai/AdvBench (gated for this
token) to mlabonne/harmful_behaviors (ungated mirror, same 416 AdvBench
behaviors in a `text` column); thread it through scripts/issue657_extract.sh.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…for K2-failed behaviors) run_behavior gated leakage computation only on marker_leakage_excluded, never on k2_pass, so a K2-failed direction still wrote raw_rho/partial-rho/CIs/reliability- band into the per-behavior dict and alignment_predictor.json — the "reported as a finding" plan §6.5/§7 forbids (gate-exclusion at _h3_summary != bake-off exclusion). Now: k2_pass is False -> leakage in_scope=False, k2_failed status recorded, no leakage stats computed/persisted. Regression: run_behavior fixture (k2_pass=False) carries no rho keys + in_scope=False; positive control (k2_pass=True) still does. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…in/eval slices) load_instruction_bank truncated the bank to n_harmful FIRST, then _build_refusal_eval_set took harmful[-k:] of that SAME truncated slice as the K2 eval set -> the K2 probes were a strict SUBSET of the diff-mean extraction prompts (an in-sample sanity gate, weaker than the registered held-out causal check, plan §4). Now: load the full bank, carve DISJOINT train (first n_train) + held-out eval (next n_eval_half, after the train slice) via new split_train_eval_bank, with a runtime set(train).isdisjoint(eval) assert and fail-loud on a too-small / duplicate-row bank. metadata records the held-out count + disjointness. Regression: split_train_eval_bank disjoint slices, fail-loud on small/dup banks, both real local banks; CPU --smoke --skip-steering runs exit 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ow/R_high semantics) The lib assigned R_low=n_high (highest reliability) and R_high=n_low (lowest reliability) — the label NAMES were inverted relative to their reliability VALUES. Under the real monotone EIV estimator (lower reliability -> more disattenuation -> lower rho), the genuinely-indeterminate case (clears 0 only at the optimistic / least-attenuated end, fails at the conservative end) collapsed to NULL instead of the plan-registered INDETERMINATE carve-out (§7 line 270/314), corrupting what the production run writes to alignment_predictor.json. Now: R_low=n_low (lowest reliability, most attenuated, conservative), R_high=n_high (highest reliability, least attenuated, optimistic). _band_clears already keys opt_clear on R_high, which is now correct; added a clarifying comment. EIV regression test: value-level rho_low < rho_obs <= rho_high preserved (now more robust), reliability ordering flipped to ascending R_low < R_expected < R_high per the corrected labels. _band_clears INDETERMINATE test rewritten with a MONOTONE fixture (R_high clears, conservative ends straddle 0) + a NULL companion (optimistic end also straddles). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…n non-K2 crash) The marker block captured MARKER_RC but never read it; the affordance-vs-fallback decision keyed SOLELY on whether the effect file existed with a truthy k2_pass. So any NON-K2 primary crash (Claude/API refusal, OOM, ImportError) that left a missing/malformed effect file silently routed to the #521 trained-shift fallback + M-Alts3 demotion — indistinguishable from a real K2 failure, corrupting primary-H3-set membership on an infra crash (violates "Fail fast — never hide failures"). Now: a new marker_fallback_decision() helper requires BOTH rc==0 AND a valid effect JSON with an explicit BOOLEAN k2_pass before deciding; a crash (rc!=0) or a missing/malformed/non-bool effect file returns non-zero -> the dispatcher fails loud and NEVER invokes the fallback. The fallback fires only for a real K2 failure (rc=0 + k2_pass==false). Added MARKER_PRIMARY_CMD injection + --test-marker-decision hook so the decision is GPU-free testable. Also fixed the stale dispatcher header (was: all 3 behaviors use the #623 recipe; only marker does). Regression: 5 shell tests (crash/missing/malformed -> fail loud no fallback; k2_pass false -> fallback; true -> affordance). Dispatcher dry-run + live marker artifact test exit 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…r gap) Pulled 6 K2-verdict eval JSONs from pod-657, byte-identical (md5-verified). marker.k2_pass=false drives the off-pod bake-off's M-Alts3 marker demotion (shift_fallback route); refusal+em k2_pass=true. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…erence docs/methodology/issue_657.md describes how the training-free alignment-predictor experiment was run: behavior-appropriate direction extraction (reused sycophancy, Arditi refusal diff-mean, #521 EM/marker trained-shifts), the K2 steering-sanity gate, the doubly-partialled LOPO bake-off DV, the complete hyperparameter table, verbatim worked examples, and the artifacts index. No interpretation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…5l phase=done after 181s. [task.py]
…itique-codex v1 --> ## Clean-Result [task.py]
…ubagent=analyzer [task.py]
[task.py]
…ubagent=clean-res [task.py]
… REVISE items confirmed addres [task.py]
… effort=high write=True pol [task.py]
…86 phase=done after 301s. [task.py]
…itique-codex v2 --> ## Clean-Result [task.py]
## Reconciler Verdict — RE [task.py]
…ubagent=analyzer [task.py]
…layer-sweep, LOBO pooled) Blocker 1 (Lens 3): strip opaque codes / verdict-caps / pre-registered / bare issue IDs from rendered figure pixels (DV-(a)/H1/H3, NULL/INDETERMINATE, FAILS, #623, K2 statistic) across all four existing figures. Blocker 2 (Lens 9): add fig_lobo_pooled for the pooled higher-N beat-the-prior read. Blocker 3 (Lens 9): add fig_layer_sweep for the per-layer refusal sign-flip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
[task.py]
…ubagent=clean-res [task.py]
…ciler blockers verified closed [task.py]
… effort=high write=True pol [task.py]
…bw phase=done after 392s. [task.py]
…itique-codex v3 --> ## Clean-Result [task.py]
## Reconciler Verdict — RE [task.py]
[task.py]
…nciler v3 recomme [task.py]
[task.py]
…issue_657.md commit=a03779a3f4efb6 [task.py]
[task.py]
[task.py]
…inal reviewer step; abs [task.py]
…g branch (round-3 relabeled) figures
superkaiba
marked this pull request as ready for review
June 19, 2026 04:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes task #657. Training-free reuse-only analysis testing whether
cosine(persona vector, behavior direction)predicts cross-persona leakage landing, AND beats the base behavioral prior — the bar #518/#545/#605/#603/#649 found unbeatable. v3 plan: bounded-attenuation reliability band onbystander_base_rate+prior_centroid_projectiongeometry-side control + EM excluded from primary H3 gate (primary_h3_behaviors = {sycophancy, refusal, marker}). Code-review pending.