Task #603: P3' write decomposition pipeline#445
Draft
superkaiba wants to merge 24 commits into
Draft
Conversation
…ture + frozen family inputs - Port activation_shift.py + svd_direction_constancy.py (origin/issue-551), issue541_personas.py + issue444_persona_distance_topic.py + personas.py courthouse_architecture_historian entry (origin/issue-541), i518/i509 condition registries (origin/issue-518). None were on main; the personas.py + issue444 ports reconcile the _resolve_persona_prompt / registry drift that main's issue541_geometry_extract.py depends on. - activation_shift.py: additive response_sink kwarg persists per-(persona, question) generated response ids + decoded texts (guard B); numerics untouched. - scripts/issue603_inputs.py: frozen per-family panels/probes/cells JSONs (24 personas, 20 probes, 21 cells).
…s (PASS) decompose_write(): signed-cosine CMF primary, norm contrast, shared/residual norms, SVD + unit-norm-SVD + leave-one-bystander-out jackknife variants; split_half_reliability() per #551 recipe. V1 gate recomputes the published mean_resp/norm_alignment statistics from the 18 private-repo tensors with this branch's code — PASS, 0 deviations > 1e-3 (v1_gate.json).
…ase-2 decomposition/strata/figures - issue603_extract_dispatch.py: smoke==sweep (--cells/--personas/--questions thread through EVERY phase), CUDA_VISIBLE_DEVICES round-robin over 21 cells, checkpoint-per-cell + immediate upload (403-quota fallback chain data->private->overflow, list_repo_files verify), poll_pipeline-conformant results sentinel + single terminal [phase=done]; never shells task.py. - issue603_extract_worker.py: one cell per GPU; list_repo_files + per-file hf_hub_download adapter resolve (snapshot_download siblings-truncation trap), adapter base-model assert, response_sink sidecar persistence (guard B Must-Fix). - issue603_source_prior.py: #518 source-self length-norm teacher-forced log P priors (verbatim #444/#541 vLLM recipe), checkpoint per source. - issue603_decompose.py: per-cell CMF/norm decomposition + plan §6 stats (exact seed-ordering, 720-perm Spearman, teacher-level joint contrast, guard A disattenuation both-must-hold, norm-floor sensitivity, Stouffer pooling, decision lattice). - issue603_expression_strata.py: guard B judge labeling (ported #541 5-way fact judge + new binary refusal/EM judges) + within-cell stratified CMF + rank-partial cross-cell conditioning. - issue603_figures.py: hero contrast grid + 9 exploratory panels (paper_plots conventions). - activation_shift.py: None system prompt = omit the system turn (the #444/#541 no_system convention; non-None prompts byte-identical). - Port reanalyze_issue444_5way.py (issue-541 shared-client EMFILE fix) + aggregate_issue500.py (judge + stated_seven labeler).
|
Deployment failed with the following error: Learn More: https://vercel.com/thomasjiralerspong-5386s-projects?upgradeToPro=build-rate-limit |
…ng, gated pooling - extract_dispatch: resume skip now validates the cell manifest (source, persona set, question count, layers, probe SHA, max_new_tokens) against the CURRENT invocation; a smoke artifact under production filenames is recomputed, never reused (blocker smoke-artifact-poisons-full-sweep). Resume check runs in dry-run too (CPU resume simulation); failure path terminates still-running GPU workers. - decompose: _pull_from_hub also stages source_priors.json so the off-pod Phase-2 read runs from HF artifacts after pod termination (blocker from-hub-source-priors-not-downloaded); pooled Stouffer now conditions on the plan-§6 family-diagnostic gate (refusal variance gate / <4 norm-floor survivors -> collapse to the surviving family alone); both-must-hold accepts a disattenuated ordering at least as strong. - expression_strata: cached judge labels reused only when the recorded with_bystanders mode matches (mismatch -> recompute, logged); new --from-hub reuses the decompose pull. - write_decomposition: _top_left_singular -> _top_right_singular (the function returns Vh[0], the H-space direction). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… strata fail-loud priors
…e implant gate (rates 0.80-1.00, zero drops)
…ty) + staleness-validated checkpoint reuse; re-judged production artifact Concern refusal-implant-judge-temperature-mismatch: #518's judge paths both set temperature=0.0 explicitly; the step-12 Batch requests ran at the API default. Pin JUDGE_TEMPERATURE=0.0 on every request, record it in the output meta + per-source checkpoints + _batches.json, and validate it on every checkpoint/batch-state reuse (mismatch -> recompute, never silent). Full 6x500 production re-judge at temp 0.0: rates unchanged (5x1.000, qwen_default 0.800), dropped_sources=[] — gate verdicts identical. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…_artifact_channel_live, guard_b survives_conditioning), expression strata + labels, figures
…/B + validity panels) + source_priors.json
…) + fix 3x2 title overlap/ylabel clipping
…, both-families villain leverage, no-support framing)
…judge (guard B)
The first guard-B run routed refusal/EM judge calls through
reanalyze_issue444_5way._judge_one, whose 5-way output_category_5way
validator rewrote every binary {"expressed": bool} verdict to null —
all 5,760 refusal/EM expression labels were vacuous and the
survives_conditioning verdicts tautological (constant 0.0 rates).
- _parse_binary_verdict: binary families get their own parser; fact
keeps the unchanged synchronous 5-way path.
- Messages Batch API pre-pass (one batch/cell, checkpoint-per-cell,
crash-resumable {cell}.batch.json sidecar) re-judged all 12 cells:
5,760/5,760 labeled, 0 null.
- Stale-cache discipline: labels_schema_version=2 + _cache_labels_valid
(mode match, binary schema version, non-all-null) — the 12 defective
caches deterministically re-judge; fact caches grandfathered+reused.
- expression_rate over labeled rows only (None when none) — an all-null
cell can no longer enter the conditioning as a vacuous 0.0.
- clean_u_reestimate: exploratory rho(prior, cmf_full_vs_clean_u) per
family; refusal -0.714, em -0.714 — the extension lean survives the
clean-text û rebuild (raw -0.60 / -0.714).
- Re-ran decompose + figures: lattice headline unchanged
(indeterminate_on_h1_artifact_channel_live; guard_a drives it),
refusal/em guard-B verdicts now non-vacuous survives_conditioning.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…oc in expression_strata The EM clean-u rebuild is near-tautological (5/6 source cells at 0/20 expression, bystander exclusions 0-12/460, clean-u == full-u to the 3rd decimal in all 6 cells), but the artifact's doc claimed EM was the family with real stratified resolution. Record per-family caveats (source expression, per-cell n_excluded_questions, max clean-u deviation, degenerate flag) in meta.family_caveats and fix the stale doc string (resolution is in refusal, not EM). Concern em-clean-u-rebuild-degenerate. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…amily_caveats Recompute from cached labels (zero new judging). per_cell, cross_family, guard_b_verdict identical to the prior committed version; only meta.doc changed + meta.family_caveats added (EM clean-u rebuild flagged degenerate with per-cell n_excluded_questions; refusal/fact recorded as genuine). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…xt rho per family + per-cell expressed-bystander counts)
superkaiba
added a commit
that referenced
this pull request
Jun 11, 2026
Branch was 2009 commits behind main (threshold 200), unsafe to blind-rebase. Cherry-picked this task's own added files only (decomposition results, expression strata + labels, figures, methodology doc); shared src/ and scripts/ stay on the open PR #445 for a later full rebase.
…gh the rig (plan v2 diffs 1-4)
- activation_shift.py: widen compute_mr gate to {same, base} (mean_resp already
computed unconditionally; this only stops discarding it; same path untouched)
- issue603_extract_worker.py: --variant flag -> extract_per_context_shifts;
manifest records the REALIZED variant + per-cell truncation rate (ride-along)
- issue603_extract_dispatch.py: --variant threaded to every worker; variant added
to the manifest-validated resume checks (parent same artifacts classify stale
for base cells); upload subdir derives from the out-dir basename (shifts_base);
per-cell truncation-rate surfaced in the main log
- issue603_decompose.py: --expect-variant fail-loud manifest assert; --hub-subdir
/ basename-derived --from-hub prefix; variant + truncation_rate in output JSON
Smokes: V1 gate re-PASS (0 deviations); 21-cell base dry-run exit 0 with sentinel;
resume guard stale-on-same / skip-on-base both exercised; decompose fail-loud +
proceed branches verified; parent same-path numbers byte-match committed results.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ant + namespace base-run defaults Concern strata-label-cache-variant-unbound (Codex v5 Major): label caches now record the manifest variant (schema v3; variant-less caches count as "same"), _cache_labels_valid re-judges on variant mismatch, batch sidecars carry the variant, and --labels-dir/--out defaults derive from the --shifts-dir basename (shifts_base -> expression_labels_base/ + expression_strata_base.json) with a fail-loud guard against pointing a non-same run at the parent locations.
…ells, --expect-variant base), expression labels + strata (text-clean check), base_text figures
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes task #603. Extraction + decomposition pipeline for the P3' common-mode-fraction test (plan v1, critic-ensemble REVISE round integrated).