Task #491: ICL vs FT equivalence (rank-1 model P7 test)#446
Draft
superkaiba wants to merge 21 commits into
Draft
Task #491: ICL vs FT equivalence (rank-1 model P7 test)#446superkaiba wants to merge 21 commits into
superkaiba wants to merge 21 commits into
Conversation
… slot eval, matching, free-gen, activations, stats, dispatcher Plan v3: band-stop LOG-ONLY (log_only=True, eval_every=2, min_steps=0, trajectory path), offline source-cell Q_test matching basis at all 27 pre-prune ckpts, 9-non-source-context H1 machinery (joint q-bootstrap, disattenuation, EIV slope, range null), own-policy substrate reads at K=8, smoke == sweep with one cell through the same dispatcher. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…x, free_gen max_model_len 10240 (K=16 prompt arithmetic) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
Deployment failed with the following error: Learn More: https://vercel.com/thomasjiralerspong-5386s-projects?upgradeToPro=build-rate-limit |
…emble-once), H1 core-pair filter + format-gap control block, post-prune rematch guard, H2 context-label permutation null, recursive eval-JSON upload, H4 joint-question bootstrap Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…tstrap-cis-missing) Reconciler r2 binding fix list: (1) h5_control_contrasts + h6_order_statistics add joint question-level bootstrap CIs (n_boot/seed/ci95) to the registered H5 control contrasts and the H6 order-SD-vs-(K3->K8)-increment read, mirroring the H4 machinery over the persisted icl_panel per-question delta_logp arrays; (2) demo-prefix token counts per H5 control on the live pinned tokenizer (chat-template diff isolates the demo block; stripped vs full differs by exactly K=8 marker tokens), with structured skip records for missing local artifacts. Synthetic smoke extended with known-structure H5/H6 fixtures + token-count stub assert. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… n_boot forwarding to H4/H5/H6 (h6-order-sd-wrong-statistic, h5-h6-bootstrap-nboot-not-forwarded) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…n aggregates, own-policy, trajectories, gate1)
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…ft_panel schema fallback for alignment assert)
…) + dose-curve and hero-scatter figures
…tion, emission asymmetry, H2 geometry)
…; per-panel label offsets) + h2_geometry_layers title (replicate ceilings 0.65-0.99)
…framing, drop banned vocabulary)
superkaiba
added a commit
that referenced
this pull request
Jun 11, 2026
Branch was 1793 commits behind main (merge-base on mainline), unsafe to blind-rebase. Cherry-picked this task's own added files only (eval_results/ issue_491, figures/issue_491, docs/methodology/issue_491.md); shared src/ stays on the open PR #446 for a later full merge.
…ell — 14th run spec + fetch-helpful-demos + --phase ft_content_followup + analyze_followup.py The missing weight-route content control (plan v4): ft_ctrl_helpful_content trains the SAME 8 chain-A questions with the parent's FROZEN helpful-voiced responses + trailing marker under the villain wrapper. data_build gains the 14th spec, a row_content branch, and the fail-loud fetch-helpful-demos HF download (pinned rev a0ca913b); dispatch gains --skip-own-policy plumbing and the ft_content_followup phase (the sweep with one cell, PASS_UNIFIED, with sentinel + [phase=done]); analyze_followup.py implements the plan §3b verdict table mechanically (primary profile rho + magnitude envelope + 28-layer gate read + equal-step + recomputed reference ceilings + secondaries) with a villain-replicate stub smoke that reproduces every registered ceiling to 1e-3. analyze.py registers the new run as a control so the parent H1 read can never pool it; activations keeps the 28-variant registry (controls have no activations phase). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…d pair, full-step slot reads, trajectory, free-gen aggregate)
… jointly — weight route content-blind on row content; wrapper carries the map)
…apper profile comparison)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes task #491. 13 LoRA runs (K in {1,3,8,16} x 3 chains) vs ICL-at-inference on the same K examples; matched-strength leakage-profile comparison.