feat: Phase 45 — Test-Time Compute Scaling (v4.0.0-alpha.5)

[aarambh-darshan]

Summary

Phase 45 adds a genuinely new inference-time capability, distinct from the
thinking engine (v1 §7): instead of controlling how many tokens a single
generation spends reasoning, this phase generates multiple candidate
completions and selects among them — the Best-of-N / self-consistency /
verifier-guided-selection pattern that sits alongside, not inside, the
existing thinking-mode budget system. The two compose freely: each of the N
candidates can itself use any thinking mode.

Bumps the workspace version to 4.0.0-alpha.5 and tags the milestone
git tag v4.0.0-alpha.5 after merge.

---

What's new

aarambh-studio-inference — 3 new modules

Module	Public surface
best_of_n.rs	SelectionStrategy enum (Verifier | SelfConsistency | Majority | ProcessReward), CompletionVerifier trait (local, so the inference crate does not depend on the finetune crate), BestOfNConfig, BestOfNEngine, BestOfNOutput, SelectionRationale
self_consistency.rs	extract_final_number (byte-identical re-declaration of aarambh_studio_finetune::extract_final_number, attributed — no cross-crate dependency), extract_final_answer, majority_vote (first-occurrence tie-breaking), self_consistency_select
process_reward.rs	ProcessRewardScorer trait, HeuristicProcessRewardScorer (transparent structural scorer), ProcessRewardHead (placeholder returning AarambhError::Unsupported, not todo!()), load_process_reward_head

BestOfNEngine wraps an InferenceEngine and reuses the existing
prepare_session + fork_with_config + decode_sessions path so the
prompt KV-cache is prefilled once and the N forks are decoded together
in one batched target forward pass. Candidate 0 inherits the input sampler's
seed unchanged (N=1 reproduces single-sample generation byte-for-byte —
the backward-compat floor); candidates 1..N are re-seeded base_seed + i.

The wrapper-struct approach (mirrors MtpSpeculativeEngine) leaves
GenerationConfig and the serve crate untouched — best-of-N is a CLI/eval
surface only, per the roadmap's explicit scope.

aarambh-studio-eval — eval-harness integration

• best_of_n_generate / sample_generate / BestOfNOptions / BestOfNResult / VerifierFn in generation.rs
• EvalConfig gains best_of_n: Option<usize>, best_of_n_selection: SelectionStrategy, best_of_n_seed: u64
• gsm8k_subset and humaneval_lite tasks compute both single-sample and best-of-N accuracy when configured, recording single_sample_accuracy, best_of_n_accuracy, and best_of_n_delta in TaskScore::details — the scorecard is the source of truth for whether best-of-N actually helped, never asserted in prose.

aarambh-studio CLI — new flags

• infer --best-of-n <N> --selection verifier|self-consistency|majority|process-reward [--ground-truth <answer>]
• eval --best-of-n <N> --best-of-n-selection <strategy> --best-of-n-seed <u64>

Best-of-N is text-only in Phase 45: combining --best-of-n with
--image/--video/--document/--audio/--tools returns
AarambhError::Unsupported (mirrors fork_with_config's existing no-tools
constraint).

---

Tests

The 4 roadmap-named acceptance tests + 9 supporting tests, all CPU, no cuda:

Test	Gate
best_of_n_with_n_equal_one_matches_single_sample_generation_exactly	N=1 backward compat (byte-identical to single-sample)
self_consistency_majority_vote_selects_the_most_common_final_answer	self-consistency majority vote on extracted answers
process_reward_score_correlates_positively_with_verifier_score_on_labelled_holdout	heuristic PR scorer correlates with verifier on synthetic labelled data (constructed inline, no external fixture)
best_of_n_accuracy_on_gsm8k_subset_is_measured_not_assumed_to_improve	eval scorecard records the delta (asserts it's reported, not that it improved)
best_of_n_generates_n_distinct_candidates_with_stochastic_sampler	re-seeding produces divergent candidates
best_of_n_greedy_candidates_are_identical	greedy best-of-N is degenerate (documented)
verifier_selection_picks_first_fully_correct_candidate	verifier selection
extract_final_number_matches_gsm8k_marker / extract_final_answer_prefers_number_then_last_line	answer extraction
majority_vote_breaks_ties_by_first_occurrence	tie-breaking determinism
heuristic_scorer_rewards_thinking_block_and_answer_marker	PR heuristic monotonicity
selection_strategy_round_trips_through_display / selection_strategy_parses_kebab_and_snake_aliases	CLI parsing
rejects_zero_candidates / rejects_verifier_strategy_without_verifier	config validation

---

CI status (verified locally before push)

• ✅ cargo fmt --all --check
• ✅ cargo check --workspace --all-targets --locked
• ✅ cargo clippy clean on the 4 modified crates (-D warnings -D clippy::undocumented_unsafe_blocks)
• ✅ RUSTDOCFLAGS="-D warnings -D missing_docs" cargo doc --workspace --no-deps --locked
• ✅ scripts/phase28_release_audit.sh (exactly 20 packages, single shared version 4.0.0-alpha.5, no TODO/FIXME/todo!()/unimplemented!() markers, no tracked *.safetensors/checkpoints/**)
• ✅ find scripts -type f -name '*.sh' | xargs -n1 bash -n
• ✅ cargo build --release -p aarambh-studio --locked → target/release/aarambh-studio --version = aarambh-studio 4.0.0-alpha.5
• ✅ CLI smoke: all 31 --help subcommands pass; infer --help shows --best-of-n/--selection/--ground-truth; eval --help shows --best-of-n/--best-of-n-selection/--best-of-n-seed
• ✅ Tests: inference 75/75, eval 45/45, binary 6/6, selflearn 29/29, train 48/48, + all other crates individually
• ✅ scripts/phase45_smoke.sh end-to-end (acceptance tests + tiny train + infer --best-of-n 2 --selection self-consistency + CLI flag verification → artifacts/phase45_test_time_smoke.json)

---

Files changed

17 modified:

• Cargo.toml, Cargo.lock (version bump → 4.0.0-alpha.5)
• ROADMAP_V4.md (Phase 45 tasks flipped [ ] → [x])
• ARCHITECTURE_V4.md (§59 ### Implementation (Phase 45, v4.0.0-alpha.5) subsection appended)
• SELF_LEARNING_V4.md (§45 status blockquote)
• CHANGELOG.md (new ## [4.0.0-alpha.5] - 2026-08-16 entry prepended)
• README.md (version bump in header prose + bibtex; doc-table row; Current Boundaries note)
• aarambh-studio/src/cmd/infer.rs (InferArgs + --best-of-n/--selection/--ground-truth flags + run_best_of_n_infer + MathVerifierAdapter)
• aarambh-studio/src/cmd/eval.rs (EvalArgs + --best-of-n/--best-of-n-selection/--best-of-n-seed flags)
• aarambh-studio/src/cmd/selflearn.rs, aarambh-studio/src/cmd/train.rs (construct the new EvalConfig/InferArgs fields)
• crates/aarambh-studio-inference/src/lib.rs (3 new module exports)
• crates/aarambh-studio-eval/src/lib.rs, generation.rs, harness.rs, tasks/gsm8k_subset.rs, tasks/humaneval_lite.rs
• crates/aarambh-studio-selflearn/src/forgetting_hook.rs (construct new EvalConfig fields)

7 new:

• crates/aarambh-studio-inference/src/best_of_n.rs
• crates/aarambh-studio-inference/src/self_consistency.rs
• crates/aarambh-studio-inference/src/process_reward.rs
• docs/phase45_test_time.md (mirrors docs/phase44_multi_node.md's 12-section structure)
• configs/best_of_n_smoke.toml (CPU smoke training config)
• scripts/phase45_smoke.sh (smoke script)
• artifacts/phase45_test_time_smoke.json (committed scorecard)

---

Honesty note on scope

• No new crate — the release audit's EXPECTED_PACKAGES=20 is unchanged; Phase 45 lives entirely inside aarambh-studio-inference + aarambh-studio-eval + the CLI binary, per the roadmap.
• No trained process-reward checkpoint ships — the ProcessRewardHead returns AarambhError::Unsupported until a checkpoint exists (explicitly future work, not stubbed with todo!()). The release audit forbids tracked model artifacts.
• serve crate is unchanged — the wrapper-struct approach keeps the server surface clean; GenerationConfig is not extended.
• i3 supports small N (2–4) for text tasks; larger N is Kaggle-scoped for cost reasons, following v1 §12's existing self-learning N-completion budget precedent. Whether best-of-N improves accuracy on a given task is measured by the eval-harness scorecard, not asserted in prose.