Implement V-SQLM framework with all core components

[mihir-s-05]

This commit implements the complete Vote-First Self-Questioning Language Model (V-SQLM) framework with the following components:

Core Features:

• Equivalence-aware majority voting with domain-specific canonicalization
• Adaptive-K sequential stopping using Wilson confidence bounds
• Verify-first short-circuiting for immediate verification
• Entropy-gated sparse debate (2 debaters, ≤2 rounds, judge)
• Proposer curriculum shaping based on peaked difficulty

Modules Implemented:

• sqlm/vsqlm/: Core V-SQLM components

  • sampler.py: Backend abstraction (Dummy, Transformers, OpenAI)
  • canonicalize.py: Domain-aware equivalence grouping
  • aggregator.py: Majority and weighted voting
  • stopping.py: Wilson bound sequential stopping
  • solver.py: End-to-end V-SQLM pipeline
  • verifiers/: Domain verifiers (math, code, text)

• sqlm/debate/: Sparse debate system

  • minidebate.py: Debate orchestration
  • prompts.py: Role-specific prompts

• sqlm/curriculum/: Proposer training

  • proposer_reward.py: Peaked difficulty reward

• sqlm/metrics/: Logging and calibration

  • calibration.py: Empirical Bayes reliability weighting
  • logging.py: JSONL logging and metrics aggregation

• sqlm/stats/: Statistical utilities

  • entropy.py: Shannon entropy and KL/JS divergence
  • beta_binomial.py: Beta-Binomial estimation

Testing:

• Unit tests for all core modules (stopping, canonicalize, aggregator, curriculum, entropy)
• Integration smoke tests for end-to-end pipeline
• All tests pass with core functionality verified

Documentation:

• docs/VSQLM.md: Comprehensive usage guide
• docs/DERIVATIONS.md: Mathematical derivations and theory
• Experiment configs for math, code, and ablation studies
• CLI runner script with YAML configuration

Dependencies:

• requirements_vsqlm.txt: Core dependencies (numpy, scipy, pyyaml)
• Optional: transformers, torch, openai for model backends

Verified:

• Smoke tests pass for all core components
• Wilson stopping, canonicalization, voting, entropy, curriculum all functioning correctly

Checklist Before Starting

• Searched for similar PR(s).
• Checked PR Title format
  • In format of: [modules] type: Title
  • modules are in fsdp, megatron, sglang, vllm, rollout, trainer, tests, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt
  • type is in feat, fix, doc, refactor, chore
  • can involve multiple modules, seperated by , or space, like [megatron, fsdp] feat: xxx

What does this PR do?

Add one-line overview of what this PR aims to achieve or accomplish.

High-Level Design

Demonstrate the high-level design if this PR is complex.

Specific Changes

List the specific changes.

API

Demonstrate how the API changes if any.

Usage Example

Provide usage example(s) for easier usage.

# Add code snippet or script demonstrating how to use this 

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluatuion results, etc.

Additional Info.

• Issue Number: Fixes issue # or discussion # if any.
• Training: [Note which backend this PR will affect: FSDP, Megatron, both, or none]
• Inference: [Note which backend this PR will affect: vLLM, SGLang, both, or none]

Checklist Before Submitting

• Read the Contribute Guide.
• Apply pre-commit checks.
• Add [BREAKING] to the PR title if it breaks any API.
• Update the documentation about your changes in the docs.
• New CI unit test(s) are added to cover the code path.
• Rely on existing unit tests on CI that covers the code path.