Phase: P2
Owner(s): Akash
Goal: Token-level CoT comparison baseline so paper claims are fair.
Where: marc/eval/baselines/cot_baseline.py, marc/eval/baselines/perturb.py.
Parallel track: Day 1 of P2. Use same test split JSON as Sparsh — no need to wait for Stage-B model for baseline runs.
-
Implement CoT baseline wrapper (API or local model) matching same problem set.
-
Implement constant perturbation: change coefficients in G, re-run solver, log success rate drop.
-
Export results to results/p2_main/cot_baseline.json for Sparsh to merge into figures.
-
Document model + prompt used in marc/eval/baselines/README.md.
Done when: CoT baseline JSON exists and Sparsh has merged it into summary table.
Push to: PR akash/p2-baseline → main.
Auto-created from the MARC task board (index.html). Update the board if scope changes.
Phase: P2
Owner(s): Akash
Goal: Token-level CoT comparison baseline so paper claims are fair.
Where:
marc/eval/baselines/cot_baseline.py,marc/eval/baselines/perturb.py.Implement CoT baseline wrapper (API or local model) matching same problem set.
Implement constant perturbation: change coefficients in G, re-run solver, log success rate drop.
Export results to
results/p2_main/cot_baseline.jsonfor Sparsh to merge into figures.Document model + prompt used in
marc/eval/baselines/README.md.Auto-created from the MARC task board (
index.html). Update the board if scope changes.