Skip to content

Milestones

List view

  • Agent Infra Experiments

    Build all preparation infrastructure for the Agent Router (CAR) sweep — default tools, eval harness, runtime, dataset partition, and third-party baseline runner — and gate the full Phase 2 experiment on a Phase 0 confound check that the cheap anchor (Haiku 4.5) actually differentiates between skill subsets. Scope: Phase 0 (confound check) + Phase 1 (ADRs) deliverables from #133. Phase 2 sweep (5,700 runs across A0–A6 + A_g + CAR on 6 models) is tracked in a follow-up milestone. Key deliverables: - Default tool primitives implemented and wired into AgentRunner (#145) - Pure-retrieval BM25 + embedding baseline + heuristic-pipeline vs agent comparison (#146) - Graph skill dimension (A_g + A3 + A4) end-to-end on smoke tests across 5 languages (#147) - Experiment runner with multi-model dispatch, min-of-3 (cost/latency) + mean-of-3 (accuracy) aggregation, prompt/completion token split logging (#148) - CAR runtime: classify(query, repo_meta) + CAR_table lookup wired into AgentRunner (#149) - Third-party agentic baseline runner (Claude Code + Codex), accuracy-only (#150) - 30/70 fitting/held-out partition (per-language repo-split) + per-instance scenario labels frozen in Phase 1 ADR (#151) - Phase 0 confound check: A0 + A6 on Haiku × 30 instances, token_gap_ratio ≥ 0.2 to lock anchor (otherwise escalate to GPT-5.5 / Sonnet) - v2 RFC body update reflecting all Phase 0/1 outcomes Exit gate for Phase 2 milestone: Phase 0 token_gap_ratio result + anchor model lock + all 7 sub-issues merged.

    Overdue by 2 month(s)
    •
    Due by May 31, 2026
    •7/7 issues closed
    100% complete0 open7 closed

  • Graph-Augmented Retrieval & Agent Pipeline

    Wire the code graph into retrieval and agent pipelines, refactor agent infrastructure, and run graph experiments on codeminer_base. Key deliverables: - ExpandContext wired into skill_agent_eval.py - Agent runner refactored (token tracking, skill allowlist, context management) - codeminer_base support in graph_retrieve and skill_agent_eval scripts - E1–E3 experiment results (BM25→graph, BM25→graph→rerank, Agent+graph_expand)

    Overdue by 3 month(s)
    •
    Due by May 15, 2026
    •4/4 issues closed
    100% complete0 open4 closed