Skip to content

[FR-008] Maintainer can reproduce single eval from manifest (make reproduce) #8

Description

@explosivebit

Trace: PRD prd-v0-1-smoke-evaluation-run · FR-008 · ADR-002 (evaluator-only reproduce) · SC-2 · SC-6

Capability: make reproduce HASH=<run_hash> EVAL_ID=<eval_id> loads cached raw_output from artifact_refs, re-runs evaluator (NO LLM call), diffs evaluator_json on deterministic fields.

Acceptance:

Implementation locus: apps/eval-core-py/src/orchestrator/reproduce.py + infra/scripts/reproduce-local-run.sh

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/adrArchitecture Decision Recordphase/2-smokePhase 2 — Smoke run executionpriority/p0Critical — blocker for current Phasetype/featNew feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions