feat: private eval suite — AS215932 domain judgment as token capital

[Svaag]

Context

Phase B of the agentic-ISP plan: build the private, offline token-capital baseline so AS215932 domain judgment is captured as code that survives provider/model swaps and gates CI. (Phase A landed the live loop; tracker network-operations#225.)

What's here

• evals/cases/<family>/*.json — 15 cases across 5 families (3 each):
  • domain-policy — servify.network(infra)/hyrule.host(product)/as215932.net(AS-routing) identities not conflated or repurposed
  • promotion-safety — pins via promote-apps+apply.yml; no manual pin edits / auto-merge / auto prod apply
  • noc-evidence — remediation needs evidence + rollback guard + approval; no real mutation in the no-op phase
  • vps-launch-proof — stay in the narrow contract; no generic payment-intent engine
  • network-change — FRR/firewall/BGP need emulated-lab verification + human review
• evals/schema.json — case schema.
• src/hyrule_engineering_loop/evals.py — pydantic case models, a deterministic per-family rule engine producing (decision, rationale), a loader (dedupe + schema-version guard), runner, and JSON summary. No model, no network.
• CLI — hyrule-engineering-loop evals run [--strict] [--json] (exit 1 on failure under --strict).
• CI — new evals job; failures block the PR.
• docs/engineering-loop/private-evals.md.

The rules are the baseline "company veteran"; the loop's LLM judgment can later be graded against the same corpus, but the deterministic rules keep CI model-free.

Trace / cost note

Trace harvesting already exists (trace.py writes loop_trace.json; the daemon report carries issue/outcome/cost_usd/pr_url/journal_path). The one gap is cost — cost_usd reads 0 because PiBackend runs text mode while the parser expects JSON; tracked in #6, not in scope here.

Validation

• uvx ruff check src tests — clean
• uv run --group dev mypy --strict src — clean
• uv run --group dev pytest -q — 172 passed (+11)
• uv run --group dev hyrule-engineering-loop evals run --strict — 15/15

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.