Parent
Grows out of a review comment on #49 (Codex MCP support). The receipt is backend-agnostic, so it lives as its own issue rather than inside the Codex slice. Per-backend population is tracked as a criterion in the individual backend issues (#49/#50/#51), not here — see "Scope" below.
What to build
When the harness materializes a mcp: block into a backend's actual MCP configuration, emit a redacted receipt into the run's results JSON / transcript metadata. The point is to turn a green "the eval passed" into a concrete, inspectable artifact: the harness wrote this isolated MCP config, and the transcript proves the tool was actually reachable.
This issue does two things:
- Defines the backend-agnostic
mcp_receipt structure once, as part of the results JSON schema.
- Provides a shared, backend-agnostic receipt builder — schema, serialization, and the redaction guarantee in one place — that every backend calls with its already-translated config. Backends feed it their materialized server config; they do not each reinvent the receipt shape or re-implement redaction.
Populate the receipt on the Claude Code stdio path first (from #47), as the reference implementation every other backend mirrors.
Fields a reviewer wants:
backend — e.g. claude-code, codex
config_path — the isolated config file that was written (e.g. per-attempt ~/.codex/config.toml)
servers — per server: name, transport, command/args shape, redacted env keys (key names only, never values), and any unsupported/dropped fields
source — spec path + task/run id
tool_evidence — observed mcp__server__tool names and/or a transcript selector proving reachability
Hard requirement: no secret values ever appear in the receipt — only env key names. This mirrors the existing host-env-var interpolation guarantee (secrets never land in the committed spec). Centralizing redaction in the shared builder is what makes this guarantee hold uniformly across backends.
The servers[].unsupported fields also make it easy to document remote/HTTP limitations (#48) without tangling them into the stdio path.
Scope
Acceptance criteria
Blocked by
Parent
Grows out of a review comment on #49 (Codex MCP support). The receipt is backend-agnostic, so it lives as its own issue rather than inside the Codex slice. Per-backend population is tracked as a criterion in the individual backend issues (#49/#50/#51), not here — see "Scope" below.
What to build
When the harness materializes a
mcp:block into a backend's actual MCP configuration, emit a redacted receipt into the run's results JSON / transcript metadata. The point is to turn a green "the eval passed" into a concrete, inspectable artifact: the harness wrote this isolated MCP config, and the transcript proves the tool was actually reachable.This issue does two things:
mcp_receiptstructure once, as part of the results JSON schema.Populate the receipt on the Claude Code stdio path first (from #47), as the reference implementation every other backend mirrors.
Fields a reviewer wants:
backend— e.g.claude-code,codexconfig_path— the isolated config file that was written (e.g. per-attempt~/.codex/config.toml)servers— per server: name, transport, command/args shape, redacted env keys (key names only, never values), and any unsupported/dropped fieldssource— spec path + task/run idtool_evidence— observedmcp__server__toolnames and/or a transcript selector proving reachabilityHard requirement: no secret values ever appear in the receipt — only env key names. This mirrors the existing host-env-var interpolation guarantee (secrets never land in the committed spec). Centralizing redaction in the shared builder is what makes this guarantee hold uniformly across backends.
The
servers[].unsupportedfields also make it easy to document remote/HTTP limitations (#48) without tangling them into the stdio path.Scope
mcp_receiptschema + builder, and populating it on the Claude Code path.Acceptance criteria
mcp_receiptstructure is defined in the results JSON schema (RunMeta/AttemptRecordas appropriate)backend,config_path,servers,source,tool_evidencetool_evidencereflects the actualmcp__server__toolcalls observed in the transcriptREADME.md,skills/evaluate-skill/REFERENCE.md,skills/grill-skill/REFERENCE.md(results JSON schema section)ruff format .andruff check .passBlocked by