Publish the eval observatory and preserve previous editions - #67

Merged

alanshurafa merged 14 commits into

codex/eval-observatory

Sep 5, 2026

Commits on Sep 4, 2026

Price every seat from its own token log at the tracked list rate

agent57zero
and
claude
committed
4d8cfe8
Key every cell by (task, condition, seed) and add --repeat K

agent57zero
and
claude
committed
3048f8b
Carry SWE-bench Verified difficulty labels in every subset

agent57zero
and
claude
committed
a74b6a1
Add the statistics module: intervals, paired contrasts, bootstrap, Rank(UB)

agent57zero
and
claude
committed
e55dfdb
Score a no-patch arm as zero instead of aborting the cell

agent57zero
and
claude
committed
f2adc63
Record whether the repair stage changed the patch, by content hash

agent57zero
and
claude
committed
3aceb8c
Track the Codex sandbox probe and record CLI versions per cell

agent57zero
and
claude
committed
ac69d76
Add the best-of-k selector: the repository's own tests choose, no reviewer

agent57zero
and
claude
committed
8c949be
Declare conditions J through P so Phase 1 needs no code changes

agent57zero
and
claude
committed
925242b
Redesign the results site around configurations with uncertainty (schema 2.0)

agent57zero
and
claude
committed
d8889c0
WIP: phase runner, status reader, orchestration protocol, Claude adapter

agent57zero
and
claude
committed
8888cb0

Commits on Sep 5, 2026

Make current eval scores clear in a public observatory

alanshurafa
and
codex
committed
1bb76ec
Preserve the live edition and prepare the site release

alanshurafa
and
codex
committed
f45c736
Avoid broken-pipe races in help-output checks

alanshurafa
and
codex
committed
ead40bd