Skip to content

Adopt agent-034/030 judge conversions; rerun claude-fable-5 at the new pin #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gaojude merged 7 commits into from
Jul 9, 2026
Merged

Adopt agent-034/030 judge conversions; rerun claude-fable-5 at the new pin #93

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
9fac5e3
Adopt agent-034/030 judge conversions; rerun claude-fable-5 at the ne…
gaojude Jul 7, 2026
42a1bcb
Add GPT 5.6 (xhigh) eval results; normalize export over the canonical…
gaojude Jul 8, 2026
5863cab
Add GPT 5.6 Sol (xhigh) eval results
gaojude Jul 8, 2026
74f8c73
Make Claude Fable 5's effort level explicit (high)
gaojude Jul 8, 2026
d2e8a6f
Add GPT 5.6 Sol (xhigh) evals; make Fable 5's effort explicit (high) …
gaojude Jul 8, 2026
41fd691
Add Grok 4.5 eval results (#96)
gaojude Jul 8, 2026
53e8177
Merge grok-4.5 results (#96 landed on the merged #95 branch)
gaojude Jul 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
11 changes: 5 additions & 6 deletions .github/workflows/eval-cache-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,11 @@ jobs:

- run: pnpm install --frozen-lockfile

# Pinned to a known-good next.js commit. To adopt new/changed evals, bump this
# SHA in a PR that runs them for the experiments you're refreshing and
# `agent-eval keep`s the rest. The check below passes as long as every
# new/changed eval is either run or kept — so models may sit on an older eval
# version intentionally without going red.
- run: pnpm sync-evals 9e4d718de41bc2bceb687c73c2a60fdf6c5c4440
# Pinned to a known-good next.js canary commit. To adopt new/changed evals,
# bump this SHA in a PR that reruns them for the experiments you're
# refreshing and lists the remaining (experiment, eval) pairs in
# check-stale.mjs's ACCEPTED_STALE.
- run: pnpm sync-evals 0240fd39a5351a4a76c73a81058379a41e26c9ff

# The "which staleness is acceptable" policy lives in this repo, not the
# framework: check-stale.mjs reads `agent-eval status --json` and fails on
Expand Down
Loading
Loading