[JasonTheAdams]

Give the harness and leaderboard one public version for the complete evaluation. Dataset components become stable wp-core and wp-cli IDs, without independent versions.

How versions work

Change	What to publish
Any dataset edit, including an assertion correction	Next benchmark version: v2, v3, …
Runtime, grading, scoring, or evaluation protocol change	Next benchmark version
Add models or repeat the same evaluation	New immutable results snapshot under the same version
Documentation or website presentation change	Same benchmark version

Visitors see WP-Bench v1 and an updated date. Earlier versions keep their own scores; results from different versions never share a leaderboard.

Review focus

• Frozen manifests cover every task, evaluation inputs, protocol, and category mapping. CI protects published manifests from edits or deletion.
• benchmark.version: v1 pins a run and rejects mismatches before model calls. The exporter requires complete baseline runs and independent repeats; the website adapter consumes that validated data.
• Initial v1 includes the audited assertion fixes used by the public site. Its scores remain unchanged. Defaults now use local wp-core.

Validation

• Ruff and mypy pass; 324 tests pass, 71 skipped.
• Corrected core and all 27 WP-CLI reference/exploit checks pass in WordPress.
• Offline conversion of all 15,834 answers reproduces every overall and category pass rate exactly.

After merge: tag benchmark-v1, distribute the pinned runtime image, and republish the Hub export with stable component IDs before using it. See the release workflow for freezing future versions and publishing results.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: JasonTheAdams <jason_the_adams@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

---

Props Bot commands

You can trigger Props Bot actions by commenting on this PR:

• @propsbot add <username> to give props to a GitHub user. Requires write access.
• @propsbot remove <username> to leave a user out of the props. Requires write access.
• @propsbot refresh to refresh this list. Adding the props-bot label works too.