Weekly benchmark review: 2026-06-29

[github-actions]

Weekly benchmark review (2026-06-29)

Automated check from scripts/weekly-benchmarks-check.mjs. Triage and either:

• Update data/benchmarks.json if a new flagship model dropped this week, then close this issue, OR
• Comment noop and close if nothing actionable surfaced.

Current state of data/benchmarks.json

• lastUpdated: 2026-06-11 (18 days ago)

• Models tracked: 20

• Benchmarks tracked: 5

• Models released within last 60 days: 2

  • 2026-06 | Anthropic | Claude Fable 5
  • 2026-05 | Anthropic | Claude Opus 4.8

Model-release-flavored news, last 7 days

Matched 13 articles (keyword scan; not all will be real releases).

Date	Source	Title
2026-06-29	Hacker News AI	Vibe Coding to Agentic Engineering: A Three-Phase Workflow with Claude Code
2026-06-29	Hacker News AI	CoreWeave ARIA: AI Research and Iteration Agent
2026-06-29	NVIDIA AI Blog	Open Models, Closed Environments: Palantir Brings Secure AI to US Agencies With NVIDIA Nemotron
2026-06-28	The Verge AI	China’s Z.ai claims it can match Mythos on cybersecurity
2026-06-28	The Verge AI	Suno launches Spark incubator program to feed independent artists to its AI machine
2026-06-28	ZDNet AI	Use Android Auto? How to limit what information Gemini learns about you
2026-06-26	WIRED AI	OpenAI Has New AI Models. Here’s Why You Can’t Use Them
2026-06-26	The Verge AI	OpenAI unveils GPT-5.6 amid US AI regulatory drama
2026-06-26	Hugging Face Blog	Run a vLLM Server on HF Jobs in One Command
2026-06-25	The Verge AI	OpenAI will delay GPT-5.6 after Trump administration request
2026-06-24	Hugging Face Blog	Introducing the FFASR Leaderboard: Benchmarking ASR in the Real World
2026-06-23	NVIDIA AI Blog	How Businesses Are Building Specialized AI They Can Trust
2026-06-23	NVIDIA AI Blog	NVIDIA Powers Over 400 of the World’s 500 Fastest Supercomputers

Sources: The Verge AI (4), NVIDIA AI Blog (3), Hacker News AI (2), Hugging Face Blog (2), ZDNet AI (1), WIRED AI (1)

HF Open LLM Leaderboard top 10

Captured: 2026-06-29

Rank	Model
1	MaziyarPanahi_calme-3.2-instruct-78b_bfloat16 (avg 52.08 · 78B)
2	MaziyarPanahi_calme-3.1-instruct-78b_bfloat16 (avg 51.29 · 78B)
3	dfurman_CalmeRys-78B-Orpo-v0.1_bfloat16 (avg 51.23 · 78B)
4	MaziyarPanahi_calme-2.4-rys-78b_bfloat16 (avg 50.77 · 78B)
5	huihui-ai_Qwen2.5-72B-Instruct-abliterated_bfloat16 (avg 48.11 · 73B)
6	Qwen_Qwen2.5-72B-Instruct_bfloat16 (avg 47.98 · 73B)
7	MaziyarPanahi_calme-2.1-qwen2.5-72b_bfloat16 (avg 47.86 · 73B)
8	newsbang_Homer-v1.0-Qwen2.5-72B_bfloat16 (avg 47.46 · 73B)
9	ehristoforu_qwen2.5-test-32b-it_bfloat16 (avg 47.37 · 33B)
10	Saxo_Linkbricks-Horizon-AI-Avengers-V1-32B_bfloat16 (avg 47.34 · 33B)

---

What "needs update" usually means

1. A flagship from Anthropic / OpenAI / Google / Meta / Mistral / DeepSeek / xAI launched this week → add a row to data/benchmarks.json.
2. A tracked model has materially-shifted benchmark scores (re-running, methodology change) → update the row.
3. A new benchmark itself (e.g. a successor to MMLU-Pro) is becoming canonical → add it.

What it usually does NOT mean

• Research papers about benchmarks (those land on /research, not /benchmarks).
• HN opinion threads about a model.
• Pricing-only changes (those go in data/pricing.json).

Bump lastUpdated in data/benchmarks.json whenever you change anything else in the file.