Skip to content

chore(ai): refresh the generated model catalog from live provider sources - #2039

Merged
xeophon merged 1 commit into
mainfrom
chore/refresh-model-catalog
Sep 4, 2026
Merged

chore(ai): refresh the generated model catalog from live provider sources#2039
xeophon merged 1 commit into
mainfrom
chore/refresh-model-catalog

Conversation

@snimu

@snimu snimu commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

What

First explicit, reviewed refresh of the generated model catalog (npm run generate-models), per the RES-1269 discussion: catalog regeneration is a deliberate, audited step rather than a CI side effect. This PR stands alone against current main: CI's build regenerates the catalog, and the committed file here is byte-identical (md5 af81f536fceafbdeac473ee4f6f5b0d8) to a fresh regeneration at validation time, so committed == live. If upstream shifts before merge and CI fails, the catalog should be regenerated and this PR updated.

Totals: 1238 -> 1262 models (41 added, 17 removed). No provider dropped wholesale or lost >30% of its rows (the silent-[] fetch-failure signature from #2008 did not occur; generator logs showed all four sources loading: models.dev 564, OpenRouter 294, Vercel AI Gateway 233, Prime Inference 107).

Per-provider changes

provider before after delta
amazon-bedrock 119 122 +3 / -0
anthropic 13 14 +1 / -0
cloudflare-ai-gateway 31 32 +1 / -0
cloudflare-workers-ai 17 18 +1 / -0
fireworks 17 19 +3 / -1
github-copilot 33 27 +2 / -8
google 17 17 +1 / -1
groq 6 7 +1 / -0
huggingface 69 71 +2 / -0
opencode 60 63 +4 / -1
opencode-go 24 27 +3 / -0
openrouter 292 295 +8 / -5
prime-inference 106 107 +1 / -0
vercel-ai-gateway 224 233 +10 / -1

Providers not listed are unchanged.

Added

  • amazon-bedrock: anthropic.claude-fable-5-1, global.anthropic.claude-fable-5-1, us.anthropic.claude-fable-5-1
  • anthropic: claude-fable-5-1
  • cloudflare-ai-gateway: claude-fable-5.1
  • cloudflare-workers-ai: @cf/zai-org/glm-5.3
  • fireworks: accounts/fireworks/models/deepseek-v4-flash-vision-exp, accounts/fireworks/models/glm-5p3, accounts/fireworks/models/glm-5p3-flash
  • github-copilot: claude-fable-5.1, gemini-3.8-flash
  • google: gemini-3.8-flash
  • groq: qwen/qwen3.8-27b
  • huggingface: deepseek-ai/DeepSeek-V4-Flash-Vision-Exp, zai-org/GLM-5.3
  • opencode: claude-fable-5-1, gemini-3.8-flash, ling-3.0-flash-fin-free, muse-spark-1.3-contributor-free
  • opencode-go: hy4-preview, muse-spark-1.3-contributor, omen-alpha
  • openrouter: anthropic/claude-fable-5.1, google/gemini-3.8-flash, ibm-granite/granite-4.2-8b, inception/mercury-2.5-preview, inclusionai/ling-3.0-flash-fin, meta/muse-spark-1.3, meta/muse-spark-1.3-contributor, ~z-ai/glm-flash-latest
  • prime-inference: x-ai/grok-4.6
  • vercel-ai-gateway: alibaba/qwen3.8-flash-next, alibaba/qwen3.8-max-0902, anthropic/claude-fable-5.1, google/gemini-3.8-flash, meta/muse-spark-1.3, meta/muse-spark-1.3-contributor, tencent/hy4-preview, xiaomi/mimo-v2.5-pro-ultraspeed, zai/glm-5.3-fast, zai/glm-5.3-promo-50

Removed

  • fireworks: accounts/fireworks/models/deepseek-v4-flash
  • github-copilot: claude-opus-4.5, claude-opus-4.6, claude-sonnet-4, claude-sonnet-4.5, gemini-3.1-pro-preview, gpt-4.1, gpt-5.2, gpt-5.2-codex
  • google: gemini-robotics-er-1.6-preview
  • opencode: hy3-free
  • openrouter: anthropic/claude-opus-4.7-fast, anthropic/claude-opus-4.8-fast, anthropic/claude-opus-5-fast, arcee-ai/virtuoso-large, kwaipilot/kat-coder-air-v2.5
  • vercel-ai-gateway: deepseek/deepseek-v3

Reference fixes (substitution table)

GitHub Copilot dropped 8 models; two of them were referenced in typed getModel("github-copilot", ...) calls (the exact breakage that motivated RES-1269 — the failing union in CI was Copilot's model list). Substitutions, applied only to (provider, id) pairs that were actually removed:

removed (github-copilot) replacement where
claude-sonnet-4.5 claude-sonnet-4.6 18 sites across 11 ai test files + 1 expect + 1 handoff pair
claude-sonnet-4.5 claude-haiku-4.5 1 site: the interleaved-thinking beta test in github-copilot-anthropic.test.ts; the beta header is only sent for non-adaptive-thinking models and haiku is the only remaining non-adaptive Claude in the Copilot catalog
gpt-5.2-codex gpt-5.3-codex 2 sites in tool-call-id-normalization.test.ts + 1 handoff pair

The same ids under other providers (anthropic claude-sonnet-4-5, openai-codex gpt-5.2-codex, openai gpt-5.2, google gemini-3.1-pro-preview, etc.) still exist and their references are untouched. No test added or removed; counts unchanged.

defaultModelPerProvider check

Every entry resolves in the refreshed catalog except zai: glm-5.1, which is missing from the OLD catalog too — a pre-existing gap, already fixed (with a catalog-existence pin) in open #2032. Not duplicated here.

Data-quality notes (report only, for the upcoming validation work)

  • huggingface / thinkingmachines/Inkling-Small: maxTokens 1048576 > contextWindow 524288 (likely swapped upstream). Pre-existing — identical values in the old catalog. Only violation of maxTokens <= contextWindow in the 1262 rows; no zero or missing contextWindow/maxTokens fields.

Verification (clean checkout, Prime sandbox, source-only sync + npm ci)

  • npm run build (which on current main regenerates the catalog — exact CI mirror): exit 0; regenerated file byte-identical to the committed one.
  • npm run check: exit 0.
  • packages/ai npm test: 50 files passed / 22 skipped, 0 failures (includes the cross-provider-handoff catalog-existence guard).
  • coding-agent model-resolver, model-registry, tree-selector, settings-manager test files: 4/4 passed.

Linear: RES-1269


Note

Medium Risk
Large generated catalog diff can break compile-time model unions and runtime routing for users pinned to removed ids; Copilot catalog shrink is the main behavioral change, mitigated by targeted test substitutions.

Overview
Refreshes the committed models.generated.ts catalog (RES-1269) so it matches a live generate-models run: 1262 models net (+41 / −17), with metadata and pricing updates across Bedrock, Anthropic, Google, Fireworks, OpenRouter, Vercel AI Gateway, opencode, and others.

Notable catalog deltas: adds Claude Fable 5.1, Gemini 3.8 Flash, and GLM-5.3 (and related variants) on multiple routes; GitHub Copilot drops eight ids (e.g. claude-sonnet-4.5, gpt-5.2-codex, older Opus/Sonnet/GPT entries) and gains claude-fable-5.1 and gemini-3.8-flash. Several rows are renamed or retuned (Fireworks DeepSeek ids, OpenRouter “fast” Opus removals, Google gemini-robotics-er removal, etc.).

Tests that typed getModel("github-copilot", …) against removed ids now use claude-sonnet-4.6 and gpt-5.3-codex; the Copilot interleaved-thinking mock test switches to claude-haiku-4.5 because it is the remaining non-adaptive Claude in that catalog. A one-line .changes note documents the refresh.

Reviewed by Cursor Bugbot for commit 8a7bc2a. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Refresh generated model catalog from live provider sources

  • Regenerates models.generated.ts with updated metadata: 41 models added, 17 removed, and existing entries updated for provider, API, capability, pricing, context-window, and cache-cost fields.
  • Migrates GitHub Copilot test fixtures from Claude Sonnet 4.5 to Claude Sonnet 4.6 and from GPT-5.2 Codex to GPT-5.3 Codex across context-overflow.test.ts, stream.test.ts, and other test suites.
  • Switches the interleaved-thinking beta-header test to Claude Haiku 4.5.
  • Risk: removed and renamed model identifiers in models.generated.ts may break callers referencing deprecated IDs.

Macroscope summarized 8a7bc2a.

…rces

Explicit, reviewed regeneration via `npm run generate-models`: 1238 -> 1262
models (41 added, 17 removed). No provider dropped wholesale or lost >30% of
its rows.

GitHub Copilot removed claude-opus-4.5, claude-opus-4.6, claude-sonnet-4,
claude-sonnet-4.5, gemini-3.1-pro-preview, gpt-4.1, gpt-5.2, and gpt-5.2-codex;
its stale claude-sonnet-4.5 entry is what broke CI type checks against the
regenerated catalog. Test references to the removed Copilot models move to
claude-sonnet-4.6 and gpt-5.3-codex; the interleaved-thinking beta test uses
claude-haiku-4.5, the only remaining non-adaptive Claude in the Copilot
catalog. All other providers' referenced ids are unchanged.

Every defaultModelPerProvider entry exists in the refreshed catalog except the
pre-existing zai glm-5.1 gap already fixed in #2032.

Linear: RES-1269
@snimu
snimu requested a review from xeophon September 4, 2026 10:26
@xeophon
xeophon merged commit 3484f06 into main Sep 4, 2026
26 checks passed
@xeophon
xeophon deleted the chore/refresh-model-catalog branch September 4, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants