feat: refresh models from a hosted catalog

[sethkarten]

Part of ENG-5435. Supersedes #1633.

What changed

• Publishes model-catalog.json to the existing Prime Agent R2 origin every day and on manual dispatch.
• Aggregates models.dev, OpenRouter, Vercel AI Gateway, Prime Inference, and the maintained static provider definitions.
• Treats the validated hosted catalog as the authoritative public built-in model list. Provider additions, removals, names, capabilities, and prices apply without a Prime Agent release.
• Refreshes at runtime with a 24-hour atomic disk cache. A stale validated cache or the release-bundled catalog is used when refresh fails.
• Keeps local models.json definitions, model overrides, private Prime routes, and extension providers above the hosted catalog.

Safety

• The publication job runs in strict mode. Any failed, malformed, or empty upstream source stops the job before upload. The resulting aggregate is then checked for schema errors and duplicate model IDs.
• Runtime responses have size, count, schema, price, and metadata bounds.
• Hosted entries cannot redirect provider credentials. api, baseUrl, and headers stay pinned to a matching transport bundled in the installed Prime Agent release.
• PI_OFFLINE=1 disables network refreshes. PRIME_AGENT_MODEL_CATALOG_URL overrides the endpoint for development.

Validation

• npm run check
• packages/ai/test/model-catalog-format.test.ts: 4 passed
• packages/ai/test/model-compat-schema.test.ts: 2 passed
• packages/coding-agent/test/remote-model-catalog.test.ts: 11 passed
• packages/coding-agent/test/model-registry.test.ts: 72 passed
• packages/coding-agent/test/version-check.test.ts: 6 passed
• packages/coding-agent/test/package-manager.test.ts: 95 passed
• packages/coding-agent/test/tools-manager.test.ts: 6 passed
• Strict live generation and validation: 1,244 models across 32 providers, with zero hosted header fields
• Authoritative merge smoke test: all 1,244 entries accepted, with 14 additions and 8 removals versus the bundled catalog

---

Note

Medium Risk
Changes which models users see and at what price, with merge rules that block remote transport redirects but still trust hosted metadata for matched transports; publication strict mode limits bad uploads, but runtime uses lenient parsing with skipInvalidModels.

Overview
Adds a hosted, versioned model catalog so public model names, capabilities, and pricing can update without a Prime Agent release.

A new GitHub Actions workflow runs daily (and on dispatch): it generates the aggregate catalog in strict mode, validates it, and uploads model-catalog.json to R2. packages/ai introduces createModelCatalog / parseModelCatalog, shared compat schemas, a validate-model-catalog script, and stricter generate-models behavior when PRIME_AGENT_MODEL_CATALOG_STRICT=1 (failed or empty upstream sources abort publication).

Prime Agent loads built-ins by merging the hosted list into the release-bundled catalog via remote-model-catalog.ts: remote rows update metadata but pin api, baseUrl, and headers to a matching bundled transport; truncated or invalid responses fall back to a 24h atomic disk cache or bundled models. ModelRegistry reads the cache at startup and refreshes asynchronously; PI_OFFLINE=1 and PRIME_AGENT_MODEL_CATALOG_URL control network behavior. Local models.json overrides and extension providers still win on conflicts.

^{Reviewed by Cursor Bugbot for commit 97d6e39. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Add hosted model catalog with daily refresh and validated disk cache

• Adds a scheduled GitHub Actions workflow that generates a validated aggregate model catalog and uploads it to an R2 bucket
• Adds createModelCatalog and parseModelCatalog in model-catalog.ts with a versioned TypeBox schema, deterministic sorting, and duplicate detection
• Adds remote-model-catalog.ts which fetches the hosted catalog with size/time limits, validates it, writes an atomic disk cache, and merges remote metadata into bundled models while pinning transport details (provider+api+baseUrl) and rejecting remote transport changes
• ModelRegistry in model-registry.ts now loads the cached catalog on startup, refreshes it asynchronously, and merges remote metadata before applying local overrides; PI_OFFLINE skips refresh
• generate-models.ts gains a strict mode (PRIME_AGENT_MODEL_CATALOG_STRICT) that fails fast on missing/empty provider catalogs and can emit a catalog to PRIME_AGENT_MODEL_CATALOG_OUTPUT; it now sets process.exitCode = 1 on fatal errors instead of silent logging
• Risk: remote entries are only accepted when they match a bundled transport (provider+api+baseUrl); remote transport changes are silently dropped — verify merge logic in remote-model-catalog.ts if a provider changes its base URL

Changes since #1928 opened

• Introduced strict validation for catalog model entries with per-model validation and optional skipping of invalid or incompatible entries [95dfe25]
• Enforced minimum catalog size thresholds to ignore materially truncated remote catalogs throughout the refresh pipeline [95dfe25]
• Refactored API compatibility checking to cover all bundled APIs with centralized schema validation [95dfe25]
• Changed model catalog refresh to run asynchronously in the background with deferred model updates [95dfe25]
• Required HTTPS scheme for remote model catalog URLs with centralized base URL construction [95dfe25]
• Selected the lexicographically lowest model id as the template when multiple bundled models share the same transport [95dfe25]
• Prevented readCachedRemoteModelCatalog from throwing on malformed caches [95dfe25]
• Added stricter cache validation to refreshRemoteModelCatalog in the coding-agent package requiring the provided now timestamp to be greater than or equal to fetchedAt before treating cached model catalog data as fresh, and added test coverage verifying that a fetch is triggered when now is earlier than the cached fetchedAt timestamp [97d6e39]

^{Macroscope summarized a1abe48.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit aeb795a. Configure here.}

[snimu]

The R2 credential used here should be confirmed bucket-scoped and write-only, since a broader token would make this workflow the catalog's weakest link. Nothing in-repo can verify this — flagging for a one-time check in the Cloudflare dashboard.

[written by prime-agent, checked by snimu]

[sethkarten]

Agreed. This cannot be verified from the repository. I am leaving this thread open and escalating a one-time Cloudflare dashboard check that the workflow credential is restricted to the intended bucket with the narrowest available object-write permission.