feat(id): adaptive on-device bird ID router (BioCLIP-2, automatic)

[jlian]

Adaptive model router for bird ID: on-device BioCLIP-2 when available, GPT otherwise. Fully automatic and invisible to the user (no setting): the model prefetches silently on a suitable network and identification transparently becomes on-device once it is cached, falling back to GPT otherwise. Backed by a spike that measured BioCLIP-2 + our range pipeline at 87% top-1 / 96% top-5 vs GPT's 83% / 87% on the 27-image benchmark.

The key design point: NOT a divergent pipeline

Both BioCLIP and GPT emit {species, confidence}[], and the entire post-processing path is shared (taxonomy grounding → range-prior tiering → confidence gate). The router only swaps which model produces the candidate list. On-device candidates go through the same range logic as GPT via a new /api/range-adjust endpoint — this is what you already do server-side today (decision 1a), just callable with a client-produced candidate list.

What's included

Server

• functions/api/range-adjust.ts — shared pipeline for on-device candidates: taxonomy grounding + range-prior tiering + dominance gate (trust the visual ID when the top candidate dominates by ≥0.5; otherwise hard-tier by range status: present > near-range > out-of-range). Mirrors the spike's winning strategy. 9 unit tests.
• functions/models/[[path]].ts — streams model assets from the MODELS R2 bucket with immutable caching + HTTP range support (files exceed CF's 25 MiB static-asset limit, so R2 is required).
• MODELS R2 binding (prod + preview) + Env type.

Client

• src/lib/bioclip-inference.ts — onnxruntime-web (WebGPU, WASM fallback), model loader with download-speed measurement + Cache API persistence, image encoder + int8 text-matmul + softmax, identifyOnDevice() → /api/range-adjust.
• src/lib/model-router.ts — identifyBirdAdaptive() + network-gated background prefetch. Policy: model ready/cached → on-device; otherwise GPT now + silently warm the model for next time. Prefetch only runs on a suitable network (not Save-Data/cellular/slow), so we never pull 307 MB on metered data; a full download never blocks an identify.
• AddPhotosFlow uses the router and warms the model on mount (network-gated). No feature flag, no settings toggle — the routing is automatic.
• onnxruntime-web is dynamically imported so the main bundle is unaffected when the feature is off (verified: it code-splits out).

Verification

• Full suite 709 tests green; new endpoint has 9 tests; tsc -b, eslint, and npm run build (Vite + Worker) all clean.
• Inference path validated end-to-end via onnxruntime (same API as the browser): faithful embeddings, ~335 ms/img CPU.

⚠️ What's left / required to actually run this (NOT in this PR)

1. Create + populate the R2 bucket. DONE. The wingdex-models bucket is created and populated (all 4 assets uploaded + size-verified: onnx 306.9 MB, text_embeds_int8 8.6 MB, scale 44 KB, species.json 509 KB). R2 buckets are account-global, so the preview reads the same bucket. On-device ID activates automatically on the preview once the model finishes its background download and caches (network-gated; no toggle).
2. Real WebGPU browser validation. Inference was validated in Node (onnxruntime, same API); actual in-browser WebGPU latency + the download/cache UX haven't been measured in a live browser yet. The Cloudflare preview for this PR is how we'll do that (on-device warms automatically on a suitable network; watch the Network tab for the model download, then it runs in-browser).
3. Prefetch/download progress UX. Prefetch runs silently in the background; there's no progress UI / "preparing offline ID…" indicator yet. loadModel() reports download progress; surfacing it is follow-up.
4. Offline range. On-device inference still calls /api/range-adjust for range refinement (needs network). True offline would ship a regional range table client-side — deferred.
5. iOS / Core ML. Not in this PR. The 307 MB model is a non-issue bundled on the Neural Engine; that's a separate track.
6. Model size. 307 MB int8 is the only variant that beats GPT (ViT-B/16 = 70/74, q4 = 78/87, both too weak — see spike/bioclip/BROWSER.md). Cold-mobile is a rude first download, so realistic UX is "GPT by default, on-device once cached." A distilled <25 MB bird-only student (BioCLIP-2 is MIT, so open-weight release is possible) would remove the download problem entirely — tracked in R&D: distill a small (<25 MB) bird-only on-device model from BioCLIP-2 #260.

Notes

• Model assets are not committed (307 MB). The spike (spike/bioclip/, incl. the standalone browser demo and all measurements) lives on the spike/bioclip-birdid branch.
• Depends on nothing else, but pairs naturally with fix(range): scan full 3x3 neighbor ring for range priors #258 (the 8-neighbor range fix), which improves coastal accuracy for both models.

[Copilot]

Pull request overview

Adds an adaptive bird-ID routing path that can use an on-device BioCLIP-2 model (when enabled and cached) while keeping the existing shared server-side post-processing (taxonomy grounding, range priors, and gating) as the single pipeline.

Changes:

• Introduces /api/range-adjust to apply the shared taxonomy + range + gating pipeline to client-produced candidate lists.
• Adds /models/* asset serving from an R2 bucket (with range support) plus an upload script for model assets.
• Adds client-side on-device inference (onnxruntime-web), an adaptive router, and a Settings feature flag with background prefetch.

Reviewed changes

Copilot reviewed 11 out of 15 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
wrangler.toml	Adds MODELS R2 bucket binding (prod + preview).
tsconfig.json	Excludes the new range-adjust test from TS compilation.
src/lib/storage-keys.ts	Adds a global (non-user-scoped) feature flag for on-device ID.
src/lib/model-router.ts	Adds identifyBirdAdaptive() routing logic and one-time background prefetch.
src/lib/bioclip-inference.ts	Implements on-device BioCLIP inference, caching, and /api/range-adjust integration.
src/components/pages/SettingsPage.tsx	Adds Settings toggle to enable on-device ID and trigger prefetch.
src/components/flows/AddPhotosFlow.tsx	Switches fast identify path to the adaptive router and warms the model on mount.
src/tests/range-adjust.test.ts	Adds unit tests for /api/range-adjust behavior.
scripts/upload-model.mjs	Adds an R2 upload helper for model assets.
package.json	Adds onnxruntime-web dependency.
package-lock.json	Lockfile updates for onnxruntime-web and transitive deps.
functions/models/[[path]].ts	Adds /models/* route that streams R2 assets with immutable caching and range support.
functions/env.d.ts	Adds optional MODELS binding to Env.
functions/api/range-adjust.ts	Adds the range-adjust endpoint implementing dominance gate + range tiering.
.gitignore	Ignores functions/tsconfig.tsbuildinfo.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 13 changed files in this pull request and generated 4 comments.

[github-actions]

🔎 Preview for this PR: https://pr-259-wingdex-app-preview.lianguanlun.workers.dev

Isolated per-PR alias, no longer shared/overwritten across PRs. Updates on every push.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 13 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/lib/model-router.ts:60

• isNetworkSuitableForPrefetch() returns true when navigator.connection is unavailable (Safari/Firefox). That means maybePrefetchModel() can start a ~300 MB background download even on cellular/metered networks, which contradicts the PR's stated policy (no Save-Data/cellular/slow). Consider treating unknown network info as "not suitable" so we only prefetch when the browser can positively identify a safe connection (or when the assets are already cached).

 * never warms there. Callers can still gate on isModelCached() for identify.
 */
export function isNetworkSuitableForPrefetch(): boolean {
  const c = getConnection()
  if (!c) return true // unknown (Safari/FF): allow; download is cached + one-time

[Copilot]

Pull request overview

Copilot reviewed 9 out of 13 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 13 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

src/lib/bioclip-inference.ts:106

• When Cache API is unavailable, the asset fetches (text embeds, scale, species.json) don't check resp.ok, so 404/500 responses get cached/parsed as bytes and later fail with confusing JSON/tensor errors. Mirror fetchCached() behavior by validating responses and throwing a clear error message.

  const [tBuf, sBuf, spBuf] = await Promise.all([
    cache ? fetchCached(cache, TEXT_EMBEDS_URL) : fetch(TEXT_EMBEDS_URL).then(r => r.arrayBuffer()),
    cache ? fetchCached(cache, TEXT_SCALE_URL) : fetch(TEXT_SCALE_URL).then(r => r.arrayBuffer()),
    cache ? fetchCached(cache, SPECIES_URL) : fetch(SPECIES_URL).then(r => r.arrayBuffer()),
  ])