feat: @catalyst/cloud-ai — multi-provider AI integration with useAI hook

[mayankmahavar1mg]

Summary

• @catalyst/cloud-ai package — consolidated AI package exporting useCloudAI, useWebAI, useNativeAI, and buildAttachmentSystemPrompt. Multi-provider server route (POST /ai/:provider/stream SSE + POST /ai/:provider/generate JSON) supporting OpenAI and Gemini. All routes gated behind AI_CONFIG.enabled.
• useAI hook (catalyst-core/hooks) — orchestrator that routes to the right sub-hook based on provider option: openai/gemini → useCloudAI, transformers → useWebAI, native → useNativeAI (falls back to useCloudAI if NativeBridge.isAIAvailable() is false). Stateful sessions supported via sessionMode='stateful'.
• Gemini stateful sessions — uses Interactions API (v1beta/interactions), correct SSE event fields (event_type, parsed.interaction.id), stateful flag derived from key presence in request body.
• Android native AI — android/ module ships inside @catalyst/cloud-ai with AIBridge/AIBridgeCallbacks interface. NativeBridge.kt uses reflection (Class.forName + Proxy.newProxyInstance) — zero static imports, compiles without :catalyst-cloud-ai on classpath. build.gradle.kts conditionally bundles LiteRT. buildAndroid/index.js syncs the package before Gradle runs.
• examples/useai — full test app with Cloud/Local/Native panels, streaming toggle, provider switcher, stateful session toggle, attachment component rendering, generation settings, and system prompt editor.
• MCP knowledge base — 5 new useAI integration entries + mcp.js robustness fixes (empty DB, Node version mismatch errors).

Test plan

• Run npm run lint — passes clean
• Start examples/useai dev server, test cloud panel with OpenAI (stream + non-stream)
• Test provider switch to Gemini (stream + non-stream) once quota resets
• Toggle sessionMode=stateful — verify turn 1 returns conversationId, turn 2 maintains context
• Test local panel with transformers provider — model download progress + inference
• Test native panel on Android — isAIAvailable() check, fallback to cloud when bridge absent
• Verify AI_CONFIG.enabled=false returns 403 on all /ai/* routes
• Copy config/config.template.json → config/config.json, fill API keys, confirm no keys in git

🤖 Generated with Claude Code

https://claude.ai/code/session_01A1tWZLoJqQQ7Y7sRDXUL9r

---

Update: OpenCL fix, native/web metrics parity, docs, CodeRabbit fixes

Root cause fix — native AI crash (Can not find OpenCL library on this device)
litertlm-android was pinned to latest.release, which silently resolved to 0.14.0 mid-session and introduced an OpenCL sampler regression (fails at first generate() call, even on Backend.CPU(), since the sampler probes OpenCL regardless of the configured backend). Confirmed via local Gradle cache timestamps (0.13.1 cached weeks earlier, 0.14.0 pulled the same day the crash appeared) and matches google-ai-edge/LiteRT#6999 and google-ai-edge/LiteRT-LM#1860. Pinned to 0.13.1 (last known-good) in packages/catalyst-cloud-ai/android/build.gradle.kts.

Native (useNativeAI) — non-streaming generate + real session metrics

• New POST /framework-{sessionId}/ai/generate Ktor route (drains the token flow, returns one JSON response) alongside the existing SSE /ai/stream route.
• useNativeAI.js gains a non-streaming branch, a historyRef accumulator, and real getSessionMetrics/resetSessionMetrics backed by a new aggregateNativeSessionMetrics() — replacing stubs. Native's aggregate shape has no cost/cache fields (no billing, one local model), unlike cloud.
• Two runtime bugs fixed before this went live: the new Kotlin route wasn't installing Ktor ContentNegotiation (would've produced invalid JSON), and totalTokens was computed inconsistently between streaming (token-frame count) and non-streaming (char count) branches, which would've corrupted session aggregation across mixed-mode sessions.

Web (useWebAI) — session metrics + stateful chat (marked experimental)

• New aggregateWebSessionMetrics() (device/dtype instead of cost, same no-billing shape reasoning as native).
• Client-side stateful mode: since Transformers.js pipelines have no server-side session, "stateful" means the hook replays accumulated {role, content} history into each generate() call, with a locally-generated conversationId for the UI to key off.
• Marked experimental throughout (code comments, KB, docs, UI label) — in-browser inference quality (esp. instruction-following on larger models like Llama 3) and WebGPU/WASM backend selection aren't reliable yet. This is plumbing parity, not a claim that local inference is production-ready.

examples/useai UI

• Panel.js's Session Stats modal now has a third isLocal branch (was only isNative ? native : cloud, which would've rendered bogus $0.00000 cost tiles for web).
• AITest.js gains a Local Session stateful toggle alongside the existing Cloud/Native ones (previously the Local panel had no sessionMode wiring at all).

Docs

• Added a full useAI reference section to docs/content/11-API Reference/01-Hooks.md, matching the file's existing per-hook pattern (Import/Options/Returns/Usage), with dedicated subsections for what actually differs per provider: genConfig/route behavior, stateful-session mechanics, and metrics shape.
• Corrected the MCP knowledge base's ai_integration entries, which previously described stateful sessions and metrics as if all three providers worked identically — they don't (see above).

CodeRabbit review fixes (coderabbit review --agent --base-commit e356eb9, scoped to this branch's own commits — 8 findings total, 1 in our code, 3 more picked up and fixed since they were straightforward)

• FrameworkServerUtils.kt: /ai/generate's tokenFlow.collect had no timeout — a stalled native supplier would hang the request forever. Wrapped in a 120s withTimeout, responds 504 on timeout.
• scripts/test-catalyst-core.sh: guarded against a silent fallback to the published catalyst-core package if npm pack doesn't produce a tarball.
• buildAndroid/assets.js: scoped the notification meta-data removal regex to a single tag (was using a dot-all wildcard that could span tag boundaries and delete unrelated <meta-data> entries).
• buildIos/assets.js: removed an unnecessary async from validateNotificationAsset (no awaits inside) so sync errors surface to the caller's try/catch instead of becoming unhandled promise rejections.

Security checks (semgrep, trivy vuln/secret/misconfig, npm audit) all came back clean on the files touched this session; the 7 moderate npm audit findings in examples/useai are pre-existing transitive deps (file-type, http-proxy-middleware, imagemin, sockjs, uuid, webpack-dev-server) requiring a catalyst-core major bump, unrelated to this PR's scope.

[deputydev-agent]

DeputyDev will no longer review pull requests automatically.To request a review, simply comment #review on your pull request—this will trigger an on-demand review whenever you need it.