Skip to content

feat(be): route ADK agents through self-hosted Claude gateway via flag - #1

Draft
ccp-manash wants to merge 2 commits into
mainfrom
use-custom-llm-proxy
Draft

feat(be): route ADK agents through self-hosted Claude gateway via flag#1
ccp-manash wants to merge 2 commits into
mainfrom
use-custom-llm-proxy

Conversation

@ccp-manash

Copy link
Copy Markdown

Summary

Adds optional support for routing every ADK agent through a self-hosted claude-internal-gateway (Anthropic Claude proxy) instead of calling Gemini directly. The change is gated by a single env var (USE_CLAUDE_GATEWAY) so the existing Gemini path remains the default and can be toggled back instantly.

Zero new npm dependencies. The adapter uses only @google/adk (already installed) and Node's built-in fetch. No @anthropic-ai/sdk, no litellm, no @google/genai.

Why

  • We want to A/B test Claude (Opus 4.6 with adaptive thinking) vs Gemini 2.5 Flash on the actual agent workflows without forking the codebase or maintaining two versions.
  • Manash's Anthropic API key needs to be shared with the team without exposing the raw key. The gateway is the auth layer; agents authenticate to it with a separate, rotatable proxy token.
  • The flag-based design means we can flip back to Gemini at any time without a code change or PR revert.

How it works

  1. src/agent/oak_claude_llm.ts — a custom BaseLlm subclass (OakClaude) that translates between ADK's GenAI request/response shape and Anthropic's Messages API. Handles text + thinking + tool_use translation and normalizes Gemini-flavored function-declaration schemas (uppercase OBJECT enum) into JSON Schema before forwarding to Anthropic.

  2. src/agent/llmFactory.ts — a tiny getModel({ gemini, claude }) factory that returns either a Gemini model string or an OakClaude instance based on USE_CLAUDE_GATEWAY. Single source of truth.

  3. All 6 agent files updated to use getModel({ gemini: \"gemini-2.5-flash\", claude: \"claude-opus-4-6\" }) instead of a hardcoded model string. Each diff is exactly 2 lines (1 import + 1 model field).

  4. scripts/try_oak_claude.ts — throwaway end-to-end smoke test that wires up an LlmAgent with OakClaude, runs it through InMemoryRunner against a real ADK tool (getCurrentTime), and verifies the full tool round-trip works. Used to validate the integration before touching production code; can be deleted after rollout is stable.

Default behavior

Without setting any new env var, nothing changes. Agents continue to use gemini-2.5-flash exactly as before. To opt in:

```
USE_CLAUDE_GATEWAY=true
OAK_GATEWAY_URL="https://tunnel.cfsprotocol.com\"
OAK_PROXY_TOKEN=""
```

What was tested

End-to-end against a locally-running gateway, in this exact repo:

```
$ USE_CLAUDE_GATEWAY=true OAK_PROXY_TOKEN=... \
OAK_GATEWAY_URL=http://127.0.0.1:8765 \
npx tsx scripts/try_oak_claude.ts

=== STAGE A: minimal LlmAgent + 1 tool via OakClaude ===
[tool_call] name=get_current_time args={}
[tool_result] name=get_current_time response={"time":"2026-04-06T16:44:50.283Z"}
[final_text] The current time is 4:44 PM UTC on April 6, 2026.
STAGE A: ok (tool called, final answer produced)
ALL OK
```

This validates the full agent loop: ADK runtime → custom adapter → gateway → Anthropic → tool_use response → ADK tool execution → tool_result → second LLM call → final text answer.

Test plan

  • Reviewer sets USE_CLAUDE_GATEWAY=false (default) and confirms the existing Gemini path still works
  • Reviewer sets USE_CLAUDE_GATEWAY=true with OAK_GATEWAY_URL + OAK_PROXY_TOKEN, runs npx tsx scripts/try_oak_claude.ts, confirms ALL OK
  • Reviewer flips the flag and runs at least one production agent via the orchestrator (ENABLE_AGENTS=true) to confirm a real autonomous loop works on Claude
  • Confirm no changes to package.json / package-lock.json / tsconfig.json (verified by CI / git diff)
  • Decide whether to delete scripts/try_oak_claude.ts before marking PR ready, or keep it as a permanent smoke test

Things NOT in this PR

  • No changes to package.json, package-lock.json, tsconfig.json, Dockerfile, or any tool / trigger / database / express code.
  • No changes to agent prompts, instructions, descriptions, or tool sets.
  • No new dependencies of any kind.
  • The gateway server itself (lives in a separate `claude-internal-gateway` repo).

Known limitations / followups

  • Cost: Claude Opus 4.6 with adaptive thinking is meaningfully more expensive than gemini-2.5-flash. Suggest cheap-tier agents (commentator, backer, root) move to `claude-sonnet-4-6` if cost is a concern. Easy follow-up: change one string per agent file.
  • Latency: Claude with extended thinking is slower (5-30s on hard prompts). Trigger timeouts in the orchestrator may need adjustment.
  • The trial script does not exercise database-touching tools (avoided to skip Turso/Firebase setup). The schema normalizer in the adapter handles complex parameters, but they have not been exercised against real arg-taking tools yet.

🤖 Generated with Claude Code

Adds the building blocks for routing ADK agents through the
self-hosted claude-internal-gateway (Anthropic Claude proxy)
without modifying any existing agent files.

- src/agent/oak_claude_llm.ts: a custom @google/adk BaseLlm subclass
  that translates between ADK's GenAI request/response shape and the
  Anthropic Messages API. Uses only @google/adk + native fetch; no
  @anthropic-ai/sdk, no litellm, no @google/genai dependency. Handles
  text + thinking + tool_use translation in both directions and
  normalizes Gemini-flavored function-declaration schemas (uppercase
  type enum) into JSON Schema before forwarding to Anthropic.

- src/agent/llmFactory.ts: a tiny factory function getModel({gemini,
  claude}) that returns either a Gemini model string or an OakClaude
  instance based on the USE_CLAUDE_GATEWAY env var. Single source of
  truth for model selection across all agents.

- scripts/try_oak_claude.ts: throwaway smoke test that verifies the
  full LlmAgent + OakClaude + tool round-trip works against a local
  gateway. Used to validate the migration before touching production
  agent files. Can be deleted after the rollout is confirmed stable.
… flag

Switches every LlmAgent's model field from a hardcoded
"gemini-2.5-flash" string to getModel({gemini, claude}).

When USE_CLAUDE_GATEWAY=true the agents will use claude-opus-4-6
through the gateway. When the flag is unset or false, they continue
to use gemini-2.5-flash exactly as before, so this change is fully
backwards-compatible by default.

Touched agents (each is a 2-line diff):
- backerAgent
- campaignCreatorAgent
- commentatorAgent
- financialAgent
- rootAgent
- scoringAgent

.env.example updated with the new USE_CLAUDE_GATEWAY,
OAK_GATEWAY_URL, and OAK_PROXY_TOKEN variables. GEMINI_API_KEY is
kept and still required when the flag is off.

No npm dependencies added; the adapter uses only @google/adk and
native fetch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant