Skip to content

coding-agents.md

Latest commit

 

History

History
64 lines (43 loc) · 1.94 KB

coding-agents.md

File metadata and controls

64 lines (43 loc) · 1.94 KB
Raw

Cookbook: Coding agents through Switchyard

Point Claude Code, Codex CLI, or OpenClaw at open models without rewriting the agent.

Why it works

Coding agents speak a fixed API (often Anthropic Messages or OpenAI). Your best execute model might be NIM / vLLM / Ollama. Switchyard:

  1. Accepts the agent’s native protocol
  2. Routes plan vs tool-heavy turns (stage router)
  3. Translates to the backend wire format
  4. Streams the response back

Setup

uv tool install --python 3.12 "nemo-switchyard[cli,server]"
export OPENROUTER_API_KEY=...   # or local-only config

# Packaged deployment
switchyard launch claude --model switchyard

# Custom Lightning + frontier split
switchyard launch claude \
  --model agent-route \
  --config configs/routes.lightning_frontier.toml

Recommended route

For PR-fix / test-fix agents, prefer stage_router:

  • File reads, greps, patches → efficient (Lightning / mini)
  • Test failures, retries, architecture questions → capable (frontier)

See configs/routes.stage_router.toml and the offline simulation in examples/02_stage_router.py.

Local models

  1. Start vLLM / Ollama / NIM with an OpenAI-compatible /v1.
  2. Add an llm_clients block without api_key_env if none is needed.
  3. Point a target at that model ID.
  4. Use passthrough or stage efficient_target → local.

Snippet in configs/routes.lightning_frontier.toml.

Validation

# After server is up
export SWITCHYARD_URL=http://127.0.0.1:4000
export SWITCHYARD_MODEL=agent-route
python examples/07_proxy_client.py

Observability

Export Prometheus metrics from the real server. In your harness (or this kit’s summarize_run), log:

  • route id / target / model
  • tokens, latency, success
  • whether the step was an escalate

Optimize $ / green CI not tokens in isolation.