Skip to content

About

Zero-latency local reverse proxy & circuit breaker for LLM client tools to prevent 429 quota deadlocks

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

quota-guard

License: MIT

A lightweight, zero-latency local reverse proxy and circuit breaker designed specifically for AI coding agents and LLM client tools.


Why quota-guard?

When running autonomous coding agents (OpenCode, Cursor, Aider, Claude Code), hitting a 429 Too Many Requests or quota exhaustion stops the entire loop dead in its tracks.

quota-guard sits locally between your tools and upstream providers. When your cloud quotas run dry or rate-limits spike:

  1. Trips the circuit breaker immediately without dropping connections.
  2. Transparently reroutes streaming requests to local engines (Ollama, LocalAI, vLLM) or a secondary provider.
  3. Probes upstream in HALF_OPEN state after a configurable cooldown and restores primary traffic automatically.

Quick Start

# Point your agent to localhost:8080 instead of https://api.openai.com
npx quota-guard --upstream https://api.openai.com --fallback http://127.0.0.1:11434

Check live traffic and circuit health anytime:

curl http://127.0.0.1:8080/_quota_guard/stats

CLI Options

Flag Short Default Description
--port -p 8080 Local port to bind proxy.
--upstream -u https://api.openai.com Primary upstream API.
--fallback -f http://127.0.0.1:11434 Fallback model server (e.g. Ollama).
--threshold -t 2 Number of consecutive 429/5xx before circuit opens.
--cooldown -c 30000 Cooldown period (ms) before testing upstream again.

License

MIT © luckdevx

About

Zero-latency local reverse proxy & circuit breaker for LLM client tools to prevent 429 quota deadlocks

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages