Cerberus: Load-Balancing AI Proxy Server

Cerberus is a high-performance, Node.js-based proxy and load-balancing engine designed to pool and manage Google AI Studio Free Tier credentials (API Keys and Service Accounts). By providing an OpenAI-compatible API endpoint (/v1/chat/completions), Cerberus abstracts the complexity of rate limits, key rotation, and quota management, enabling uninterrupted and scaled API consumption for downstream applications.

System Architecture

The architecture of Cerberus is modular, separating concerns into discrete core components:

• ProxyEngine (src/core/proxyEngine.js): The primary request handler. It intercepts incoming OpenAI-format requests, delegates credential selection to the AccountManager, maps the request schema to the Google Gemini API format, and streams (or returns) the completion response back to the client.
• AccountManager (src/core/accountManager.js): Responsible for state management of all registered credentials. It loads configurations from accounts/accounts.txt and keys/, maintains an in-memory pool, handles round-robin distribution, and dynamically quarantines accounts that encounter 429 (Too Many Requests) or 503 errors.
• QuotaMonitor (src/core/quotaMonitor.js): Executes proactive and reactive quota telemetry. It monitors quota depletion across all loaded Service Accounts, providing metrics to preemptively switch keys before hard limits are reached.
• Logger (src/core/logger.js): A centralized event stream processor. It structures internal state changes, HTTP events, and errors, broadcasting them via WebSocket to the dashboard for real-time observation.

Features

• OpenAI API Compatibility: Fully mimics the OpenAI chat/completions REST interface, allowing seamless drop-in replacement for libraries such as openai-node or langchain.
• Advanced Load Balancing: Implements intelligent round-robin key rotation with automatic quarantine mechanisms for rate-limited credentials.
• Service Account & API Key Support: Supports parsing both bare API keys and complete Google Cloud IAM Service Account JSON structures.
• Real-Time Telemetry Dashboard: Serves a static frontend via Express, backed by a WebSocket server (ws), streaming active logs, quota statistics, and account health statuses to the client.
• Quarantine & Auto-Release: Temporarily blocks exhausted credentials and re-introduces them to the active pool based on configurable TTL parameters.

Installation & Setup

Prerequisites

• Node.js runtime (v18.0.0 or higher recommended)
• NPM or Yarn package manager

Initialization

1. Clone the repository and install dependencies:

   npm install

2. Environment Configuration: Duplicate the provided .env.example file to create your active environment configuration.

   cp .env.example .env

   Adjust the .env variables (e.g., PORT, QUOTA_POLL_INTERVAL) as dictated by your deployment environment.

Credential Provisioning

Cerberus strictly ignores sensitive credential files via .gitignore to prevent accidental credential leakage to version control. You must manually provision the following directories:

• accounts/accounts.txt: Add your API keys and Service Account JSON structures here. The file is parsed sequentially. Follow the delimiter format established in the internal documentation.
• keys/: Place any external .json Service Account key files within this directory. The AccountManager will automatically ingest them during the initialization phase.

Usage

Starting the Server

To launch the server in a production environment:

npm start

For development (utilizing Node's native watch mode):

npm run dev

API Endpoints

• HTTP POST /v1/chat/completions: The primary inference endpoint. Accepts standard OpenAI completion schemas.
• HTTP GET /v1/models: Returns the list of virtual models mapped and supported by the internal transformer logic.
• HTTP GET /v1/health: Returns system uptime and active account pool size.

Dashboard Access

The administrative dashboard can be accessed via a standard web browser at the configured port:

http://localhost:3000/dashboard

The dashboard establishes a WebSocket connection to receive pushed updates regarding the AccountManager snapshot and Logger output.

Security Considerations

The .gitignore has been rigorously configured to exclude .env, accounts/*.txt, keys/*.json, and local database files. Ensure that the proxy is deployed within a secure VPC or behind a reverse proxy (e.g., NGINX) enforcing TLS/SSL and adequate authentication layers, as the default configuration exposes endpoints openly.