GitHub - x1051445024/octopus: One Hub All LLMs For You | 为个人打造的 LLM API 聚合服务|澪雨的Fork改版|来点人提交issue，或者来点pr吧

Octopus

A Simple, Beautiful, and Elegant LLM API Aggregation & Load Balancing Service for Individuals

English | 简体中文 | Changelog

✨ Features

🔀 Multi-Channel Aggregation - Connect multiple LLM provider channels with unified management
🔑 Multi-Key Support - Support multiple API keys for a single channel
⚡ Smart Selection - Multiple endpoints per channel, smart selection of the endpoint with the shortest delay
⚖️ Load Balancing - Support round robin, random, failover, weighted, and auto strategies
🤖 Auto Strategy - Explore candidates first, then prefer higher in-window success rate automatically
🧠 AI Routing, Auto Grouping & Conditional Groups - Generate the full routing table from the route page, fill a single group from the edit dialog, and gate groups with JSON conditions
🔄 Protocol Conversion - Seamless conversion between OpenAI Chat / OpenAI Responses / OpenAI Embeddings / Anthropic API formats
🌐 Multi-Provider Support - Built-in support for OpenAI-compatible, Anthropic, Gemini, Volcengine, and MiMo channels
🛰️ Media & Utility Relay - Relay OpenAI Images, audio, video, search, rerank, and moderation endpoints through the same group / retry / circuit-breaker infrastructure
🧾 API Key Governance - Supported-model allowlists, expiry, max-cost caps, RPM / TPM limits, per-model quotas, and IP / CIDR allowlists
🔐 Role-Based Admin Access - Built-in admin, editor, and viewer roles with server-side permission enforcement
🚨 Alerts & Notifications - Alert rules for error rate, cost threshold, quota exceeded, and channel down with webhook, Gotify, email, Telegram, Feishu, DingTalk, WeCom, and ntfy notification channels and history
💎 Model Market - Unified model catalog with pricing, channel coverage, enabled key counts, latency, and success metrics, plus create / edit / delete / refresh price workflows
🔃 Model Sync - Automatic synchronization of available model lists with channels
📊 Analytics & Evaluation - Overview, provider / model / API key utilization, route health, latency distribution, semantic-cache evaluation, provider prompt-cache analytics, and live entry points for group testing / AI routing
🛠️ Ops & Audit - Telemetry, quota, health, system, and audit dashboards for daily operations, plus a management-write audit trail
🧠 Semantic Cache - Embedding-backed semantic cache for non-streaming and streaming OpenAI Chat / OpenAI Responses text requests, with runtime status and effectiveness metrics
🧭 Configurable Navigation - Persist top-level console page order and visibility in settings and reuse it across browsers
💾 Runtime State Persistence - Persist auto strategy windows and circuit breaker state to the database
🔗 Site Management - Manage upstream relay platforms (New-API, One-API, One-Hub, Sub2API, etc.) with multi-account support, projected channels, auto-sync, and auto-checkin
🌍 Proxy Pool - Named proxy configurations with direct / system / pool / inherit modes and reference tracking across sites, accounts, and channels
🔁 Model Mapping - Global model name rewriting rules with exact, wildcard, and regex matching, priority ordering, and optional group scope
☁️ WebDAV Cloud Backup - Automated cloud backup via WebDAV with configurable schedule, remote file management, and one-click restore
🔑 API Credential Profiles - Reusable Base URL + API Key pairs with health verification probes and CLI config export
📤 CLI Config Export - Generate configuration snippets for Claude Code, Codex, Gemini CLI, Cherry Studio, and Kilo Code
🎨 Elegant UI - Clean and beautiful web management panel with dark mode, activity heatmap, share snapshot, and responsive mobile layout
🗄️ Multi-Database Support - Support for SQLite, MySQL, PostgreSQL with live migration between database types

🚀 Quick Start

🐳 Docker

Run directly:

docker run -d --name octopus \
  --restart unless-stopped \
  -p 8080:8080 \
  -v octopus-data:/app/data \
  -e OCTOPUS_AUTH_JWT_SECRET="replace-with-a-long-random-secret" \
  lingyuins/octopus:latest

Recommended on Windows Docker Desktop:

docker run -d --name octopus `
  --restart unless-stopped `
  -p 8080:8080 `
  -v octopus-data:/app/data `
  -e OCTOPUS_AUTH_JWT_SECRET="replace-with-a-long-random-secret" `
  lingyuins/octopus:latest

Or use docker compose:

services:
  octopus:
    image: lingyuins/octopus:latest
    container_name: octopus
    restart: unless-stopped
    ports:
      - "8080:8080"
    volumes:
      - ./data:/app/data
    environment:
      OCTOPUS_AUTH_JWT_SECRET: "replace-with-a-long-random-secret"

Then run:

docker compose up -d

Note: The official image runs as the non-root user octopus with UID/GID 1000. The docker run example above uses a named volume because it avoids most host-permission issues, especially on Windows Docker Desktop. If you bind-mount a host directory to /app/data, make sure that directory is writable by UID/GID 1000, otherwise startup will fail with permission denied when creating config.json or data.db.

The official Docker image rebuilds the frontend during image build and embeds the latest exported UI into the Go binary, so the container includes the matching management UI for that release.

🕐 Timezone: The image defaults to Asia/Shanghai. When running with docker run (not compose), pass -e TZ=Asia/Shanghai or your target IANA timezone (e.g. -e TZ=America/Los_Angeles). The server's log timestamps, statistics day boundaries, and frontend time display all depend on the container's timezone setting.

If you are upgrading from an older web build and still see stale frontend errors in the browser, clear the site data / service worker cache once after upgrading so the latest embedded assets are loaded.

📦 Download from Release

Download the binary for your platform from Releases, then run:

./octopus start

🛠️ Build from Source

Requirements:

Go 1.24.4
Node.js 20+
pnpm

# Clone the repository
git clone https://github.com/lingyuins/octopus.git
cd octopus
# Optional: bootstrap the initial admin via environment variables
export OCTOPUS_INITIAL_ADMIN_USERNAME="admin"
export OCTOPUS_INITIAL_ADMIN_PASSWORD="change-this-password-long"
# Optional but recommended: set a persistent JWT secret
export OCTOPUS_AUTH_JWT_SECRET="replace-with-a-long-random-secret"
# Start the backend service directly (API-only mode works even before frontend assets are built)
go run main.go start

If static/out/ already contains built frontend assets, the Go binary serves the management UI directly. Otherwise, Octopus still starts normally and exposes the API endpoints, but the management UI is unavailable until you build the frontend and place the exported assets under static/out/ before running go build / go run.

Build frontend assets for the embedded management UI

cd web && pnpm install && NEXT_PUBLIC_APP_VERSION="$(git describe --tags --always 2>/dev/null || printf 'dev')" pnpm build && cd ..
# Move frontend assets to the embed directory expected by the Go binary
mkdir -p static/out
mv web/out/* static/out/
# If Next.js exports an empty _not-found directory, add a placeholder before building Go
printf 'placeholder for go:embed\n' > static/out/_not-found/.keep
# Start the backend service with embedded UI assets available in the repository
go run main.go start

Development Mode

cd web && pnpm install && NEXT_PUBLIC_API_BASE_URL="http://127.0.0.1:8080" NEXT_PUBLIC_APP_VERSION="$(git describe --tags --always 2>/dev/null || printf 'dev')" pnpm dev
## Open a new terminal, optionally set initial admin credentials for automatic bootstrap
export OCTOPUS_INITIAL_ADMIN_USERNAME="admin"
export OCTOPUS_INITIAL_ADMIN_PASSWORD="change-this-password-long"
## Optional but recommended: set a persistent JWT secret
export OCTOPUS_AUTH_JWT_SECRET="replace-with-a-long-random-secret"
## Start the backend service
go run main.go start
## Access the frontend at
http://localhost:3000

🔐 Initial Admin Setup

On first launch, you can initialize the admin account in either of these ways:

Provide OCTOPUS_INITIAL_ADMIN_USERNAME and OCTOPUS_INITIAL_ADMIN_PASSWORD to bootstrap automatically at startup
Or open the Web UI on first visit and create the initial admin account through the guided setup wizard

⚠️ Security Notice: The initial admin password must be at least 12 characters long.

⚠️ Security Notice: If OCTOPUS_AUTH_JWT_SECRET or auth.jwt_secret is not configured, Octopus will generate an in-memory JWT secret at startup. Existing login tokens will become invalid after a restart.

👥 Admin Roles

The management API and embedded Web UI use three built-in roles:

admin: full access, including user management
editor: operational write access for channels, groups, settings, API keys, logs, alerts, and AI routing
viewer: read-only access to operational data

Role checks are enforced on the server side, using the currently stored role rather than trusting only the JWT claim.

📝 Configuration File

The configuration file is located at data/config.json by default and is automatically generated on first startup.

Complete Configuration Example:

{
  "server": {
    "host": "0.0.0.0",
    "port": 8080
  },
  "database": {
    "type": "sqlite",
    "path": "data/data.db"
  },
  "log": {
    "level": "info"
  },
  "auth": {
    "jwt_secret": "replace-with-a-long-random-secret"
  },
  "security": {
    "encryption_key": "replace-with-another-long-random-secret"
  }
}

Most operational knobs are not stored in config.json. Retry policy, circuit breaker thresholds, auto-strategy tuning, relay log retention, public API base URL, AI-route service settings, semantic-cache switches, WebDAV backup, proxy pool, and model mapping rules are managed at runtime from the Settings page / management API and stored in the database.

Configuration Options:

Option	Description	Default
`server.host`	Listen address	`0.0.0.0`
`server.port`	Server port	`8080`
`database.type`	Database type	`sqlite`
`database.path`	Database connection string	`data/data.db`
`log.level`	Log level	`info`
`auth.jwt_secret`	JWT signing secret	empty (ephemeral secret generated at startup if unset)
`security.encryption_key`	Encryption key for sensitive stored data (credential profiles, site passwords, etc.)	empty (falls back to JWT secret)
`relay.max_json_body_bytes`	Maximum JSON request body size	`67108864` (64 MB)
`relay.max_multipart_body_bytes`	Maximum multipart request body size	`67108864` (64 MB)

💡 Tip: Set OCTOPUS_AUTH_JWT_SECRET or auth.jwt_secret before running Octopus in production so login tokens stay valid across restarts.

Database Configuration:

Three database types are supported:

Type	`database.type`	`database.path` Format
SQLite	`sqlite`	`data/data.db`
MySQL	`mysql`	`user:password@tcp(host:port)/dbname`
PostgreSQL	`postgres`	`postgresql://user:password@host:port/dbname?sslmode=disable`

MySQL Configuration Example:

{
  "database": {
    "type": "mysql",
    "path": "root:password@tcp(127.0.0.1:3306)/octopus"
  }
}

PostgreSQL Configuration Example:

{
  "database": {
    "type": "postgres",
    "path": "postgresql://user:password@localhost:5432/octopus?sslmode=disable"
  }
}

💡 Tip: MySQL and PostgreSQL require manual database creation. The application will automatically create the table structure.

🌐 Environment Variables

All configuration options can be overridden via environment variables using the format OCTOPUS_ + configuration path (joined with _):

Environment Variable	Configuration Option
`OCTOPUS_SERVER_PORT`	`server.port`
`OCTOPUS_SERVER_HOST`	`server.host`
`OCTOPUS_DATABASE_TYPE`	`database.type`
`OCTOPUS_DATABASE_PATH`	`database.path`
`OCTOPUS_DATA_DIR`	Default directory for `config.json` and the SQLite DB when `database.path` is not explicitly set
`OCTOPUS_LOG_LEVEL`	`log.level`
`OCTOPUS_AUTH_JWT_SECRET`	`auth.jwt_secret`
`OCTOPUS_SECURITY_ENCRYPTION_KEY`	`security.encryption_key`
`OCTOPUS_INITIAL_ADMIN_USERNAME`	Bootstrap the initial admin username at startup
`OCTOPUS_INITIAL_ADMIN_PASSWORD`	Bootstrap the initial admin password at startup
`OCTOPUS_GITHUB_PAT`	For rate limiting when getting the latest version (optional)
`OCTOPUS_RELAY_MAX_SSE_EVENT_SIZE`	Maximum SSE event size (optional)

📸 Screenshots

Note: The screenshots below show the core console surfaces. Current builds keep the same visual system and navigation, with Model presented as Model Market and additional Analytics / Ops entries in the sidebar.

🖥️ Desktop

Dashboard	Channel Management	Group Management

Model Market	Logs	Settings

📱 Mobile

Home	Channel	Group	Model Market	Logs	Settings

📖 Documentation

🧭 Management Console Modules

The embedded management UI currently ships with these top-level modules:

Module	What it covers
Home	Version, runtime status, high-level summaries, trend chart, activity heatmap, and ranking panel
Hub	Remote site management with tabs: Sites (with balance chart & prediction), Check-in, Announcement, Redemption, Usage, Credential, and Site Channels
Channel	Upstream provider configuration, keys, headers, sync, latency probing, proxy mode, and request rewrite profiles
Group	Model routing, load-balancing strategies, sticky sessions, group test, AI route generation, endpoint provider, zashboard-style collapsible group list, and CC Switch deep link
Model Market	Model catalog, custom pricing, channel coverage, enabled key counts, latency, success metrics, and capabilities dual-view
Analytics	Cache overview, utilization, route health, latency distribution, evaluation, and share snapshot
Log	Relay request history, error details, token usage, and cost records
Alert	Alert rules, notification channels (webhook, Gotify, email, Telegram, Feishu, DingTalk, WeCom, ntfy), state, and history
Ops	Telemetry (hero metrics, P95 latency, provider health, prompt-cache analytics), quota, health, system, and audit trail
APIKey	API key create, edit, delete, supported-model allowlists, expiry, max-cost caps, RPM / TPM quotas, IP allowlists, and per-model quotas
Setting	Version/update info, appearance and nav preferences (order + visibility), runtime tuning, semantic cache, AI route services, API key defaults, database migration, WebDAV backup, site automation, backup/restore, and dangerous operations
User	Admin user management and roles

Additionally, the following features are accessible from the app shell toolbar or within other modules:

Feature	What it covers
Proxy Pool	Named proxy configurations CRUD, connectivity testing, and reference tree tracking
Model Mapping	Global model name rewriting rules with exact / wildcard / regex matching, priority, and group scope
API Credential Profiles	Reusable Base URL + API Key pairs with health verification and CLI export

📡 Channel Management

Channels are the basic configuration units for connecting to LLM providers.

Channel Templates:

The UI provides 9 built-in channel templates for quick creation: OpenAI, OpenAI Responses, Anthropic, Gemini, DeepSeek, OpenRouter, SiliconFlow, Volcengine, and MiMo.

Base URL Guide:

The program automatically appends API paths based on channel type. You only need to provide the base URL:

Channel Type	Auto-appended Path	Base URL	Full Request URL Example
OpenAI Chat	`/chat/completions`	`https://api.openai.com/v1`	`https://api.openai.com/v1/chat/completions`
OpenAI Responses	`/responses`	`https://api.openai.com/v1`	`https://api.openai.com/v1/responses`
OpenAI Embeddings	`/embeddings`	`https://api.openai.com/v1`	`https://api.openai.com/v1/embeddings`
OpenAI Images	`/images/generations`, `/images/edits`, `/images/variations`	`https://api.openai.com/v1`	`https://api.openai.com/v1/images/generations`
Anthropic	`/messages`	`https://api.anthropic.com/v1`	`https://api.anthropic.com/v1/messages`
Gemini	`/models/:model:generateContent`	`https://generativelanguage.googleapis.com/v1beta`	`https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent`
Volcengine	`/responses`	`https://ark.cn-beijing.volces.com/api/v3`	`https://ark.cn-beijing.volces.com/api/v3/responses`
MiMo Chat	`/chat/completions`	`https://api.xiaomimimo.com/v1`	`https://api.xiaomimimo.com/v1/chat/completions`

💡 Tip: Base URLs now support Auto detect and Custom. Auto detect appends the version suffix based on the channel type, while Custom keeps the URL exactly as you entered it.

Proxy Mode:

Each channel can configure a proxy mode:

Mode	Description
`direct`	No proxy, connect directly
`system`	Use system proxy settings
`pool`	Select from the named proxy pool
`inherit`	Inherit proxy from the parent site or account

Request Rewrite Profiles:

Per-channel request rewriting for upstream compatibility:

Profile	Description
`preserve`	No body rewrite — forward as-is
`openai_chat_compat`	Strip incompatible fields for standard OpenAI Chat format

Parameter Override:

Each channel supports a param_override JSON configuration that injects or overrides specific parameters in outbound requests to the upstream provider, enabling per-channel parameter customization without modifying the client request.

Header and message strategies:

Strategy	Options	Description
Header Profile	`none`, `codex`	Codex-specific header shaping
Tool Role	`keep`, `stringify_to_user`	How to handle tool role messages
System Message	`keep`, `merge`	How to handle system messages

🌐 Public Relay Endpoints

The public relay API supports both OpenAI-style and Anthropic-style clients:

OpenAI-style clients: Authorization: Bearer sk-octopus-...
Anthropic-style clients: x-api-key: sk-octopus-...

Category	Paths	Notes
OpenAI-compatible LLM	`/v1/chat/completions`, `/v1/responses`, `/v1/embeddings`, `/v1/models`	JSON request / response
Anthropic-compatible LLM	`/v1/messages`	Anthropic-style request / response
JSON media / utility	`/v1/images/generations`, `/v1/audio/speech`, `/v1/videos/generations`, `/v1/music/generations`, `/v1/search`, `/v1/rerank`, `/v1/moderations`	Uses the same group / retry / circuit-breaker pipeline
Multipart media	`/v1/images/edits`, `/v1/images/variations`, `/v1/audio/transcriptions`	Multipart upload forwarding

JSON media endpoints can also proxy upstream SSE streams when the provider supports stream=true.

Semantic cache is currently evaluated for non-streaming and streaming OpenAI Chat and OpenAI Responses text requests (streaming cache hits replay from the SSE session buffer). Anthropic, embeddings, and media / utility requests bypass the cache and continue through the normal relay flow.

Zen Direct Model Routing:

Requests with model name prefixed zen/<model> bypass group model mapping and route directly to the upstream model. Octopus performs smart channel-type detection based on the model name (e.g., Claude → Anthropic, Gemini → Gemini, GPT → OpenAI).

Response ID Affinity:

For the OpenAI Responses API, follow-up requests referencing the same response ID are automatically routed to the same upstream channel to maintain conversation continuity.

Model Mapping:

Global model name rewriting rules are applied in the relay pipeline before group resolution. Rules support exact, wildcard (glob), and regex matching with priority ordering and optional group scope.

🔍 Model Discovery & Capabilities

Octopus exposes multiple levels of model visibility:

`/v1/models` — Flat Compatible Model List

Returns all model names that have at least one enabled channel. Compatible with OpenAI SDKs.

This is the broadest view — if a model appears here, Octopus has a channel that declares it.

`/v1/models?endpoint=<type>` — Endpoint-Filtered List

Narrows the list to models whose declared endpoint type matches the filter:

?endpoint=chat — conversation models (chat / responses / messages / deepseek / mimo)
?endpoint=embeddings — embedding models
?endpoint=image_generation — image models
?endpoint=music_generation — music models
… and so on for audio_speech, audio_transcription, video_generation, search, rerank, moderations

When endpoint is omitted or set to *, all models are returned.

Boundaries between some endpoints are not absolute. Models from the conversation family (chat, responses, messages, deepseek, mimo) are visible to one another through the endpoint filter because Octopus can bridge these formats transparently.

`GET /api/v1/model/capabilities` — Declared Capability Table (Management API)

A management-only endpoint that returns the aggregated capability view of every routable model:

{
  "code": 200,
  "message": "success",
  "data": [
    {
      "name": "gpt-4o",
      "endpoints": ["chat"],
      "conversation": true,
      "available": true
    },
    {
      "name": "music-2.6",
      "endpoints": ["music_generation"],
      "conversation": false,
      "available": true
    }
  ]
}

Field	Meaning
`name`	Model name as exposed to clients
`endpoints`	Endpoint types the model declares (deduplicated, sorted)
`conversation`	Whether the model belongs to the conversation family
`available`	Whether the model has at least one enabled channel

This is the declared capability — what your Group configuration says. The actual routable capability may be narrower; see * group behaviour below.

`*` Group Semantics

A group with endpoint type * (EndpointTypeAll) is a universal pass: it can be selected by any endpoint type, including chat, embeddings, image_generation, etc.

However, universal selection does not mean every item in the group actually supports the endpoint. For non-conversation endpoints (image / video / music / audio / search / rerank / moderation), the relay layer now filters * group items before the balancer:

Only items whose channel type or model name hint at support for the requested endpoint are kept.
If no items survive filtering, the request returns 404 model not found instead of blindly trying incompatible channels.
Conversation endpoints (chat, responses, messages, deepseek, mimo) are not affected by this filtering.

Tip: When you see a model in /v1/models or /api/v1/model/capabilities but it still returns model not found for a specific endpoint, check whether the * group's items actually support that endpoint — the relay narrowing may have filtered them all out.

📁 Group Management

Groups aggregate multiple channels into a unified external model name.

Core Concepts:

Group name is the model name exposed by the program
When calling the API, set the model parameter to the group name
First Token Timeout: unit in seconds, only effective for streaming responses, 0 means no limit
Session Keep Time: unit in seconds, keeps using the same channel for the same API key + model within the configured session window, 0 means disabled
Condition (JSON): optional AND rules currently evaluated in the main LLM relay path; the built-in request context currently includes model, api_key_id, and hour
Endpoint Provider: provider-aware request rewriting (openai, deepseek, mimo, siliconflow, newapi) that strips incompatible reasoning fields (e.g., reasoning_content, reasoning_signature) from messages for upstream compatibility

Load Balancing Modes:

Mode	Description
🔄 Round Robin	Cycles through channels sequentially for each request
🎲 Random	Randomly selects an available channel for each request
🛡️ Failover	Prioritizes high-priority channels, switches to lower priority only on failure
⚖️ Weighted	Orders candidates by weight from high to low, then tries them in that order
🤖 Auto	Explores under-sampled candidates first, then prefers the candidate with the best success rate inside the configured window

Auto Strategy Defaults:

Minimum samples: 10
Time window: 300 seconds
Sliding window size: 100 records per channel-model pair
Latency weight: 30
Before a candidate reaches the minimum sample count, Octopus prioritizes exploration
After candidates are explored, Octopus sorts by success rate, then uses sample count, weight, priority, and latency tuning as tie-breakers
Auto-strategy windows are restored from the database at startup and saved periodically plus on graceful shutdown

AI Routing Behavior:

Clicking AI Route on the route page sends all models to AI and generates the full routing table in batch
Existing groups with the same name only receive missing route items; existing groups are not cleared or replaced
Clicking AI Fill Current Group in the edit dialog sends all models to AI and appends only the matched route items to that group
The setting previously named AI route target group now acts as the default target group for the single-group compatibility flow only
AI route tasks are persistent with heartbeat, progress tracking, batch management, and interruption recovery

CC Switch Integration:

The group toolbar includes a CC Switch deep link generator that creates provider import links for 5 target apps: Claude Code, Codex, Gemini, OpenCode, and OpenClaw. For Claude Code, it supports mapping Haiku / Sonnet / Opus models to specific route groups.

💡 Example: Create a group named gpt-4o, add multiple providers' GPT-4o channels to it, then access all channels via a unified model: gpt-4o.

💎 Model Market & Pricing

The Model route is a model market view with a dual-tab interface: Market (pricing and coverage) and Capabilities (endpoint support declarations).

Market tab data merged on each card:

Custom or synced pricing from the LLM price catalog
Channel coverage and enabled key counts from channel-model relationships
Average latency and success / failure counts from recorded model stats

Summary metrics:

Metric	Meaning
Models	Number of currently visible model cards
Coverage	Total channel-to-model coverage count in the current result set
Unique Channels	Distinct channels represented by the visible cards
Average Latency	Weighted average latency derived from model request stats

Capabilities tab:

The Capabilities panel shows per-model endpoint support declarations, conversation flag, availability status, and auto-endpoint detection indicators. Models can be searched and filtered by name with status badges (Active, Down, Non-conversation).

Data Sources:

The system periodically syncs model pricing data from models.dev
When creating or syncing channels, if a model is not yet in the local catalog, Octopus automatically creates a local model-price record so the price can still be maintained manually
Manual creation of models that exist in models.dev is also supported for custom pricing

Price Priority:

Priority	Source	Description
🥇 High	This Page	Prices set by user in the model market page
🥈 Low	models.dev	Auto-synced default prices

💡 Tip: To override a model's default price, simply set a custom price for it in the model market page.

Operational actions preserved on the page:

Create a custom model price record
Edit input / output / cache prices for an existing model
Delete a custom model entry
Refresh upstream pricing from the page header
Keep the scheduled price refresh policy in the Settings LLM Price card

📈 Analytics

The Analytics module is a read-oriented operations view with five tabs:

Tab	What it shows
Cache	Semantic cache effectiveness and provider-side prompt-cache analytics (cache rate, reuse ratio, estimated cost savings per provider)
Utilization	Provider, model, and API key breakdowns for the selected time range
Route Health	Health score, enabled / disabled item counts, and recent failure pressure for each group
Evaluation	Group readiness, AI route progress, group test progress, and semantic-cache effectiveness
Latency	Request latency metrics (Avg, P50, P95, P99), first-token-user-time (FTUT) metrics, and latency distribution histogram

Time ranges: 1d, 7d, 30d, 90d, ytd, and all

The overview metrics API still exists as /api/v1/analytics/overview, but the primary UI entry point for those summary cards is now the Home page. Home also carries an independent 7d / 30d / 90d overview-range switch, plus a daily hero summary, trend chart, GitHub-style activity heatmap, and ranking panel.

The Evaluation tab is intentionally lightweight: it acts as an entry point into group testing, AI routing, and semantic-cache tuning instead of duplicating those full workflows.

Share Snapshot:

The Analytics page includes a Share button that generates a visual PNG snapshot of the current analytics state, which can be downloaded or copied to the clipboard. The snapshot includes key stats (requests, tokens, cost, providers, cache hit rate) and a timestamp.

🛠️ Ops

The Ops module focuses on runtime posture and operational diagnostics:

Tab	What it shows
Telemetry	Hero metrics (uptime, total requests, avg latency, error rate, active connections, memory usage), P95 latency, throughput RPS, database health, session & quota activity, semantic cache snapshot, provider health table with success rates
Quota	API key limit posture across RPM, TPM, max-cost, and per-model quota settings
Health	Database reachability, cache readiness, task-runtime sanity, recent error count, and failing groups
System	Build metadata, database type, public API base URL, proxy, retention intervals, AI route mode, and AI route services
Audit	Paginated audit history for management-side write operations

Provider Prompt Cache Analytics:

The Telemetry tab includes provider-side prompt cache monitoring, tracking upstream provider prompt caching effectiveness: cache rate, cache reuse ratio, cache read / write tokens, estimated cost savings per channel, and a 24-hour cache trend chart. This is separate from the semantic cache.

Audit scope:

Covers selected management write routes such as channel / group / model / setting / API key / alert / user mutations, AI route generation, log clearing, price refresh, import, and self-update
Does not record public /v1/... relay traffic

⚙️ Settings

Global system configuration.

Statistics Save Interval (minutes):

Since the program handles numerous statistics, writing to the database on every request would impact read/write performance. The program uses this strategy:

Statistics are first stored in memory
Periodically batch-written to the database at the configured interval
Relay balancer runtime state uses the same periodic persistence pattern

Runtime State Persistence:

Auto strategy windows are loaded from the database on startup
Circuit breaker state is loaded from the database on startup
Both are saved periodically using the same interval as statistics persistence
Both are also saved during graceful shutdown

Key settings cards in the current UI:

Card	Purpose
Info	Current version, latest release lookup, cache-mismatch detection, and in-place self-update entry with version mismatch notification
Appearance	Theme, locale, alert language, drag-and-drop top-level navigation order, and per-page visibility toggles
System	Public API base URL, proxy URL, CORS allowlist (tag-style management), and stats persistence interval
Account	Login-session/account preferences and application timezone selection (10 time zones)
Semantic Cache	Enablement, TTL, similarity threshold, max entries, embedding base URL / API key / model / timeout
AI Route	Default compatibility group, timeout, parallelism, and service-pool configuration
API Key	API key creation defaults and quota-related controls
Retry / Auto Strategy / Circuit Breaker	Relay retry and candidate-selection tuning
Log / LLM Price / LLM Sync	Retention (time-based and count-based), price refresh cadence, and upstream model synchronization
Site Automation	Auto-sync interval, auto-checkin interval, and manual sync / checkin triggers for remote sites
WebDAV Backup	WebDAV cloud backup configuration: connection settings, auto-backup interval, max backups retention, manual trigger, remote file listing, restore, and delete
Backup	Database export, import, and live database migration between SQLite / MySQL / PostgreSQL with connection testing and per-table row count results
Route Group Danger	Delete all route groups with explicit confirmation

Semantic Cache Scope:

Applies to non-streaming and streaming OpenAI Chat and OpenAI Responses text requests
Streaming cache hits replay from the SSE session buffer with stable stream-session recovery
Namespaces cache entries by api_key_id + endpoint_family + requested_model
If the embedding client is not fully configured, or embedding lookup / store fails, Octopus bypasses the cache and relays the request normally
Runtime state and effectiveness are visible in both Analytics -> Evaluation and Ops -> Telemetry
Cache entries are preserved across unchanged runtime config refreshes

Database Live Migration:

The Backup settings card includes a live database migration feature beyond simple export/import:

Target database types: SQLite, MySQL, PostgreSQL
Connection testing before migration
Optional inclusion of logs and stats in migration
Migration result display with per-table row counts
Post-migration restart reminder (the backend continues using the old database until restart)

Dangerous Operation in Settings:

The Settings page provides Delete All Route Groups
The action requires a second confirmation before execution
It deletes all groups and group items, then resets the default target group for single-group AI routing to 0 to avoid dangling references

Settings Card Order:

The Settings page supports drag-and-drop reordering of its 14+ card sections, with order persisted to local storage. A "Reset to Default" button restores the original order.

⚠️ Important: When exiting the program, use proper shutdown methods (like Ctrl+C or sending SIGTERM signal) to ensure in-memory statistics are correctly written to the database. Do NOT use kill -9 or other forced termination methods, as this may result in statistics data loss.

🔗 Site Management

The Site module manages upstream relay platforms as a first-class entity, distinct from the Hub (remote site connections). Sites represent platforms like New-API, One-API, One-Hub, Done-Hub, Sub2API, AnyRouter, OpenAI, Claude, and Gemini.

Features:

Multi-account support per site with username/password, access_token, or api_key credentials
Auto-sync of channels, tokens, and models at configurable intervals
Auto-checkin with configurable intervals and random time windows
Projected channels: automatically creates local Octopus channels from site account groups with per-group key management, model routing, and history tracking
Route type inference per model (openai_chat, openai_response, anthropic, gemini, volcengine, embedding)
Manual model add / delete and route type override
Source key and projected key management with model history tracking
Bulk import from AllAPIHub and MetAPI formats
Proxy pool integration with per-site, per-account, and per-channel proxy selection

🌍 Proxy Pool

A shared proxy configuration pool accessible from the app shell toolbar:

Named proxy configurations with URL, scheme (SOCKS5 / HTTP / HTTPS), enable/disable, and remarks
4 proxy modes: direct, system, pool, inherit
Proxy connectivity testing against a configurable test URL
Reference tree showing which sites, site accounts, managed channels, and channels use each proxy
Jump-to-reference navigation that deep-links to the referencing entity
Deletion protection when a proxy has active references

🔁 Model Mapping

Global model name rewriting rules applied in the relay pipeline before group resolution:

Match types: Exact, Wildcard (glob), and Regex
Target model: the rewritten model name
Priority ordering: rules are evaluated in priority order
Group scope: optionally apply only to a specific group
Enable/disable toggle per rule

☁️ WebDAV Cloud Backup

Automated cloud backup via WebDAV with full lifecycle management:

Configurable base URL, credentials, remote path, auto-backup interval (default 6 hours), and max backups retention
Connection testing before enabling
Manual backup trigger
Remote backup file listing with size info
One-click restore from any remote backup
Delete remote backups
Included in the Settings page as a dedicated card

🔑 API Credential Profiles & CLI Export

Reusable API credential profiles store Base URL + API Key pairs for quick access:

Health verification probes: text_gen, models_list, tool_calling, structured_output
Health status tracking per credential
Encryption at rest via security.encryption_key
Tags and notes for organization

CLI Config Export:

Generate ready-to-use configuration snippets for 5 client tools:

Tool	Format
Claude Code	Environment variables for `~/.claude/settings.json`
Codex	Environment variables for `~/.codex/auth.json` and `config.toml`
Gemini CLI	Environment variables
Cherry Studio	JSON provider import configuration
Kilo Code	JSON settings block

🚨 Alerts & Notifications

Alert rules monitor system health and trigger notifications:

Alert rule types: Error rate, cost threshold, quota exceeded, and channel down.

Notification channels:

Channel	Configuration
Webhook	URL, method, headers
Gotify	Server URL, app token
Email	SMTP settings, recipients
Telegram	Bot token, chat ID
Feishu	Webhook key
DingTalk	Robot access token, optional HMAC-SHA256 signing secret
WeCom	Group robot key
ntfy	Topic URL, optional access token

Alert state and history are tracked per rule, with configurable evaluation intervals.

🔌 Client Integration

OpenAI SDK

from openai import OpenAI
import os

client = OpenAI(   
    base_url="http://127.0.0.1:8080/v1",   
    api_key="sk-octopus-P48ROljwJmWBYVARjwQM8Nkiezlg7WOrXXOWDYY8TI5p9Mzg", 
)
completion = client.chat.completions.create(
    model="octopus-openai",  # Use the correct group name
    messages = [
        {"role": "user", "content": "Hello"},
    ],
)
print(completion.choices[0].message.content)

Claude Code

Edit ~/.claude/settings.json

{
  "env": {
    "ANTHROPIC_BASE_URL": "http://127.0.0.1:8080",
    "ANTHROPIC_AUTH_TOKEN": "sk-octopus-P48ROljwJmWBYVARjwQM8Nkiezlg7WOrXXOWDYY8TI5p9Mzg",
    "API_TIMEOUT_MS": "3000000",
    "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
    "ANTHROPIC_MODEL": "octopus-sonnet-4-5",
    "ANTHROPIC_SMALL_FAST_MODEL": "octopus-haiku-4-5",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "octopus-sonnet-4-5",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "octopus-sonnet-4-5",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "octopus-haiku-4-5"
  }
}

Codex

Edit ~/.codex/config.toml

model = "octopus-codex" # Use the correct group name

model_provider = "octopus"

[model_providers.octopus]
name = "octopus"
base_url = "http://127.0.0.1:8080/v1"

Edit ~/.codex/auth.json

{
  "OPENAI_API_KEY": "sk-octopus-P48ROljwJmWBYVARjwQM8Nkiezlg7WOrXXOWDYY8TI5p9Mzg"
}

CLI Config Export

For other clients (Gemini CLI, Cherry Studio, Kilo Code), use the built-in CLI Export feature from the API Credential Profiles panel in the management console to generate ready-to-use configuration snippets.

🏗️ Architecture

Octopus follows a clean layered architecture in Go:

cmd/                    # Entry points (Cobra CLI)
internal/
├── conf/               # Configuration loading & build metadata
├── client/             # HTTP client utilities
├── db/                 # Database connection & migrations (SQLite/MySQL/PostgreSQL)
│   └── migrate/        # Versioned schema migrations (001-011)
├── model/              # Domain types (Channel, Group, APIKey, User, Site, ProxyConfiguration, ModelMapping, …)
├── op/                 # Business logic operations split by domain
│   ├── airoute/        # AI route generation, progress tracking, service pool, and compatibility helpers
│   ├── alert/          # Alert rule evaluation and notification dispatch
│   ├── analytics/      # Dashboard, utilization, route-health, evaluation, and latency queries
│   ├── apikey/         # API key CRUD and validation
│   ├── audit/          # Audit log persistence
│   ├── backup/         # Database export/import, WebDAV cloud backup scheduler
│   ├── cacheusage/     # Cache usage tracking
│   ├── channel/        # Channel CRUD, sync, grouping, keys, managed channel projection, and base URL helpers
│   ├── credential/     # API credential profile management with encryption
│   ├── dbmigration/    # Live database migration between SQLite/MySQL/PostgreSQL
│   ├── group/          # Route-group CRUD, auto-grouping, group items, tests, and cache-backed lookups
│   ├── llm/            # LLM price catalog operations
│   ├── modelmapping/   # Model mapping rule management
│   ├── navorder/       # Navigation order and visibility persistence
│   ├── ops/            # Ops dashboard data aggregation (telemetry, quota, health)
│   ├── ratelimitstore/ # RPM/TPM rate limit state
│   ├── relaylog/       # Relay log persistence with async flush worker
│   ├── remotesite/     # Remote Hub site operations (balance, checkin, announcements, usage, tokens, redemption)
│   ├── setting/        # Settings CRUD and validation
│   ├── stats/          # Request statistics aggregation, cache, and site-model backfill
│   └── user/           # User management and authentication
├── relay/              # Core relay pipeline
│   ├── balancer/       # Load balancing strategies (RoundRobin, Random, Failover, Weighted, Auto)
│   └── condition/      # Request condition evaluation
├── server/             # HTTP layer (Gin)
│   ├── auth/           # JWT auth & permissions
│   ├── handlers/       # Route handlers (one per resource)
│   ├── middleware/     # Auth, RBAC, CORS, rate-limit, audit, security, IP allowlist, …
│   ├── resp/           # Response envelope helpers
│   └── router/         # Route registration system
├── task/               # Background periodic jobs
├── transformer/        # Protocol adapters
│   ├── inbound/        # Client→Internal (OpenAI, Anthropic)
│   ├── outbound/       # Internal→Upstream (OpenAI, Anthropic, Gemini, Volcengine, MiMo)
│   ├── rewrite/        # Request normalization with configurable profiles
│   └── model/          # Shared transformer types & interfaces
├── hub/                # Remote site adapter interface, registry, HTTP client, and platform-specific adapters
├── helper/             # Cross-cutting helpers (AI route, channel/group probes, price, notify)
├── price/              # LLM price catalog (models.dev sync)
├── update/             # Self-update mechanism
└── utils/              # Utilities (cache, ratelimit, semantic_cache, tokenizer, crypto, …)

Relay data flow:

Client Request
    ↓
Model Mapping (global name rewriting)
    ↓
inbound.TransformRequest (raw → internal format)
    ↓
outbound.TransformRequest (internal → upstream format)
    ↓
http.Do (forward to upstream provider)
    ↓
outbound.TransformResponse (upstream response → internal format)
    ↓
inbound.TransformResponse (internal → client format)
    ↓
Client Response

For streaming, the same pipeline processes each SSE event through TransformStream.

Hub adapters:

The Hub remote site management uses an adapter-based architecture with 7 registered site adapters:

Adapter	Site Type
`common`	`new-api` (fallback for One API / New API family)
`octopus`	`octopus` (self-aware adapter)
`aihubmix`	`aihubmix`
`axonhub`	`axonhub`
`claudecodehub`	`claude-code-hub`
`ldoh`	`ldoh`
`sub2api`	`sub2api`

Each adapter implements the 15-method SiteAdapter interface covering user info, check-in, models, pricing, tokens, channels, announcements, status, redemption, and usage logs.

Frontend (Next.js 16 App Router):

web/src/
├── api/               # API client & endpoint hooks (TanStack Query)
├── app/               # Next.js App Router pages
├── components/
│   ├── modules/       # Domain modules (channel, group, apikey, remote-site, site, proxy-pool, model-mapping, credential, …)
│   ├── ui/            # UI primitives (Radix-based)
│   ├── common/        # Shared components
│   └── nature/        # Animated backgrounds & effects
├── hooks/             # Custom hooks
├── lib/               # Utilities, i18n, logger, time zone helpers
├── provider/          # React context providers
├── route/             # Lazy-loaded route config
└── stores/            # Zustand client state

🕐 Timezone Architecture

Octopus involves three independent timezone layers:

Layer	Controlled By	Affects
Container timezone	`ENV TZ` / `-e TZ=`	Server log timestamps, `time.Now()` return value
Stats timezone	Admin UI → `stats_timezone_offset`	Which date hourly/daily statistics roll into
Frontend display timezone	Admin UI → user preference (10 time zones)	How all timestamps appear on pages

The three layers are independent: the container timezone affects the server runtime, the stats timezone affects data aggregation, and the frontend timezone only changes how users see time text.

🔐 Security

JWT Authentication: Management API uses JWT tokens with configurable expiry. Login rate limiting protects against brute-force attacks (configurable window and max failed attempts).
Role-Based Access Control: Server-side RBAC with admin, editor, viewer roles, reloaded from DB each request.
API Key Security: API keys (sk-octopus-...) support model allowlists, IP/CIDR allowlists, expiry, max-cost caps, RPM/TPM limits, and per-model quotas.
Encryption at Rest: Sensitive stored data (credential profiles, site passwords) is encrypted via AES-256-GCM using security.encryption_key.
CORS Management: Tag-style CORS allowlist manager with * for all, specific domains, or deny-all (empty).
Viewer Domain Masking: Hub-related management data masks domains for viewer accounts across sites, remote sites, credentials, channels, and URL settings.

🤝 Acknowledgments

🙏 looplj/axonhub - The LLM API adaptation module in this project is directly derived from this repository
📊 sst/models.dev - AI model database providing model pricing data
💡 qixing-jk/all-api-hub - The Hub concept and feature design inspiration
🛠️ Hureru/octopus - The original Hub implementation

Name		Name	Last commit message	Last commit date
.github		.github
cmd		cmd
internal		internal
scripts		scripts
static		static
web		web
.commitlintrc.json		.commitlintrc.json
.dockerignore		.dockerignore
.editorconfig		.editorconfig
.gitattributes		.gitattributes
.gitignore		.gitignore
.golangci.yml		.golangci.yml
.lintstagedrc.json		.lintstagedrc.json
1.cmd		1.cmd
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
README_zh.md		README_zh.md
docker-compose.yml		docker-compose.yml
go.mod		go.mod
go.sum		go.sum
main.go		main.go
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml

Latest commit

History

Folders and files

Repository files navigation