Skip to content
BranchesTags

Repository files navigation

AIMUX

AI Multiplexer

Multi-Model AI Orchestrator for Developer Workflows

AIMUX is a terminal-native multi-model AI orchestrator that classifies coding tasks, selects the cheapest viable model, launches the right CLI, tracks cost, and fails over across providers when rate limits or tool failures happen.

tests License: MIT Python: 3.10+ Platform: Linux · macOS · WSL Status: Alpha Claude Code OpenCode

Quickstart · Features · Architecture · Configuration · Security · Report a Bug

Status: Alpha. AIMUX is usable for experimentation, but routing, workflow classification, and upstream CLI output parsing are still under active development. Expect rough edges and check run metadata before trusting automation.

AIMUX splash banner

Table of Contents

Why AIMUX

AI coding assistants are powerful, but model choice is still too manual. Many developer tasks do not need the most expensive model: summaries, straightforward reviews, small fixes, prompt cleanup, and repository scans can often run on cheaper models. Other tasks need stronger reasoning, a critic pass, or a fallback when the provider hits a rate limit.

AIMUX sits in front of Claude Code and OpenCode as a local CLI orchestrator. It does not proxy API calls and it does not read credentials. It shells out to the tools you already authenticated, chooses models per workflow, records what happened, and keeps local cooldown memory when a provider fails.

AIMUX is built for developers looking for an AI multiplexer, Claude Code router, OpenCode router, LLM routing layer, AI coding agent orchestrator, terminal AI assistant, provider fallback system, model cooldown manager, and developer AI cost optimization tool.

Features

Feature What it does
Workflow classification Detects review, debug, implement, architecture, summarize, and plan tasks.
Cost-aware routing Scores enabled models by capability, priority, latency, cost tier, context window, and active cooldowns.
Claude Code and OpenCode launchers Runs upstream CLIs directly so authentication and permissions stay inside those tools.
Provider fallback Applies cooldowns for rate limits, quota exhaustion, usage limits, auth errors, invalid models, and unavailable models.
Prompt refinement Optionally rewrites vague requests into clearer execution prompts using a cheap model or local fallback.
Project context Adds git status, repository metadata, project context, and previous session information to prompts.
Multi-step orchestration Supports single-run, plan-then-execute, critic, consensus, and auto recommendation modes.
Readable transcripts Stores prompts, route decisions, commands, attempts, transcripts, and run metadata under .aimux/runs/.
Cost telemetry Logs steps and plans to .aimux/sessions/steps.jsonl and .aimux/sessions/plans.jsonl.
Local-first security Never reads API keys, OAuth tokens, or provider credential files.

Quickstart

Requirements

  • Python 3.10 or newer
  • Linux, macOS, or WSL
  • At least one authenticated upstream tool: claude or opencode
  • POSIX-like terminal support for PTY-based streaming

Install

git clone https://github.com/jtsr-hub/AIMUX.git
cd AIMUX
./install.sh

First Run

aimux                                # interactive TUI
aimux "review my latest changes"     # one-shot task
aimux doctor                         # environment and config checks

Setup Guide

AIMUX uses your existing upstream CLI logins. You authenticate Claude Code, OpenCode, OpenAI, DeepSeek, or other providers in those tools; AIMUX only launches them and reads their normal command output.

1. Authenticate Upstream Tools

Use whichever tools/providers you already rely on:

claude                         # starts Claude Code login/setup if needed
opencode auth login            # opens OpenCode provider login flow
opencode providers             # shows configured OpenCode providers

AIMUX does not read OAuth files, token files, API keys, or .env files. Your Claude/OpenAI OAuth and DeepSeek token remain managed by Claude Code/OpenCode or your shell environment.

2. Run The Setup Wizard

Run the standalone setup wizard:

aimux setup

Or start the interactive TUI and open the same wizard with /setup:

aimux
/setup

The wizard walks through auth status, model discovery, model selection, routing preferences, and diagnostics.

3. Bootstrap Local Configuration

Verify the environment:

aimux doctor

On first run, AIMUX creates local state under .aimux/. Your personal .aimux/config.yaml, caches, sessions, runs, and contexts are gitignored.

4. Discover And Select Models

Inside AIMUX:

/models refresh
/models import-discovered
/models picker
/models doctor

Use /models picker to enable the models you want AIMUX to route to. Use /routing cheap, /routing balanced, or /routing strong to control how aggressively AIMUX optimizes cost vs quality.

5. Verify Before Real Work

aimux --dry-run "review my latest changes"
aimux "summarize this repository"

If routing looks wrong, inspect .aimux/runs/<latest>/run.json and .aimux/runs/<latest>/routing_attempts.json.

Detailed setup documentation: docs/setup.md.

Usage

# Preview routing and prompt assembly without launching a model
aimux --dry-run "fix the failing auth tests"

# Prefer a routing strategy
aimux --routing cheap "summarize this repository"
aimux --routing strong "design a migration-safe database plan"

# Force a specific model while preserving fallback safety
aimux --model anthropic-sonnet "debug the startup crash"

# Run without the Textual chrome
aimux --no-tui

Useful interactive commands:

Command Purpose
/help Show commands.
/models List configured models.
/models refresh Discover models from available tools.
/models picker Enable, disable, and prioritize models.
/routing cheap|balanced|strong|auto Set routing strategy.
/mode single|plan-then-execute|critic|consensus|auto Set orchestration mode.
/cooldowns Show active provider/model cooldowns.
/c Continue the last run.

Architecture

AIMUX is layered so routing, failure handling, execution, and local state stay separable.

flowchart TD
    Developer([Developer]):::actor

    L1["<b>1 · Interface</b><br/>CLI · Textual TUI · Slash commands"]:::blue
    L2["<b>2 · Workflow Understanding</b><br/>Classifier · task type, risk, complexity"]:::violet
    L3["<b>3 · Local Context Retrieval</b><br/>Git status &amp; diff · repo metadata · project memory · recent sessions"]:::green
    L4["<b>4 · Prompt Improvement</b><br/>Refiner · workflow templates · preview, edit, approve"]:::amber
    L5["<b>5 · Orchestration</b><br/>Single · plan-then-execute · critic · consensus"]:::pink
    L6["<b>6 · Routing &amp; Safety</b><br/>Model registry · capability/cost scoring · cooldowns · failure classifier"]:::red
    L7["<b>7 · Execution Adapter</b><br/>Launch spec · PTY launcher · JSON stream launcher · one-shot runner"]:::indigo
    L8["<b>8 · Upstream AI Tool</b><br/>Claude Code CLI · OpenCode CLI · future adapters"]:::gray

    State[("<b>Local State</b> (.aimux/)<br/>config.yaml · runs · sessions · cache · contexts")]:::lime

    Developer --> L1 --> L2 --> L3 --> L4 --> L5 --> L6 --> L7 --> L8

    State -. config .-> L1
    State -. repository memory .-> L3
    State -. cache &amp; cooldowns .-> L6
    State -. transcripts &amp; telemetry .-> L7

    classDef actor fill:#0f172a,stroke:#38bdf8,color:#ffffff,stroke-width:2px;
    classDef blue fill:#2563eb,stroke:#111827,color:#ffffff,stroke-width:1px;
    classDef violet fill:#7c3aed,stroke:#111827,color:#ffffff,stroke-width:1px;
    classDef green fill:#16a34a,stroke:#111827,color:#ffffff,stroke-width:1px;
    classDef amber fill:#d97706,stroke:#111827,color:#ffffff,stroke-width:2px;
    classDef pink fill:#db2777,stroke:#111827,color:#ffffff,stroke-width:1px;
    classDef red fill:#dc2626,stroke:#111827,color:#ffffff,stroke-width:1px;
    classDef indigo fill:#4f46e5,stroke:#111827,color:#ffffff,stroke-width:1px;
    classDef gray fill:#374151,stroke:#111827,color:#ffffff,stroke-width:1px;
    classDef lime fill:#65a30d,stroke:#111827,color:#ffffff,stroke-width:2px;
Loading

See docs/architecture.md for the layer-by-layer architecture, module map, data flow, and extension points.

The context retrieval tier is RAG-like, but local and file-based: AIMUX retrieves git state, repository metadata, project memory, and previous session summaries, then injects that context into the improved prompt. It does not currently run a vector database, embedding index, or semantic search service.

What It Saves

Computed from AIMUX's pricing model against a 20-task synthetic mix: 8 reviews, 5 debugs, 4 summaries, and 3 architecture tasks.

Scenario Total cost over 14 days Latency
Always Opus $5.85 ~7s avg
AIMUX auto-mode $2.74 ~5s avg

That synthetic mix shows a 53% cost reduction. Treat this as a reproducible benchmark, not a universal promise. Real savings depend on your model lineup, upstream pricing, task mix, and how often fallback is triggered.

Full details: docs/savings-comparison.md.

AIMUX stats comparison

Orchestration Modes

Mode What it does Best for
single One model, one execution path. Most everyday tasks.
plan-then-execute A cheaper planner outlines the work, then an executor runs it. Multi-file or ambiguous changes.
critic Executor output is reviewed by a critic before a final pass. Reviews, refactors, risky edits.
consensus Multiple executors run in parallel, then a synthesizer chooses. High-stakes decisions where disagreement is useful.
auto AIMUX recommends an orchestration mode from task signals. When you want a guided default.

The recommender is currently heuristic and transparent. It prints the recommendation before auto-mode execution.

Supported Tools

Tool Status Notes
Claude Code Supported Best telemetry when structured JSON is available.
OpenCode Supported Supports discovered providers and JSON event streaming in embedded mode.
Aider Planned Not implemented yet.
Native Ollama Planned Currently available indirectly through OpenCode setups.

Configuration

.aimux/config.yaml is bootstrapped on first run. Tune model lineup, tools, cooldowns, refinement, orchestration modes, disabled providers, and cost caps there.

routing:
  mode: balanced
  max_attempts: 3
  prefer_different_provider_on_fallback: true

cooldowns:
  persist: true
  rate_limit_minutes: 30

cost:
  baseline_model: anthropic-sonnet
  max_cost_usd_per_plan: 0.20

Full schema: docs/configuration.md.

Current Limitations

  • AIMUX is alpha software and is not production-ready.
  • Workflow classification is rule-based and can misclassify short or vague prompts.
  • Claude Code and OpenCode output formats can change; parser updates may be needed after upstream CLI releases.
  • Cost reporting is best-effort when upstream tools do not emit structured usage telemetry.
  • The Textual TUI is functional but still rough around embedded streaming, layout polish, and resize behavior.
  • Multi-step orchestration modes are experimental and should be reviewed before trusting their output.
  • Native Windows is unsupported. Use WSL.

Roadmap

  • Local model registry with capability and cost scoring.
  • Provider cooldowns and cross-provider fallback.
  • Claude Code and OpenCode launch paths.
  • Prompt refinement and project context injection.
  • Alpha issue templates for routing and parser bugs.
  • More real-world stream parser fixtures.
  • More reliable workflow classification.
  • Cleaner Textual TUI layout and transcript controls.
  • Aider adapter.
  • Native Ollama adapter.
  • Stable release once core routing, tests, and setup are consistently reliable.

See open issues for bugs, parser drift, and planned adapters.

What's In .aimux/

.aimux/
  config.yaml                # local settings, gitignored
  config.example.yaml        # shipped defaults
  cache/cooldowns.json       # provider/model cooldown memory
  sessions/                  # input history, last_run.json, plans.jsonl, history.jsonl
  contexts/<project>/        # per-project context.md
  runs/<project>/<ts>-.../    # run logs, prompts, routes, attempts, transcripts
  prompts/                   # editable workflow prompt templates

FAQ

Is my code sent anywhere?

Only to the model/tool selected for the run. AIMUX does not call model APIs directly. It launches claude or opencode, and those tools communicate with their configured providers.

Does AIMUX read API keys or OAuth tokens?

No. Credentials stay in the upstream tools. AIMUX never reads token files, auth JSON, .env files, or provider credential stores.

Can AIMUX reduce my AI coding costs?

It can, when your task mix includes work that cheaper models can handle. It is a router and orchestrator, not a pricing guarantee.

Why did AIMUX choose the wrong workflow or model?

The classifier and scorer are transparent but still heuristic. Use /routing, /mode, or --model to override, and open an issue with run.json plus routing_attempts.json if the decision looks wrong.

How is this different from OpenRouter, Aider, Cline, Continue, or Cursor?

OpenRouter is an API gateway. Aider, Cline, Continue, Cursor, Claude Code, and OpenCode are coding tools or agents. AIMUX sits in front of local CLIs and decides which tool/model should handle each workflow, while leaving auth and execution inside those tools.

Contributing

Contributions are welcome, especially around tool adapters, parser robustness, workflow classification, and TUI usability.

Security

AIMUX is local-first and credential-avoidant by design. See SECURITY.md for the full policy.

License

MIT. See LICENSE.

Back to top

About

AIMUX (AI Multiplexer) — terminal-native multi-model AI orchestrator for developer workflows. Classifies tasks, routes to the cheapest viable model, launches the right CLI, tracks cost, and fails over across providers.

Topics

ai-cost-optimizationai-multiplexerai-orchestratorclaude-codeclideveloper-toolsllm-routingmodel-routingopencodeprovider-fallback

Resources

Readme
MIT license

Code of conduct

Code of conduct

Contributing

Contributing

Security policy

Security policy
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages