⚡ vromvrom-engine

Async multi-agent LLM orchestrator — hybrid routing, Elo scoring, self-healing

Vroom Vroom — running at full throttle 🏁

Built for Home Assistant · Works with any project

⚠️ Project Status: Active Development Personal tool shared with the community. Backend (DAG, async, Elo routing) is used daily. The React dashboard (ihm-v2/) is included — build it once, then gui_server.py serves it at /. Expect rough edges.

🇫🇷 Version française disponible → README.fr.md  ·  🗺️ Which LLMs to use? → STRATEGIES.md

---

Stop configuring. Start learning. vromvrom-engine routes your tasks to the right LLM automatically — using Elo scoring that adapts from your own usage history. Async, cost-aware, self-healing. Built on a Raspberry Pi budget.

🎯 What is it?

vromvrom-engine is a fully asynchronous (asyncio) multi-agent orchestration engine that coordinates multiple LLMs to solve complex tasks. It was born from the need to drive a home automation display (M5Stack Tab5) through Home Assistant — but its architecture is generic and reusable for any project.

💡 Experiment with GPT-4o & Llama 70B for free The engine comes pre-configured to connect to the GitHub Models API, offering free access (subject to rate limits) to models like GPT-4o and Llama 70B for your tests using your standard GitHub account.

What makes it different

Feature	Description
🧠 4-level hybrid routing	Regex → ML (sklearn) → LLM → Elo scoring — 0ms to 200ms depending on complexity
🏆 Dynamic Elo scoring	Each model gains/loses Elo points per task domain. Best model selected automatically
🔄 Async parallel DAG	Independent tasks run concurrently via asyncio.gather()
🛡️ Self-Healing	Circuit breaker + exponential backoff retry + automatic provider fallback
💰 Cost-aware routing	Availability/budget cascade: falls back to the next configured provider when one is unavailable, rate-limited, or over budget — not a quality-based escalation
🔍 Local hybrid RAG	TF-IDF + BM25 + ChromaDB Embeddings fused via RRF (k=60) — zero cloud cost
👁️ HITL	Human-In-The-Loop: pause/resume orchestration for human validation
📊 FastAPI dashboard	Glassmorphism HTML/JS UI with real-time SSE, workflow editor, Elo charts
🔌 Drop-in Proxy API	100% OpenAI-compatible API (/v1/chat/completions). Plug Cursor, Cline, or Continue.dev directly into the engine
🏠 Native Home Assistant	Built-in /api/ha routes and entity ingestion tools to let LLMs see and control your smart home
🌐 Distributed Swarm	Dispatch tasks to remote workers (Raspberry Pi, VMs, etc.)
🔌 Plugin system	Add custom agents via plugins/<name>/agent.py + plugin.json

---

🏗️ Architecture

vromvrom-engine/
├── gui_server.py          # FastAPI entry point (lifespan + 15 routers)
├── main.py                # CLI — direct launch without HTTP
│
├── core/                  # Engine core
│   ├── engine.py          # Main orchestrator (DAG → Agents)
│   ├── llm_gateway.py     # Multi-provider gateway (18+ providers)
│   ├── router.py          # Hybrid routing (fast-path + ML + LLM + Elo)
│   ├── dag_runner.py      # Async parallel execution by stages
│   ├── factory.py         # Agent instantiation (Planner/Executor/Reviewer)
│   ├── state.py           # Thread-safe Pydantic GlobalState
│   ├── checkpoint.py      # ACID snapshots (SQLite WAL)
│   ├── healing.py         # Self-healing + retry
│   ├── review_loop.py     # Reviewer → Correction loop
│   ├── elo_scorer.py      # Per-domain model Elo scoring
│   ├── elo_router.py      # Routing type Elo scoring
│   ├── circuit_breaker.py # Async circuit breaker (CLOSED/OPEN/HALF_OPEN)
│   ├── hitl.py            # Human-In-The-Loop (asyncio.Event)
│   ├── models_db.py       # SQLite SSOT — model catalog, pricing, quotas
│   └── ...
│
├── agents/                # Specialized agents
│   ├── planner.py         # Breaks task into a JSON DAG
│   ├── executor.py        # Executes tasks (ReAct loop + tools)
│   ├── reviewer.py        # Validates result quality
│   └── tool_maker_agent.py  # Generates new Python tools automatically
│
├── memory/                # Semantic memory + RAG
│   ├── rag.py             # Hybrid RAG (TF-IDF + BM25 + Embeddings + RRF)
│   ├── facts.py           # Fact store (SQLite FTS5 BM25)
│   ├── episodes.py        # Episodic memory (Jaccard similarity)
│   ├── embeddings.py      # ChromaDB vector store
│   └── skills.py          # Procedural memory (successful tool sequences)
│
├── tools/                 # Agent-usable tools
│   ├── tool_registry.py   # Registry with per-tool timeouts
│   └── sanitizer.py       # Secret masking (6 patterns)
│
├── api/routes/            # 15 FastAPI route modules
├── services/              # Business logic decoupled from HTTP
├── plugins/               # Custom plugins (dynamic loading)
├── workflows/             # JSON workflow definitions (graphs)
├── static/                # HTML/JS/CSS UI (glassmorphism, fallback if ihm-v2 is not built)
├── ihm-v2/                # React/Vite/TS dashboard (Chat, HA, LLM registry, Prompt Studio…)
├── tests/                 # 60 pytest files
└── docs/                  # Architecture documentation

Hybrid routing pipeline

User request
      │
      ▼
┌─────────────────────┐
│ 1. Regex Fast-Path  │ ──→ 0ms    (simple commands detected by pattern)
└─────────────────────┘
      │ ambiguous
      ▼
┌─────────────────────┐
│ 2. ML Router sklearn│ ──→ 0ms    (local classifier, 75% confidence threshold)
└─────────────────────┘
      │ confidence < 75%
      ▼
┌─────────────────────┐
│ 3. LLM Slow-Path    │ ──→ ~200ms (Gemini Flash to resolve ambiguity)
└─────────────────────┘
      │
      ▼
┌─────────────────────┐
│ 4. Elo Scoring      │ ──→ selects the best model for the task domain
└─────────────────────┘
      │
      ▼
  Planner → DAG → Parallel Executor(s) → Reviewer → Response

---

⚙️ Quick Start

Requirements

• Python 3.11+
• At least one LLM API key (free Gemini tier works, see .env.example)

Setup

# 1. Clone the repo
git clone https://github.com/Axellum/vromvrom-engine.git
cd vromvrom-engine

# 2. Virtual environment
python -m venv .venv
# Windows:
.venv\Scripts\activate
# Linux/Mac:
source .venv/bin/activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Configure
cp .env.example .env
# Edit .env — at minimum set GEMINI_API_KEY (free at aistudio.google.com)

# 5. Engine config
cp config.example.json config.json
# Optional: adjust default models

# 6. Launch
#    (optional) React UI: cd ihm-v2 && npm install && npm run build
python gui_server.py
# → Dashboard at http://localhost:8000

Minimal configuration (.env)

The engine works with a single free key:

# Free Gemini key: https://aistudio.google.com/apikey
GEMINI_API_KEY=AIza...

# Local API authentication key (generate a random value)
MOTEUR_API_KEY=change-me-with-secrets-token-urlsafe-32

All other providers (DeepSeek, Anthropic, Mistral...) are optional — the engine degrades gracefully.

---

🚀 Usage

Web Dashboard

cd ihm-v2 && npm install && npm run build   # → ihm-v2/dist/
python gui_server.py                        # http://localhost:8000

If ihm-v2/dist is missing, the engine falls back to the legacy UI in static/ (also under /v1).

Dev with HMR: cd ihm-v2 && npm run dev (Vite :5173, proxies /api to :8000).

The React dashboard includes Chat (SSE + tools), Home Assistant / voice, LLM registry, Prompt Studio, observability, and config.

Via the REST API

import httpx

response = httpx.post(
    "http://localhost:8000/api/execute",
    json={"message": "Analyze this Python code and suggest improvements"},
    headers={"Authorization": "Bearer <MOTEUR_API_KEY>"}
)
print(response.json()["result"])

As a Drop-in Proxy for IDEs (Cursor, Cline, Continue)

vromvrom-engine exposes a standard OpenAI-compatible endpoint. You can point your favorite AI coding assistant to the engine to benefit from free routing and the circuit breaker:

• Base URL: http://localhost:8000/v1
• API Key: <MOTEUR_API_KEY> (from your .env)
• Model: gemini-2.0-flash (or any model available in your routing)

🏠 Home Assistant Integration (Optional but Native)

vromvrom-engine ships with built-in tools (ha_entity_ingest.py) and routes (/api/ha/*) to interact with your Home Assistant instance out of the box.

Just add these to your .env:

HASS_URL=http://192.168.1.X:8123
HASS_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

Your agents can now instantly read your sensors, check states, and control devices without writing any extra YAML!

Via the CLI

python main.py "Explain hexagonal architecture in 3 key points"

---

🔌 Supported LLM Providers

Provider	Free tier	Notes
Gemini (Google AI Studio)	✅	Recommended to start — generous free tier
GitHub Models	✅	GPT-4o, Llama 3.3 70B via GitHub token
DeepSeek	💰	Excellent price/quality ratio (~$0.14/M tokens)
Anthropic Claude	💰	Via API or Claude Code CLI (Pro subscription)
Mistral	✅	European GDPR-friendly models — free tier
OpenRouter	💰	Aggregator (200+ models)
LM Studio	✅	Local inference (Qwen, Llama, Mistral...)
Ollama	✅	Alternative local inference
MiniMax	💰	Built-in <think> reasoning
Cohere	✅	Free trial models (Command R / R+)
Cerebras	✅	100% free — ultra-fast inference (GPT-OSS 120B, GLM 4.7)
Zhipu AI (GLM)	💰	8 GLM models, strong in code/agentic tasks
xAI (Grok)	💰	Paid only

---

🧪 Tests

# Install dev dependencies
pip install -r requirements-dev.txt

# Run all tests
pytest

# With coverage report
pytest --cov=. --cov-report=html

---

🗺️ Which LLMs should I use?

See STRATEGIES.md for 4 ready-to-use configurations:

• 🆓 Free & Local only — $0/month
• 🐉 Chinese models — ~$5–15/month, near-frontier quality
• ⚡ Claude API/CLI — near-zero extra cost with a Pro subscription
• 🚀 Full Power + cost-optimized — state of the art at ~10× lower cost

---

🤝 Contributing

Contributions are welcome! See CONTRIBUTING.md.

Priority areas:

• 🐳 Dockerization (minimal image + docker-compose)
• 📖 API documentation (see docs/API_REFERENCE.md)
• 🧪 Increased test coverage
• 🌍 Support for additional LLM providers
• 🎨 UI improvements

---

📄 License

MIT — See LICENSE.

---

🙏 Inspirations

• LangChain — orchestration patterns
• CrewAI — multi-agent coordination
• Home Assistant — home automation ecosystem
• FastAPI — API framework

---

_{Built with ❤️ for the home automation and open-source AI community}