Agents Discussion

Multi-agent automated root-cause analysis for production incidents.

Agents Discussion runs a structured debate between three specialized LLM agents — a diagnostician, a skeptic, and a moderator — to find the root cause of a technical incident. Each agent can run a different frontier model, and they don't just talk: they invoke real, read-only diagnostic tools (SSH, kubectl, Prometheus, Loki, SQL EXPLAIN, Git) to gather evidence, with human approval gates for anything sensitive.

The hypothesis map after a debate: one root cause confirmed (green, the moderator's leader) and two alternatives refuted with the exact metric that killed each one (red).

📖 Full documentation: https://luisgf.github.io/ia_agents_discussion/

---

Why

A single LLM gives you a fluent, confident guess — but it can't doubt itself, can't see your systems, and throws away its reasoning. A wrong root-cause analysis sends you to revert the wrong deploy while the real cause keeps paging.

Agents Discussion behaves like a good incident bridge: a diagnostician proposes hypotheses, a skeptic tries to falsify them with discriminating tests, and a moderator keeps everyone honest until the evidence actually points somewhere — recording every step as an auditable trail you can paste into a postmortem.

What it does

When an incident occurs — a latency spike, 5xx errors, data inconsistency, a memory leak — it:

1. Assembles a panel of three agents (diagnostician / skeptic / moderator), each a potentially different model.
2. Probes your infrastructure through a ReAct tool loop: SSH, DB EXPLAIN, Prometheus/Loki queries, Git history, HTTP checks, read-only kubectl.
3. Tracks structured evidence independently of the prose, and verifies it before committing to a diagnosis.
4. Debates hypotheses across rounds — each hypothesis a first-class object with a stable id, a state (active / confirmed / rejected), a calibrated probability, and a transition log of who changed it, when, and why.
5. Plans discriminating tests — the single check that separates two competing causes.
6. Compresses history and routes adaptively (skips settled steps, early-outs on overwhelming confidence) to keep token usage bounded.
7. Keeps humans in the loop — approval gates for sensitive tools, an optional pause between rounds, and a full audit.jsonl.
8. Produces a report — Markdown with the transcript, tool audit, hypothesis timeline, evidence, a recommended fix, and a token/cost breakdown. Runs are persisted and replayable.

---

A worked example

The screenshots below come from the demo run that ships with the project.

The incident: checkout-api p99 latency jumped from ~120 ms to 2.4 s a minute after the 14:10 deploy; error rate and traffic are flat. PostgreSQL with an ~8M-row orders table behind PgBouncer.

Round 1 — cast a wide net, then start killing hypotheses. The diagnostician pulls git_recent_changes, finds the 14:08 commit, runs EXPLAIN, sees a Seq Scan on orders, and proposes three hypotheses. The skeptic goes straight for the cheapest to disprove — Prometheus shows the PgBouncer pool is not saturated — and rejects hypothesis 2 in round 1, recording the metric that killed it.

Round 2 — the discriminating test. Both query plans are run on a replica (index scan vs seq scan, same data, only the predicate differs), GC is ruled out, and the moderator closes with a typed decision: propose_fix, confidence 88 %, risk low, a concrete index fix, and the rejected alternatives with their reasons — plus the per-agent token/cost breakdown.

What you get at the end isn't "the AI thinks it's the index." It's a timeline of evidence and counter-evidence: three hypotheses, two killed with the exact metric that killed them, one confirmed by a controlled test, and a reversible fix with a stated risk level.

---

Installation

git clone https://github.com/luisgf/ia_agents_discussion
cd ia_agents_discussion

python -m venv .venv
source .venv/bin/activate

pip install -e .

cp .env.example .env
# Edit .env with your GITHUB_TOKEN, or authenticate GitHub Copilot:
agents-discuss-copilot-auth

Requirements: Python 3.11+, Linux / macOS / WSL. Models run on GitHub Models or GitHub Copilot.

---

Quick start

CLI

# Minimal
agents-discuss "The /orders endpoint p95 latency is 2s since the 14:00 deploy"

# With context files and project source
agents-discuss "Memory leak in worker pods" \
  --file logs/oom-kills.txt \
  --base-context docs/architecture.md \
  --project ./backend --include "src/**/*.py"

Web UI

agents-discuss-web
# open http://127.0.0.1:8000

Describe the incident, pick a model per role, and launch. The UI streams the debate live (SSE), groups tool calls into collapsible cards, gates sensitive tools behind an Approve button, and lets you pause between rounds to add evidence. Every run is saved and can be replayed or exported.

---

How it works

Under the hood it's a LangGraph StateGraph: each agent is a node, a single typed state flows between them, and conditional edges decide what runs next.

START → diagnostic → skeptic → rebuttal → moderator → summarize → [human gate] → (loop)
                                              └────────────────────────────────→ finalize → END

• Structured memory is the primary state. Hypotheses, evidence, tool results, and the moderator decision are typed objects (Pydantic / TypedDict) merged by reducers — not just a growing text transcript. This is what makes runs inspectable, replayable, and drift-resistant.
• Evidence is tracked independently and verified. Findings are recorded as first-class evidence and checked before the moderator is allowed to close early, so a confident-but-unsupported diagnosis can't slip through.
• Hypothesis probabilities are calibrated by the skeptic each round, and the moderator plans the discriminating test that would separate the live hypotheses.
• Adaptive control flow. The moderator can skip the skeptic or rebuttal when a point is settled; the diagnostician can signal an early-out; a history-compression node summarizes old rounds so round 7 doesn't re-send rounds 1–6.
• Snapshots & resume. Investigation snapshots are restored when a run is resumed with new evidence, so the debate continues from where it left off.
• Replayable & auditable. Every non-ephemeral event is persisted as JSON; the UI rebuilds the full thread and hypothesis map from it, and every tool call is appended to audit.jsonl.

See Architecture for the full state schema, topology, and decisions.

---

Key features

• Multi-model panel — assign a different model (GPT-4o, Claude, Gemini, …) to each agent role.
• Grounded ReAct tools — SSH, read-only kubectl, Prometheus/Loki/Elasticsearch queries, PostgreSQL EXPLAIN, HTTP, Git — all read-only, with unsafe verbs rejected at the tool level.
• First-class hypotheses — stable ids, state lifecycle, calibrated probabilities, and an interactive hypothesis map (Cytoscape) you can filter and replay.
• Evidence verification — structured evidence tracked and checked before an early diagnosis.
• Human-in-the-loop — tool approval gating and an optional pause between rounds.
• Token economy — history compression, a per-round tool cache, scoped context, and a built-in cost estimate (tokens + USD) per run.
• Reproducible benchmarks — benchmark.py + tests/benchmark_cases.json for regression-testing diagnostic quality.
• Bilingual prompts — tuned templates in English and Spanish for performance / errors / data / security incidents (language selectable; English by default).

---

Diagnostic tools

The agents gather evidence through a ReAct loop over read-only tools; safety is enforced in the tool, not left to the prompt:

Tool	Notes
run_ssh_command	Honors your OpenSSH config (~/.ssh/config) and configured auth
run_kubectl	Read-only verbs only (get, describe, logs, …); mutating verbs rejected
run_db_explain	EXPLAIN without ANALYZE, SELECT/WITH only, multi-statement blocked
query_prometheus / query_loki / query_elasticsearch	Instant/range queries, output truncated
git_recent_changes	Because the cause is so often the last deploy
http_get / run_local_command	HTTP health checks and read-only local diagnostics

Sensitive tools (SSH, local command, kubectl, DB) require an operator Approve click in the web UI; identical calls are cached per round so two agents don't re-run the same query. The full generated list lives in the tools reference.

---

Configuration

Everything is set via environment variables or a .env file (see .env.example):

Variable	Purpose	Example
GITHUB_TOKEN / COPILOT_TOKEN	Authentication for GitHub Models / Copilot	ghp_… / ghu_…
DIAGNOSTIC_MODEL	Model for the diagnostician	copilot/gpt-4o
SKEPTIC_MODEL	Model for the skeptic	copilot/claude-sonnet-4.6
MODERATOR_MODEL	Model for the moderator	copilot/claude-sonnet-4.6
PROMPT_LANGUAGE	Debate language (en | es)	en
TOOL_APPROVAL_REQUIRED	Require approval for sensitive tools	true
COMPRESS_HISTORY	Summarize old rounds	true

The complete, always-current list is generated from the code in the configuration reference.

---

REST API

The web server exposes the API the UI consumes (full list in the API reference):

POST   /api/runs                 # start a debate
GET    /api/runs                 # list runs
GET    /api/runs/{id}            # run details
GET    /api/runs/{id}/events     # live SSE stream
GET    /api/runs/{id}/report     # Markdown report
POST   /api/runs/{id}/resume     # resume with new evidence
POST   /api/runs/{id}/approval   # resolve a tool approval
POST   /api/runs/{id}/comment    # human-in-the-loop comment

---

Documentation

Docs are published at https://luisgf.github.io/ia_agents_discussion/ (MkDocs Material). The docs/reference/ section is generated from the code and verified in CI, so it can't drift.

Doc	Audience	Content
Architecture	Developers, architects	Design, state schema, topology, decisions
Usage	End users	CLI, web UI, API, templates, tools
Operations	DevOps, SREs	Deploy, config, monitoring, troubleshooting
Coding style	Contributors	Conventions, patterns, versioning & releases
Reference	Everyone	Generated: API, CLI, config, tools, events

---

Contributing

1. Read the coding style and architecture.
2. Before committing: ruff check ., ruff format --check ., and pytest — all enforced in CI.
3. If you change routes, env vars, tools, or SSE events, run python scripts/gen_docs.py and commit the regenerated docs/reference/ (a CI gate and consistency tests keep the docs in sync).

Versioning follows SemVer driven by Conventional Commits; pushing a vX.Y.Z tag builds and publishes a GitHub Release.

---

License

Licensed under the GNU General Public License v3.0 or later (GPL-3.0-or-later). See LICENSE for the full text.

Copyright (C) 2025 Luis González Fernández

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. It is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.