Skip to content

README.md

Latest commit

 

History

History
106 lines (82 loc) · 4.17 KB

README.md

File metadata and controls

106 lines (82 loc) · 4.17 KB
Raw

CodeCouncil — Multi-Agent PR Review System

A production-ready multi-agent system where 4 specialized AI agents debate GitHub pull requests and deliver consensus code reviews with conflict detection, confidence scoring, and cross-session memory.

Architecture

What is CodeCouncil?

CodeCouncil deploys 4 specialized AI agents that analyze, debate, and arbitrate pull request reviews in 3 rounds. Unlike single-agent reviewers, CodeCouncil agents have distinct specializations and genuinely disagree with each other — producing more thorough, multi-dimensional reviews.

Agents

  • 🔴 SecurityAuditor — SQL injection, hardcoded secrets, XSS, auth bypass
  • 🟡 PerfEngineer — N+1 queries, complexity issues, memory leaks, blocking calls
  • 🟢 CleanCodeReviewer — naming conventions, SOLID principles, error handling
  • 🟣 TechLead — reads the full debate and makes the final arbitrated verdict

How it works

  1. PR opened on GitHub → webhook fires
  2. All 3 specialist agents analyze in parallel (Round 1)
  3. Agents read each other's opinions and explicitly agree or disagree (Round 2)
  4. TechLead reads the full debate transcript and posts verdict to GitHub PR (Round 3)
  5. Slack notification sent to #pr-reviews channel automatically

Features

  • Parallel execution — agents run simultaneously via ThreadPoolExecutor, 3x faster than sequential
  • Conflict detection engine — algorithmic detection of genuine agent disagreements with severity scoring
  • Confidence calibration — tracks how much agents change their confidence through debate
  • Debate impact score — quantifies how much the debate influenced final verdict (0-100)
  • PR history memory — remembers recurring issues across past reviews for the same repo
  • Live debate dashboard — real-time UI at /dashboard showing agents debating
  • GitHub integration — auto-posts structured review comment to the PR
  • Slack notifications — sends formatted verdict to your team channel

Benchmark Results

CodeCouncil vs single-agent baseline on identical PR diffs:

  • +22% issue detection rate
  • 2x more issues found per PR (5.3 vs 2.7 average)
  • Full results: benchmark_results.json

Tech Stack

  • Python — core language
  • FastAPI — webhook receiver and API gateway
  • Qwen Cloud — powers all 4 agents via qwen-plus model
  • GitHub API — PR diff fetching and comment posting
  • Slack API — team notifications
  • ThreadPoolExecutor — parallel agent execution
  • JSON — lightweight PR memory store

Setup

git clone https://github.com/Efrrowini/CodeCouncil.git
cd CodeCouncil
python -m venv venv
venv\Scripts\activate  # Windows
pip install -r requirements.txt
cp .env.example .env
# Add your keys to .env:
# DASHSCOPE_API_KEY=your_qwen_key
# GITHUB_TOKEN=your_github_token
# SLACK_WEBHOOK_URL=your_slack_webhook
uvicorn main:app --reload --port 8000

Open http://localhost:8000/dashboard for the live debate UI.

API Endpoints

Endpoint Method Description
/ GET Health check
/dashboard GET Live debate UI
/webhook POST GitHub webhook receiver
/api/review POST Manual PR review trigger
/memory/{owner}/{repo} GET PR history for a repo

Project Structure

CodeCouncil/ ├── agents/ │ ├── base_agent.py # Qwen Cloud API calls │ ├── security.py # SecurityAuditor │ ├── performance.py # PerfEngineer │ ├── cleancode.py # CleanCodeReviewer │ └── techlead.py # TechLead arbitrator ├── core/ │ ├── debate.py # 3-round debate orchestration │ ├── conflict.py # Conflict detection engine │ ├── memory.py # PR history memory │ ├── github.py # GitHub API integration │ ├── slack.py # Slack notifications │ └── baseline.py # Single-agent baseline ├── frontend/ │ └── index.html # Live debate dashboard ├── main.py # FastAPI app └── benchmark_results.json # Benchmark data

License

MIT