Syndicate — Autonomous AI Research System

Spin up a team of AI experts on demand to research, debate, and produce publication-quality reports.

Syndicate dynamically builds a team of AI domain specialists for any research topic, has them search the web and analyse data, lets them consult each other across disciplines, evaluates the quality of their output, and synthesises everything into a polished research report.

Unlike traditional pipelines, Syndicate creates its own agents at runtime, enabling dynamic specialization per problem.

Built on AutoGen Core with a production-hardened "agents creating agents" pattern.

---

How It Works

Your topic
    │
    ▼
┌─ Planner ──────────────────────────────────────────────────┐
│  Decomposes topic into N specialist roles (structured JSON) │
└────────────────────────────────────────────────────────────┘
    │
    ▼  (parallel)
┌─ ForgeAgent ───────────────────────────────────────────────┐
│  For each role:                                             │
│  1. LLM generates a custom RoutedAgent Python file         │
│  2. Validates syntax + required structure                  │
│  3. Loads via importlib.util.spec_from_file_location()     │
│  4. Registers into the live AutoGen runtime                │
└────────────────────────────────────────────────────────────┘
    │
    ▼  (parallel, streams per-specialist)
┌─ Specialist Agents (dynamically spawned) ──────────────────┐
│  Each specialist:                                           │
│  • Has a unique domain persona generated by the LLM        │
│  • Uses web search + Python REPL tools                     │
│  • Probabilistically consults a peer specialist            │
│  • Writes a 400-600 word section with inline citations     │
└────────────────────────────────────────────────────────────┘
    │
    ▼
┌─ EvaluatorAgent ───────────────────────────────────────────┐
│  Reads all sections, scores each 1-10, writes revision     │
│  notes, produces structured EvaluatorReport                │
└────────────────────────────────────────────────────────────┘
    │
    ▼
┌─ SynthesisAgent ───────────────────────────────────────────┐
│  Assembles sections + applies revision notes into a        │
│  publication-quality markdown report                       │
└────────────────────────────────────────────────────────────┘

---

Features

• Meta-agent pattern — a Forge agent uses LLM code generation + importlib to spawn custom RoutedAgents at runtime; no specialist agents are hardcoded
• Reliable code generation — LLM output validated with ast.parse() + structure checks; retried once on failure; template-assembly fallback guarantees the pipeline never stalls
• Two-phase spawning — all specialists registered before any tasks run, enabling reliable peer consultation
• Per-specialist streaming — Gradio UI updates as each specialist finishes, not waiting for the slowest
• Graceful degradation — individual specialist failures are logged and skipped; pipeline continues with available sections
• Serper / DuckDuckGo search — uses Serper if a key is provided, falls back to DuckDuckGo automatically
• Session persistence — every report saved with metadata; reload any past report from the UI
• Research modes — Quick (2 specialists) / Balanced (4) / Deep (6) selectable from the UI

---

Quickstart

git clone https://github.com/neerajaanil/syndicate.git
cd syndicate
pip install -r requirements.txt

cp .env.example .env
# Edit .env — set OPENAI_API_KEY at minimum

python app.py

Open http://localhost:7860.

---

Configuration

All settings use the HIVE_ prefix and can be set in .env or as environment variables.

Variable	Default	Description
OPENAI_API_KEY	(required)	OpenAI API key
SERPER_API_KEY	—	Serper key for Google search (optional; falls back to DuckDuckGo)
HIVE_RESEARCH_MODE	balanced	quick | balanced | deep
HIVE_MAX_SPECIALISTS	4	Number of specialists to spawn (1–8)
HIVE_PEER_CONSULT_CHANCE	0.4	Probability a specialist consults a peer (0.0–1.0)
HIVE_PLANNER_MODEL	gpt-4o	Model for planning (needs structured output)
HIVE_FORGE_MODEL	gpt-4o	Model for code generation
HIVE_SPECIALIST_MODEL	gpt-4o-mini	Model for each specialist (cost-sensitive)
HIVE_EVALUATOR_MODEL	gpt-4o-mini	Model for evaluation
HIVE_SYNTHESIS_MODEL	gpt-4o-mini	Model for final synthesis

---

Programmatic Usage

import asyncio
from syndicate import ResearchPipeline, SyndicateConfig

async def main():
    config = SyndicateConfig()
    pipeline = ResearchPipeline(
        topic="The economics of lithium-ion battery recycling",
        config=config,
    )
    async for session in pipeline.run():
        if session.final_report:
            print(session.final_report)

asyncio.run(main())

---

Development

pip install -e ".[dev]"
pytest

---

Commercial Applications

Use case	How
Market research	Spawn analysts per vertical; get a structured competitive report
Due diligence	Assemble financial, legal, and technical specialists per target
Policy analysis	Multi-discipline crew covering economic, social, and legal angles
Technical deep-dives	Architect, security, and performance specialists review a system
SaaS wrapper	Add auth + billing to the multi-session-ready pipeline

---

Acknowledgements

This project builds on concepts and example implementations from the ed-donner/agents repository.

Syndicate extends his ideas into a more cohesive, production-oriented system, including:

• Dynamic agent generation ("agents creating agents")
• Multi-stage evaluation and synthesis pipeline
• Runtime validation and fault tolerance
• Streaming and session-based architecture

Huge credit to Ed Donner for the foundational work and inspiration.

---

License

MIT