🔭 AgentOps Center

AI Operations Center for Multi-Agent Systems powered by OpenTelemetry, SigNoz & Model Context Protocol (MCP)

Observe. Debug. Chaos Test. Explain.
Gain end-to-end visibility into autonomous AI agent workflows using real telemetry, evidence-backed reasoning, and official SigNoz observability infrastructure.

🌐 Live Deployment & Media Links

🎥 Demo & Resources

Resource	Link
🌐 Live Demo	https://agent-ops-center-kohl.vercel.app
⚡ Backend API	https://agentops-center-backend.onrender.com
📖 API Documentation	https://agentops-center-backend.onrender.com/docs
📝 Technical Blog (Medium)	https://medium.com/@swapnilnicolsondadel/why-is-my-ai-agent-taking-45-seconds-when-nothing-is-even-broken-5308685e5da7
💻 GitHub Repository	https://github.com/swapitsneil/AgentOps-Center
💼 LinkedIn	https://www.linkedin.com/in/swapnil-nicolson-dadel/
🐦 X (Twitter)	https://x.com/swappingcodes
🎥 Demo Video	https://youtu.be/eLWzVLf20Mo

Built for the Agents of SigNoz Hackathon 2026, AgentOps Center demonstrates how OpenTelemetry, SigNoz, and the Model Context Protocol (MCP) can be combined to observe, debug, and explain multi-agent AI workflows using live telemetry rather than assumptions.

💡 Problem Statement

Large Language Model (LLM) applications are rapidly transitioning from single-prompt scripts into complex multi-agent systems. Autonomous agents break down tasks, call external APIs, query databases, and pass state between specialized nodes.

However, traditional APM tools were designed for REST APIs and microservices—not non-deterministic AI agent loops. When an AI workflow fails or degrades, engineers struggle to answer critical questions:

Which agent node failed or looped indefinitely?
Which external tool or API returned rate limits or errors?
Why did a workflow take 45 seconds when no exception was thrown?
How much money in token cost was consumed by each sub-agent?
Is the Root Cause Copilot providing verified facts or hallucinating?

Most AI monitoring tools attempt to solve this by storing unverified prompt logs or asking LLMs to guess root causes without real system context.

✨ The Solution: Telemetry-Backed Agent Operations

AgentOps Center replaces guesswork with empirical telemetry.

Every LLM call, agent node transition, tool execution, token count, latency measurement, and runtime exception is instrumented using OpenTelemetry GenAI Semantic Conventions (gen_ai.*), exported to SigNoz ClickHouse (v0.144.6), and queried by an Evidence Engine over the SigNoz Model Context Protocol (MCP).


                                      ┌────────────────────────────┐
                                      │          👤 User           │
                                      └─────────────┬──────────────┘
                                                    │
                                                    ▼
                                  ┌────────────────────────────────┐
                                  │   Next.js 15 Frontend (Vercel) │
                                  │ Dashboard • Copilot • Chaos UI │
                                  └───────────────┬────────────────┘
                                                  │ REST / SSE
                                                  ▼
                               ┌────────────────────────────────────────┐
                               │ FastAPI Backend + LangGraph (Render)   │
                               │ Monitor • Diagnose • Fix • Report      │
                               └──────────────┬──────────────┬──────────┘
                                              │              │
                         OpenTelemetry        │              │ MCP
                           OTLP gRPC          │              │ JSON-RPC
                                              ▼              ▼
                                 ┌─────────────────┐   ┌─────────────────────┐
                                 │ OTel Collector  │   │  SigNoz MCP Server  │
                                 └────────┬────────┘   └──────────┬──────────┘
                                          │                       │
                                          ▼                       │
                                 ┌─────────────────┐              │
                                 │    SigNoz       │◄─────────────┘
                                 │ Query Service   │
                                 └────────┬────────┘
                                          │
                                          ▼
                                 ┌─────────────────┐
                                 │   ClickHouse    │
                                 │ Traces • Logs   │
                                 │ Metrics         │
                                 └────────┬────────┘
                                          ▲
                                          │
                                 Evidence Retrieval
                                          │
                                          ▼
                            ┌────────────────────────────────┐
                            │      Evidence Engine           │
                            │ Correlates Traces, Logs,       │
                            │ Metrics & MCP Responses        │
                            └───────────────┬────────────────┘
                                            │
                                            ▼
                           ┌─────────────────────────────────┐
                           │      Root Cause Copilot         │
                           │ Evidence-backed Diagnosis       │
                           │ Confidence Scoring              │
                           └───────────────┬─────────────────┘
                                           │
                                           ▼
                              👤 Actionable Insights

🏆 Key Architectural Highlights

1. Native OpenTelemetry GenAI Instrumentation

Full adherence to standard OTel GenAI conventions (gen_ai.system, gen_ai.operation.name, gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.usage.cost_usd).
Custom span attributes for agent transitions (agent.name, agent.node, agent.workflow_id, agent.transition.from, agent.transition.to).
Rich tool execution spans tracking exact parameters, duration, and error status (tool.name, tool.input, tool.output, tool.duration_ms).

2. SigNoz MCP Tool Integration

The Root Cause Copilot connects to the official signoz/signoz-mcp-server:latest via HTTP JSON-RPC 2.0. The Copilot invokes official SigNoz tools:

signoz_search_traces: Fetches exact trace hierarchies and span execution times.
signoz_search_logs: Queries structured error logs and exception stack traces.
signoz_query_metrics: Retrieves aggregated token consumption, cost metrics, and duration histograms.
signoz_list_alert_rules: Evaluates active observability alerts.

3. Evidence Engine & Grounded Confidence Badging

The Copilot never exposes raw unverified data or hallucinations to the user. The Evidence Engine:

Invokes SigNoz MCP tools to gather empirical telemetry.
Constructs a VerifiedEvidence data structure.
Automatically grades the response with evidence badges:
- 🟢 HIGH Confidence: Verified by 2+ real MCP telemetry signals (traces/logs/metrics).
- 🟡 MEDIUM Confidence: Partial telemetry signals verified.
- 🟠 LOW Confidence: Unverified or local fallback context only.

4. Chaos Engineering & Incident Injection

Built-in Chaos Engine allows on-demand fault injection during agent workflow runs to evaluate SigNoz observability in real-time:

LLM_TIMEOUT: Simulates provider timeout (504).
LLM_ERROR: Injects 429 Rate Limiting / 500 Server Errors.
TOOL_FAILURE: Forces tool execution failure.
SLOW_RESPONSE: Introduces latency delays (2s - 8s).
AGENT_CRASH: Injects unhandled agent exceptions.

🛠️ Technology Stack

Layer	Technologies Used
Agent Orchestration	LangGraph, LangChain, Python 3.11+
Backend API	FastAPI, Uvicorn, AsyncIO, SSE (Server-Sent Events)
LLM Router	LiteLLM (Groq `llama-3.1-8b-instant`, OpenAI `gpt-4o-mini`)
Observability SDK	OpenTelemetry Python SDK, OpenInference LangChain Instrumentor
Telemetry Pipeline	SigNoz OTel Collector `v0.144.6`, ClickHouse `24.1.2-alpine`
Query Engine	SigNoz Query Service `v0.144.6`
Protocol Bridge	SigNoz MCP Server `v0.9.0` (Model Context Protocol HTTP transport)
Frontend UI	Next.js 15 (App Router), React 19, TypeScript, Tailwind CSS, Lucide Icons
Deployment	SigNoz Foundry (`casting.yaml`), Docker Compose, Vercel, Render

📁 Repository Structure

.
├── backend/
│   ├── agents/
│   │   └── graph.py          # LangGraph Multi-Agent StateGraph (Monitor, Diagnosis, Fix, Report)
│   ├── api/
│   │   ├── copilot.py        # Copilot SSE streaming endpoints (/api/copilot/ask)
│   │   ├── runs.py           # Workflow execution endpoints (/api/runs/trigger)
│   │   └── metrics.py        # Telemetry metrics endpoints
│   ├── chaos/
│   │   └── injector.py       # Chaos Engineering fault injection engine
│   ├── copilot/
│   │   ├── evidence_engine.py# Evidence Engine (gathers & structures MCP telemetry)
│   │   └── reasoning.py      # LLM reasoning engine & confidence grading logic
│   ├── instrumentation/
│   │   ├── setup.py          # OTel TracerProvider & MeterProvider initialization
│   │   ├── agent_spans.py    # OTel GenAI semantic convention context managers
│   │   └── cost_tracker.py   # Token usage counter & cost calculation
│   ├── mcp/
│   │   └── client.py         # SigNoz MCP HTTP JSON-RPC 2.0 Client
│   ├── Dockerfile            # Container definition for FastAPI backend
│   └── main.py               # FastAPI application entrypoint
├── frontend/
│   ├── src/
│   │   ├── app/              # Next.js 15 App Router pages (Dashboard, Copilot, Chaos, Timeline)
│   │   └── components/       # UI Components & Live Telemetry Viewers
│   └── Dockerfile            # Container definition for Next.js frontend
├── otel-collector/
│   └── config.yaml           # OTel Collector pipeline configuration (OTLP gRPC -> ClickHouse)
├── scratch/
│   └── smoke_tests.py        # End-to-end automated verification suite
├── casting.yaml              # Official SigNoz Foundry deployment manifest
├── casting.yaml.lock         # Foundry lockfile
├── docker-compose.yml        # Full 9-container local stack definition
├── requirements.txt          # Python dependencies
└── README.md                 # Project documentation

🚀 Quickstart & Local Development

Prerequisites

Docker Desktop (with Docker Compose v2+)
Python 3.11+
Node.js 18+ (optional, for local frontend development)

1. Clone the Repository

git clone https://github.com/swapitsneil/AgentOps-Center.git
cd AgentOps-Center

2. Configure Environment Variables

Copy .env.example to .env:

cp .env.example .env

Provide at least one LLM provider API key in .env:

GROQ_API_KEY=your_groq_api_key_here
# OR
OPENAI_API_KEY=your_openai_api_key_here

3. Start the Complete Docker Stack

Start all 9 containers (ClickHouse, Zookeeper, OTel Collector, Query Service, Schema Migrator, Frontend UI, Backend API, SigNoz MCP Server, SigNoz UI):

docker compose up -d

4. Verify Container Health

docker compose ps

All services should be in state running or healthy:

AgentOps Center UI: http://localhost:3000
Backend API: http://localhost:8000
SigNoz UI: http://localhost:8080
SigNoz MCP Server: http://localhost:18080/mcp
ClickHouse HTTP: http://localhost:8123

🚢 SigNoz Foundry Deployment

AgentOps Center includes an official SigNoz Foundry manifest (casting.yaml) for 1-step deployment:

apiVersion: v1alpha1
metadata:
  name: agentops-center
  version: "0.1.0"
spec:
  deployment:
    mode: docker
    flavor: compose
    composeFile: ./docker-compose.yml

To deploy via Foundry:

foundryctl cast -f casting.yaml

🧪 Running Automated Smoke Tests

Verify end-to-end telemetry flow (Workflow execution ➔ OTel export ➔ ClickHouse persistence ➔ MCP retrieval ➔ Copilot reasoning with HIGH confidence):

python scratch/smoke_tests.py

Expected output:

=======================================================
SMOKE TEST SUMMARY RESULTS
=======================================================
Run #1: PASSED (wf-958c2184 | Spans: 2340 | Confidence: HIGH)
Run #2: PASSED (wf-5dd5034a | Spans: 2915 | Confidence: HIGH)
Run #3: PASSED (wf-064a9377 | Spans: 3559 | Confidence: HIGH)

ALL 3 SMOKE TEST RUNS PASSED CLEANLY!

📸 Interface Preview & Screenshots

Command Center Dashboard

Overview of active workflows, total token consumption, estimated USD cost, and agent execution graph:

Root Cause Copilot

Real-time SSE streaming copilot displaying verified SigNoz trace evidence and 🟢 HIGH Confidence badges:

Agent Timeline & Span Hierarchy

Detailed trace span visualization showing agent transition flow and sub-span durations:

Chaos Engineering Control Plane

Runtime fault injection interface for testing observability under simulated failures:

Cost & Token Intelligence

Granular breakdown of input/output token usage and USD cost across monitor, diagnosis, fix, and report agents:

🚀 Future Improvements

While AgentOps Center is fully functional and demonstrates end-to-end AI observability with OpenTelemetry, SigNoz, and MCP, there are several areas planned for future enhancement:

Cloud Infrastructure

Migrate backend from the current demo deployment to a production-grade Kubernetes environment.
Add automatic scaling and zero-downtime deployments.
Introduce distributed caching for improved performance.

Authentication & Multi-Tenancy

User authentication and role-based access control.
Multi-workspace support for different teams.
Organization-level dashboards and permissions.

AI & Root Cause Analysis

Historical incident comparison.
Similar incident recommendations.
Automatic remediation suggestions.
Cross-workflow anomaly detection.

Observability

Custom dashboards for individual agent performance.
Long-term telemetry retention.
Additional OpenTelemetry semantic convention coverage.
Real-time alerting and notification integrations.

Integrations

Slack notifications.
Microsoft Teams integration.
GitHub issue creation.
Jira ticket generation.
PagerDuty incident creation.

Deployment

Helm Charts for Kubernetes.
Terraform deployment templates.
AWS, Azure and GCP deployment guides.
One-click cloud deployment options.

Reporting

PDF incident reports.
Executive dashboards.
Cost optimization reports.
Weekly AI operations summaries.

📄 License & Acknowledgements

This project is open-source under the MIT License.

Special thanks to the SigNoz Team for building world-class open-source observability infrastructure and organizing the Agents of SigNoz Hackathon 2026.

Name		Name	Last commit message	Last commit date
backend		backend
clickhouse		clickhouse
docs		docs
frontend		frontend
images		images
infra		infra
observability		observability
otel-collector		otel-collector
tests		tests
.env.example		.env.example
.gitignore		.gitignore
README.md		README.md
casting.yaml		casting.yaml
casting.yaml.lock		casting.yaml.lock
docker-compose.yml		docker-compose.yml
hackathon_guide.md		hackathon_guide.md
migration_report.md		migration_report.md
release_report.md		release_report.md
requirements.txt		requirements.txt
signoz_dashboard.json		signoz_dashboard.json

Latest commit

History

Folders and files

Repository files navigation