CC-RAGOS

Enterprise, self-hosted, explainable multimodal RAG platform (NotebookLM-style) — by Abhisek Bose. Own lightweight FastAPI orchestration (no Dify) + a Next.js visualization layer = the USP.

• PRD: PRD-CC-RAGOS.md
• Tech stack & flow: techstackwithflow.md
• Scaling: docs/SCALING.md · Eval: docs/EVAL-IMPROVEMENTS.md · Prompts: prompts/README.md

Demo

Ask the workspace a question → streamed, grounded answer with clickable citations and a highlighted visual citation. Full-length walkthrough (login → chat → Knowledge Graph → embeddings → mind map → the "Explain the Pipeline" tour): docs/demo/ragos-demo.mp4.

Regenerate: cd scripts/demo && npm install && npx playwright install chromium, then set -a; source ../../.env; set +a; node record.mjs (records to out/), and convert with ffmpeg.

Architecture

web (Next.js) ──► /api/chat  ──► retriever /chat (SSE) → steps + answer tokens + citations
      └─────────► /api/agent-chat ──► retriever /agent/chat (SSE) → plan → multi-hop retrieve → answer + scratchpad
      └─────────► retriever viz/tool APIs (chunks, embeddings/UMAP, playground, graph, study, audio, eval, analytics)
retriever (FastAPI)  condense(history) → guardrail gate → embed → retrieve (± rerank, ± source subset) → grounded prompt → stream LLM → citations
                     [agentic] DeepAgent planner → researcher/critic sub-agents, write_todos + retrieve tools → state-backed scratchpad (study_artifacts)
ingestion (FastAPI)  Docling / vision / PyMuPDF → chunk (fixed|structure|sentence|parent_child|semantic|agentic) → contextualize → embed → Qdrant  (also MCP pull)
stores: Qdrant (vectors) · networkx (graph, per workspace) · SQLite (chats, study, eval, feedback, agent state)

Standard chat orchestration (services/retriever/app/chat.py + /chat): every pipeline step emitted over SSE for the "Explain the Pipeline" USP.

Agentic chat (services/retriever/app/agent.py + /agent/chat): DeepAgents/LangGraph with multi-hop retrieval, sub-agent critique, and a state-backed virtual filesystem. Frontend shows activity timeline, todos checklist, and scratchpad viewer (file download + sandboxed HTML preview).

Prompts live in /prompts as editable .txt templates (loaded via each service's app/prompts.py). New agentic chunking strategy uses LLM-detected boundaries with verbatim anchors + chunk titles.

Stack decisions (locked)

Concern	Choice
Chat Orchestrator	Own FastAPI + SSE (standard) · DeepAgents/LangGraph (agentic, multi-hop with sub-agents)
Models	OpenRouter (LLM / vision / embeddings / rerank / agent models)
Embeddings	text-embedding-3-large (dense, 3072-d) + in-process BM25 (rank-bm25) fused via RRF for hybrid
Retrieval	semantic · hybrid · HyDE · GraphRAG, ± Cohere rerank, ± source subset; history-aware + contextual retrieval
Multimodal	vision-LLM caption + query-time bbox (visual citations); attach a reference image in chat → vision caption+OCR augments retrieval; ColPali deferred (GPU)
Chunking	structure (default) + fixed, sentence, parent_child, semantic, agentic (LLM boundary + chunk titles)
Vector DB	Qdrant (native binary, HNSW)
Graph	networkx in-process (per-workspace JSON); Neo4j deferred (JVM)
TTS	Deepgram Aura-2 (Audio Overview); Dia dropped (~6 GB + GPU)
Store	SQLite (data/ccragos.db, chat history + agentic state via AsyncSqliteSaver) → Postgres at scale
Guardrails (chat)	relevance gate (entry dense probe + per-retrieval) + prompt-injection defense + scope + grounding
Guardrails (agent)	entry dense probe + per-retrieval dense probe, configurable thresholds (settings.relevance_threshold)
Agent Model Allowlist	settings.tool_capable_models (default: deepseek/deepseek-v4-flash, anthropic/claude-sonnet-4-6, openai/gpt-4o)
Observability	Langfuse (chat + ingestion + agent traces, token cost); no-op if unset
Auth / RBAC	Self-hosted JWT (bcrypt + HS256), roles viewer/editor/admin on both services; conversation ownership enforced on /chat + /agent/chat; AUTH_ENABLED flag (off = open). Keycloak dropped (JVM).

Run (host-native — no Docker)

cp .env.example .env       # fill OPENROUTER_API_KEY (+ DEEPGRAM / LANGFUSE optional)
scripts/start-host.sh      # qdrant (native binary) + ingestion + retriever + web

Open http://localhost:3000. Stop:

pkill -f 'uvicorn app.main'; pkill -f 'bin/qdrant'; pkill -f 'next start'

• Qdrant: native binary bin/qdrant, data in qdrant-storage/.
• Ingestion + Retriever: share .venv-ingestion (python3.12); each has a run-host.sh.
• Web: web/ via npm run dev (dev) or next build/next start (prod); host URLs in web/.env.local.

Auth (optional)

Off by default (AUTH_ENABLED=false) → the app runs open. To enforce login + roles set in .env:

AUTH_ENABLED=true
AUTH_SECRET=<long random string ≥16 chars>   # required when enabled
AUTH_ADMIN_USER=admin
AUTH_ADMIN_PASSWORD=<strong, not "admin">     # seeds the first admin on empty DB

Roles: viewer (read + chat), editor (+ upload/create/delete), admin (+ workspaces + users). Sign in at /login; admins manage accounts at /users. Restart the retriever by port after changing .env (lsof -ti tcp:8100 | xargs kill -9 — both services share uvicorn app.main, so a broad pkill would also stop ingestion).

colima/Docker was abandoned locally (the Docling/torch build corrupted its VM disk). The Docker files were removed in cleanup — re-add containerization when deploying to a Linux/prod host. start-host.sh runs the web in production (next build + next start) for fast loads.

Verify

curl localhost:8100/health   # retriever → {"status":"ok"}
curl localhost:8101/health   # ingestion → {"status":"ok"}

# ingest a doc (streamed per-stage SSE, survives disconnect via semaphore)
curl -N -F "file=@sample.pdf" -F "collection=ccragos_chunks" localhost:8101/ingest/stream

# standard chat (SSE stream) — add  -H "Authorization: Bearer <token>"  if AUTH_ENABLED
curl -N -X POST localhost:8100/chat -H "Content-Type: application/json" \
  -d '{"query":"...", "collection":"ccragos_chunks", "top_k":5}'

# agentic chat (SSE stream, multi-hop with sub-agents and scratchpad)
curl -N -X POST localhost:8100/agent/chat -H "Content-Type: application/json" \
  -d '{"query":"...", "collection":"ccragos_chunks"}' \
  -H "Authorization: Bearer <token>" # if AUTH_ENABLED

Testing

Pytest scaffold: run all tests with .venv-ingestion/bin/python -m pytest -q

Test coverage:

• services/retriever/tests/ — agent build, route parity, events translation, scratchpad persistence, DB concurrency, tool binding
• services/ingestion/test_agentic/ — agentic chunking boundary detection and fallback

Gated live tests: set RUN_AGENT_INTEGRATION=1 to run OpenRouter-dependent tests. Default: 75 passed, 2 skipped.

Layout

services/retriever/app/
  agent.py, agent_tools.py, agent_events.py      DeepAgent build + tools (retrieve, list_strategies)
                                                   + event translator (to SSE)
  chat.py                                          Standard orchestration: history condense, guardrails, LLM stream
  main.py                                          FastAPI app: /chat, /agent/chat, /conversations, /study/artifacts,
                                                   /audio, /eval, /graph, /auth (JWT + RBAC), viz + playground APIs
  config.py                                        Env-based settings (agent_model, tool_capable_models,
                                                   agent_recursion_limit, relevance_threshold, etc.)
  db.py                                            SQLite: chats, conversations, study artifacts, eval, agent state
  
services/ingestion/app/
  chunkers.py                                      6 strategies: structure, fixed, sentence, parent_child,
                                                   semantic, agentic (LLM boundaries + titles)
  main.py                                          FastAPI app: /ingest/stream (detached, semaphore-capped),
                                                   /visual-cite, /mcp/* (MCP pull)
  config.py                                        Env-based settings (chunk_size, agentic_chunk_model, etc.)
  
web/app/
  agent/page.tsx                                   Frontend page: activity timeline + todos + scratchpad
  api/agent-chat/route.ts                          Next.js proxy to retriever /agent/chat
  api/chat/route.ts                                Next.js proxy to retriever /chat
  page.tsx                                         Main workspace: sources, chat, studio (create/inspect/library)
  
prompts/
  agent_system.txt, agent_researcher.txt, agent_critic.txt     Agentic RAG prompts
  chunk_agentic.txt                                LLM boundary-detection prompt
  *_*.txt                                          Editable prompt templates (loaded by both services)

scripts/start-host.sh                              Launch qdrant + ingestion + retriever + web host-native
bin/qdrant                                         Native Qdrant binary
data/, qdrant-storage/, graph-storage/, media/    Local state (gitignored)
pytest.ini                                         Root pytest config (pythonpath, asyncio_mode)

Scaling → see docs/SCALING.md (Qdrant HNSW/indexing, sparse hybrid, quantization, caching, K8s, Postgres).

Roadmap → see PRD §11 (Phases P0–P6). P0–P6 shipped, incl. auth + RBAC (self-hosted JWT).

Remaining hardening for multi-user prod: httpOnly-cookie tokens (currently localStorage), HA/Helm, and an optional OIDC/Keycloak SSO adapter.