Agentic RAG

This repo hosts the code implementation of an end-to-end agentic RAG system.

async web crawler
    ꜜ
ann index
    ꜜ
retrieve/rerank
    ꜜ
MCP tool serve
    ꜜ
FastAPI chat (streaming/sync)

About

I wrote this agentic RAG system to learn and get hands on experience on:

• asynchronous processing & concurrency
  • asyncio's queue, task
  • httpx's AsyncClient
• web crawler
  • how to handle robots.txt properly, by using urllib.robotparser
  • html content parsing via BeautifulSoup or trafilatura
  • discover links and keep track of links crawled, based on http status code
  • detect and filter irrelevant mime_types
  • handle retry and backoff
  • write file to disk
  • a naive BFS solution, and a asyncio queue + worker approach
• RAG (Retrieval-Augmented Generation)
  • use ChromaDB to build a local ANN index, by Sentence Transformer's text embedding model
  • indexer to index crawled contents stored locally
  • retriever to retrieve top_k indexed documents that are close to user's input in the embedding space, based on cosine similarity
  • a simple re-ranker to re-rank retrieved documents for better result accuracy
• local LLM hosting
  • use ollama to host qwen2.5:7b, and optionally, gemma4:26b
• agent
  • a simple agent class to establish connections with local hosted LLM, using OpenAI python SDK
  • configure custom tools
    • fetch_url
  • ReAct loop to trigger/handle tool usages
• MCP server
  • understand what tool means in mcp/agent
  • use MCP python library to build a simple MCP server locally
  • MCP client that establishes connections to locally hosted MCP server
• FastAPI based API server for streaming chat UI
  • expose /chat and /chat/stream API
  • use pydantic BaseModel to define JSON parsable request/response

Repo Overview

This repo uses uv to manage python dependencies.

Please refer to each sub folder's readme file for additional contexts and commands to run components locally.

Claude Opus 4.8 was used for code review and constructing Git commit messages.

Customize / Configuration

To customize the pipeline, simply swap the crawl seed (I use Books to Scrape to safely crawl the internet) with other sources you may want, change to a larger model, such as gemma-4-26B-A4B, to better suit your hardware capacity.

Todo Items

• make each component configurable
• dockerize
• organize blog posts