Docs Q&A Bot

Docs Q&A Bot is a retrieval-augmented generation (RAG) application for turning documentation sites into a searchable question-answering experience. It crawls documentation pages, chunks the content, stores embeddings in Pinecone, and uses an LLM to answer questions with source-backed responses.

What It Does

• Ingests a documentation URL and crawls readable pages.
• Splits content into searchable chunks and stores them in Pinecone.
• Answers questions using retrieved context and returns the source URLs used.
• Lets you list indexed documentation namespaces.
• Provides a clean and responsive browser UI for switching between document collections and chatting with the indexed docs.

Collection dashboard

The main interface shows indexed documentation collections and a chat panel for asking questions.

Answer with sources

Responses include grounded answers and a sources section so you can verify where the information came from.

Switching collections

The UI supports multiple collections, making it easy to move between documentation sets such as LangChain and React.

Architecture

The app is built with FastAPI and organized into a small set of focused services:

• services/crawler.py crawls documentation pages.
• services/chunker.py converts documents into chunks.
• services/vector_store.py manages Pinecone storage and semantic search.
• services/llm_service.py generates answers from retrieved context.
• models/schemas.py defines the API request and response models.

The main API surface is exposed from backend/main.py.

API Endpoints

• GET / - Web UI.
• POST /ingest - Crawl and index a documentation URL.
• GET /collections - List indexed documentation namespaces.
• POST /chat - Ask a question against a namespace.
• DELETE /collections/{namespace} - Delete a namespace from Pinecone.

Environment Variables

The app reads configuration from environment variables at runtime.

Required:

• PINECONE_API_KEY - Pinecone API key.
• PINECONE_INDEX_NAME - Pinecone index name.
• GEMINI_API_KEY - Embedding provider key used for document embeddings.
• GROQ_API_KEY - LLM provider key used for answer generation(You can also use gemini).

Optional:

• MODEL_NAME - LLM model name, default: gemini-2.5-flash.
• EMBEDDING_MODEL - Embedding model, default: gemini-embedding-001.
• EMBEDDING_DIMENSION - Embedding vector size, default: 768.
• DEV_BASE_URL - Default documentation URL used by the crawler.
• MAX_PAGES - Maximum pages to crawl, default: 50 because of API rate limits.
• CRAWL_DELAY - Delay between crawl requests, default: 1.0.
• REQUEST_TIMEOUT - URL validation timeout, default: 20.
• MAX_RETRIES - Retry count for crawling, default: 2.
• CHUNK_SIZE - Chunk size for document splitting, default: 1000.
• CHUNK_OVERLAP - Chunk overlap, default: 200.
• EMBED_BATCH_SIZE - Pinecone upsert batch size, default: 100.
• SEARCH_TOP_N - Number of search results returned to the LLM, default: 5.
• TEMPERATURE - LLM temperature, default: 0.3.
• MAX_OUTPUT_TOKENS - Maximum LLM output tokens, default: 2048.

Run Locally

From the repository root:

cd backend
python -m venv env
source env/bin/activate
pip install -r requirements.txt
uvicorn main:app --reload --host 0.0.0.0 --port 8000

Then open http://localhost:8000.

Run With Docker

Build the image from the repository root:

docker build -f backend/Dockerfile -t docs-qa-bot .

Run the container with your environment file:

docker run --rm -p 8000:8000 --env-file .env docs-qa-bot

Live Deployment

The app is deployed at docs-qa-bot.onrender.com.

Development Notes

• The Docker image is built from files under backend/, so code changes require rebuilding the image unless you mount the source directory into the container.
• Environment variable changes only require restarting the container when they are supplied at runtime.

Project Structure

backend/
├── main.py
├── config.py
├── models/
├── services/
├── static/
├── templates/
└── assets/