Dochat Notebook

Agentic AI workspace for document chat, web search, transparent reasoning, and conversation memory.
Transform your insights into blog posts or social media content to share with your professional network.

Table of Contents

• Overview
• Architecture Diagram
• Installation
  • Prerequisites
  • Option 1: Local Development
    • Step 1: Download the project
    • Step 2: Create a virtual environment
    • Step 3: Install dependencies
    • Step 4: Configure the backend environment
    • Step 5: Launch the app
    • Step 6: Open the app
  • Option 2: Docker Deployment
• Configuration Notes
• Supporting LLM Models

Overview

Dochat Notebook Design is an agentic document workspace for grounded Q&A across uploaded files. It combines document parsing, hybrid retrieval, conversation memory, optional web search, and traceable agent actions in a single application.

You can upload documents, ask follow-up questions, inspect how the agent reached an answer, save notes, and turn grounded outputs into blog drafts or social media posts.

Dochat Notebook works as follows:

1. Uploaded documents are parsed and split into searchable chunks.
2. Chunks are indexed in Pinecone for semantic retrieval.
3. User questions are routed through an agent workflow.
4. The agent retrieves relevant document context, memory, and optional web search results.
5. The final answer is streamed back with source-aware context and visible reasoning steps.

Feature	Description
Document workspace	Upload and manage PDF, DOCX,OCR, TXT, and Markdown files
Hybrid retrieval	Combines dense vector search, sparse retrieval, and optional reranking
Agentic routing	Routes each question to document retrieval, web search, or answer generation
Traceable answers	Shows the agent actions behind each response in the UI
Conversation memory	Reuses recent context and stored conversation history across turns
Source-aware chat	Streams answers with retrieved context and source references
Notes and sessions	Keep conversations, notes, and workspace state organized
Content creator	Generate blog drafts and social post variants from document context
Provider flexibility	Supports OpenAI, Anthropic, Google Gemini, and llama.cpp models
Docker	Provides single-container deployment via Docker Compose

Architecture Diagram

Dochat Notebook has two main layers:

• API: A Python/FastAPI service for parsing documents, indexing chunks, running retrieval, orchestrating agents, storing memory, and streaming responses.
• UI: A React interface for file management, chat, agent trace visibility, notes, model controls, and content generation.

Installation

Prerequisites

• Docker with Docker Compose, or for local development:
  • Python 3.11 or 3.12 (3.13 is not supported by the pinned torch==2.4.1)
  • Node.js 20+ and npm (only needed to run the UI dev server)
• At least one supported LLM provider:
  • OpenAI API key, or
  • Anthropic API key, or
  • Google API key, or
  • A running llama-server reachable over HTTP (install separately from llama.cpp; see Step 4)
• Pinecone API key for document retrieval

Important

Pick your providers before setup. Configure .env in Step 4 with your chat provider, chat model, embedding model, and Pinecone dimension. (see Supporting LLM Models) For llama.cpp, Dochat connects to an external llama-server over its OpenAI-compatible HTTP API. Start the server separately and point Dochat at it through LLAMACPP_BASE_URL.

Option 1: Local Development

Step 1: Download the project

git clone https://github.com/genchandenur/dochat.git
cd dochat

Step 2: Create a virtual environment

cp .env.example .env

macOS / Linux

python3 -m venv .venv
source .venv/bin/activate

Windows

python -m venv .venv
.venv\Scripts\activate

Step 3: Install dependencies

pip install -r requirements.txt
pip install -e .

Step 4: Configure the backend environment

Create a .env file in the project root using the template below.

# Both LLM_PROVIDER and LLM_MODEL are required for every provider.
LLM_PROVIDER=openai     # openai | anthropic | google | llamacpp
LLM_MODEL=gpt-4o

# OpenAI (default)
OPENAI_API_KEY=

# Anthropic
ANTHROPIC_API_KEY=

# Google Gemini
GOOGLE_API_KEY=

# for llama.cpp (external OpenAI-compatible server)
# Install and start `llama-server` yourself (see Prerequisites), then change
# the two lines above and uncomment LLAMACPP_BASE_URL below:
#llama-server \
#  --hf-repo Qwen/Qwen3-8B-GGUF \
#  --hf-file Qwen3-8B-Q4_K_M.gguf \
#  --port 8080 \
#  -ngl 99 \
#  --jinja \
#  --ctx-size 32768 \
#  --flash-attn on \
#  --parallel 2
# LLM_PROVIDER=llamacpp
# LLM_MODEL=Qwen3-8B-Q4_K_M.gguf
# LLAMACPP_BASE_URL=http://host.docker.internal:8080/v1   # Docker
# LLAMACPP_BASE_URL=http://localhost:8080/v1              # bare-metal


EMBEDDING_PROVIDER=sentence-transformers
EMBEDDING_MODEL=all-mpnet-base-v2

PINECONE_API_KEY=
PINECONE_INDEX=dochat
PINECONE_REGION=us-east-1
PINECONE_DIMENSION=768 # for all-mpnet-base-v2
PINECONE_METRIC=cosine
PINECONE_CLOUD=aws

TAVILY_API_KEY=

Step 5: Launch the app

Build the UI once — the backend serves the result on the same port:

cd ui && npm install && npm run build && cd ..

Then start the backend:

UI_DIST_DIR=ui/dist uvicorn dochat.api.app:app --host 0.0.0.0 --port 8000

or:

UI_DIST_DIR=ui/dist dochat api

Step 6: Open the app

Open: http://localhost:8000

For active frontend development with hot reload:

cd ui
npm run dev

Open: http://localhost:5173

The Vite dev server proxies API requests to the backend on port 8000.

Option 2: Docker Deployment

Create and configure your .env file first. (see Step 4).

docker compose -f docker/docker-compose.yml up -d

Configuration Notes

Setting	Purpose
LLM_PROVIDER	Selects the chat model provider: openai, anthropic, google, or llamacpp. Required.
LLM_MODEL	Chat model identifier for the active provider. Required.
OPENAI_API_KEY / ANTHROPIC_API_KEY / GOOGLE_API_KEY	API keys for the selected cloud provider
LLAMACPP_BASE_URL	HTTP endpoint of an external OpenAI-compatible llama-server. Use http://host.docker.internal:8080/v1 from Docker, http://localhost:8080/v1 bare-metal.
EMBEDDING_PROVIDER	Embedding provider: sentence-transformers or openai.
EMBEDDING_MODEL	Embedding model used for document indexing and retrieval.
PINECONE_API_KEY	Required for document retrieval
PINECONE_INDEX	Pinecone index name
PINECONE_DIMENSION	Vector dimension. Must match the embedding model.
TAVILY_API_KEY	Optional; enables web search when configured
DB_PATH	Optional SQLite database path; defaults to storage/<PINECONE_INDEX>.db
DOCLING_NUM_THREADS	Optional; CPU threads for parsing. Defaults to 4. Affects ingest speed only.
DOCLING_PERF_PAGE_BATCH_SIZE	Optional; PDF pages per model pass. Defaults to 4. Set higher values for faster on large docs, more memory.

Uploaded files are stored under storage/uploads. Conversation and note records are stored in SQLite.

Supporting LLM Models

Dochat builds chat models through LangChain and the OpenAI-compatible llama.cpp server. Model and provider selection is environment-driven: set LLM_PROVIDER and LLM_MODEL in .env. Defaults to OpenAI gpt-4o for chat and sentence-transformers/all-mpnet-base-v2 for embeddings.

Cloud chat models

Provider	Env value	Model setting	Required secret
OpenAI	LLM_PROVIDER=openai	LLM_MODEL=gpt-4o	OPENAI_API_KEY
Anthropic	LLM_PROVIDER=anthropic	LLM_MODEL=claude-sonnet-4-6	ANTHROPIC_API_KEY
Google Gemini	LLM_PROVIDER=google	LLM_MODEL=gemini-2.5-pro	GOOGLE_API_KEY

Local chat models

Provider	Env value	Model identifier	Server
llama.cpp	LLM_PROVIDER=llamacpp	LLM_MODEL=<GGUF served by llama-server>	External llama-server reachable at LLAMACPP_BASE_URL; Dochat does not start or manage it.

Choose your embedding model

Set EMBEDDING_PROVIDER=sentence-transformers (default; weights download to the local Hugging Face cache on first run):

Embedding model	Dim	Max tokens	Notes / card
all-mpnet-base-v2	768	384	Semantic search baseline Card
all-MiniLM-L6-v2	384	256	Smaller and faster; useful for lower latency or smaller indexes Card
paraphrase-multilingual-MiniLM-L12-v2	384	128	Multilingual MiniLM variant for mixed-language collections Card
multi-qa-mpnet-base-dot-v1	768	512	Tuned for question-answer retrieval workloads Card
BAAI/bge-m3	1024	8192	Multilingual retrieval model with long-context support Card
BAAI/bge-large-en-v1.5	1024	512	English retrieval model; often benefits from query/document instructions Card
intfloat/e5-large-v2	1024	512	Works best with query: and passage: prefixes Card

See model catalog: sentence-transformers/models.

Set EMBEDDING_PROVIDER=openai:

Embedding model	Dim	Max tokens	Notes / card
text-embedding-3-small	1536	8191	Fast and cost-efficient OpenAI embedding model. Card
text-embedding-3-large	3072	8191	Higher-capacity OpenAI embedding model. Card

Important

PINECONE_DIMENSION must match the Dim value of the selected embedding model. If you change embedding models, create a new Pinecone index with the matching dimension before indexing documents, because Pinecone index dimensions cannot be changed in place.

---

Todo

• Switch the vector db from Pinecone to Chroma, Qdrant, pgvector, or Milvus to run completely local with no API costs.
• Add optional tracing/metrics (Langfuse, LangSmith, Phoenix, OpenTelemetry) to track agent latency and token cost.