Context-and-History-Aware Chatbot API

A simple FastAPI REST API for a context-aware, history-aware chatbot built with LangChain, ChatGroq, Chroma, and HuggingFace embeddings. The chatbot answers questions grounded in a PDF document and uses chat history to reformulate follow-up questions into standalone queries before retrieval.

Features

• FastAPI-based REST API for chatbot interaction.
• Loads the source PDF from data/uploads.
• Uses PyPDFLoader to read the PDF document.
• Splits text with RecursiveCharacterTextSplitter using chunk size 800 and overlap 150.
• Uses sentence-transformers/all-MiniLM-L6-v2 for embeddings.
• Stores embeddings in Chroma with persistence enabled.
• Uses ChatGroq with model llama-3.3-70b-versatile.
• Supports history-aware retrieval using create_history_aware_retriever.
• Supports session-based chat history using RunnableWithMessageHistory.

Project Structure

app/
├── __init__.py
├── main.py
├── config.py
├── schemas.py
├── api/
│   ├── __init__.py
│   └── routes.py
├── core/
│   ├── __init__.py
│   ├── llm.py
│   ├── embeddings.py
│   ├── vectorstore.py
│   ├── prompts.py
│   ├── history.py
│   └── rag_chain.py
└── services/
    ├── __init__.py
    └── chat_service.py

How It Works

The API loads a PDF from the data/uploads folder, splits it into chunks, creates embeddings, and stores them in a persisted Chroma vector database.
When a user sends a message, the system first rewrites follow-up questions into standalone questions using chat history, then retrieves relevant chunks, and finally generates a grounded answer using ChatGroq.
Chat history is maintained per session_id, so follow-up questions in the same session remain context-aware.

Expected Folder Layout

Place your PDF in the data/uploads directory. The vector database will be stored in data/groq_rag_db.

project-root/
├── app/
├── data/
│   ├── uploads/
│   │   └── Example.pdf
│   └── groq_rag_db/
└── ...

Configuration

The current app expects the following values in code or environment configuration. The notebook uses these exact settings.

• PDF folder: data/uploads
• PDF file name: Example.pdf
• Chroma persist directory: data/groq_rag_db
• Embedding model: sentence-transformers/all-MiniLM-L6-v2
• Groq model: llama-3.3-70b-versatile
• Retriever k: 2

Prerequisites

Before running the API, make sure the following are available:

• Python 3.10+ installed
• A valid GROQ_API_KEY environment variable
• The PDF file present inside data/uploads

Run the API

1. Create and activate a virtual environment

Linux / macOS

python -m venv .venv
source .venv/bin/activate

Windows

python -m venv .venv
.venv\Scripts\activate

2. Install dependencies

Install the packages required by the FastAPI app. A sample install command is below.

pip install fastapi uvicorn langchain langchain-community langchain-text-splitters langchain-huggingface langchain-chroma langchain-groq chromadb pypdf sentence-transformers python-dotenv

3. Set the Groq API key

Linux / macOS

export GROQ_API_KEY="your_groq_api_key"

Windows PowerShell

$env:GROQ_API_KEY="your_groq_api_key"

4. Verify the PDF path

Make sure the PDF exists here:

data/uploads/Example.pdf

This path is important because the app loads the document from data/uploads and raises a file-not-found error if the PDF does not exist.

5. Start the FastAPI server

uvicorn app.main:app --reload

Once started, the API should be available at:

• API root: http://127.0.0.1:8000
• Swagger UI: http://127.0.0.1:8000/docs
• ReDoc: http://127.0.0.1:8000/redoc

API Endpoints

The app exposes a health route, a chat route, and a route to clear a session’s chat history. This matches the session-based conversational design used in the notebook.

1. Health Check

Endpoint

GET /api/v1/health

Example

curl -X GET "http://127.0.0.1:8000/api/v1/health"

Sample response

{
  "status": "ok",
  "message": "FastAPI RAG chatbot service is running"
}

2. Chat with the Bot

Endpoint

POST /api/v1/chat

Request body

{
  "session_id": "resume-chat-1",
  "message": "Summarize the candidate's background."
}

Example

curl -X POST "http://127.0.0.1:8000/api/v1/chat" \
  -H "Content-Type: application/json" \
  -d '{
    "session_id": "resume-chat-1",
    "message": "Summarize the candidate'\''s background."
  }'

Sample response

{
  "session_id": "resume-chat-1",
  "answer": "The candidate has experience in developing custom chatbots, designing conversational AI systems, and building interactive dashboards.",
  "context": [
    {
      "page_content": "...",
      "metadata": {
        "page": 0
      }
    }
  ]
}

3. Ask a Follow-up Question

Use the same session_id so the bot can use previous chat history for follow-up understanding. The notebook demonstrates this pattern by asking a second question in the same session after an initial summary query.

Request body

{
  "session_id": "resume-chat-1",
  "message": "What about his latest role?"
}

Example

curl -X POST "http://127.0.0.1:8000/api/v1/chat" \
  -H "Content-Type: application/json" \
  -d '{
    "session_id": "resume-chat-1",
    "message": "What about his latest role?"
  }'

Because the same session_id is reused, the API can treat this as a contextual follow-up rather than an isolated question.

4. Clear Session History

Endpoint

DELETE /api/v1/chat/{session_id}

Example

curl -X DELETE "http://127.0.0.1:8000/api/v1/chat/resume-chat-1"

Sample response

{
  "session_id": "resume-chat-1",
  "message": "Session history cleared successfully"
}

5. Rebuild / Initialize the Vector Index

If you expose the rebuild endpoint from the suggested FastAPI version, it can be used to initialize or reload the vector index. This fits the notebook flow where documents are embedded and stored in Chroma before querying.

Endpoint

POST /api/v1/rebuild-index

Example

curl -X POST "http://127.0.0.1:8000/api/v1/rebuild-index"

Example Workflow

A typical workflow looks like this:

1. Put the PDF in data/uploads/Example.pdf.
2. Start the FastAPI server.
3. Call POST /api/v1/chat with a new session_id.
4. Ask one or more follow-up questions using the same session_id.
5. Clear the session with DELETE /api/v1/chat/{session_id} when done.

Notes

• The chatbot is grounded only in the uploaded PDF content and is instructed to say "I don't know based on the document." when the answer is not present in context.
• The current chat memory is in-memory, so restarting the server clears session history. This follows the notebook’s store = {} approach.
• Chroma persistence allows document embeddings to survive restarts as long as the vector DB directory remains available.
• The current retriever uses similarity search with k=2.

Troubleshooting

GROQ_API_KEY is not set

Set the environment variable before starting the app, because the notebook explicitly raises an error if the Groq API key is missing.

PDF not found

Make sure the file exists exactly at:

data/uploads/Example.pdf

The document loader checks this path before loading.

Empty or weak answers

This may happen if:

• The PDF was not indexed correctly
• The query is outside the document scope
• The vector store needs rebuilding

Follow-up questions not working as expected

Make sure you use the same session_id for related questions, because session memory is keyed by session_id.

Future Improvements

• Add a file upload endpoint for dynamic PDF ingestion
• Replace in-memory session storage with Redis or a database
• Add logging and centralized exception handling
• Add startup initialization for vector DB loading
• Add authentication for protected API usage