🧠 PoliSense — LLM-Powered Policy Query-Retrieval System

A CLI-based Retrieval-Augmented Generation (RAG) system that reads any policy document from a URL and answers natural language questions with structured, justifiable decisions.

Built for HackRx 6.0, PoliSense combines Google Gemini embeddings, a local Qdrant vector store, and Groq-hosted LLaMA inference to deliver fast, explainable answers grounded in the actual document text.

---

✨ Features

Feature	Details
Multi-format ingestion	PDF via pypdf, DOCX/email/others via unstructured-io
Semantic chunking	RecursiveCharacterTextSplitter (1000 tokens, 200 overlap) with page-level metadata
Google Gemini embeddings	embedding-001 model for high-quality vector representations
Local Qdrant vector DB	Persisted on disk at ./qdrant_db — no cloud dependency
Dual LLM support (test,py)	Groq llama-3.1-8b-instant (fast) + Gemini 1.5-pro-latest (accurate) side-by-side
Structured JSON output	Every answer includes Decision, Amount, Justification, and PageReference
Progress bar download	Real-time download progress for large documents
Token usage tracking	Reports estimated embedding tokens + LLM tokens consumed

---

🗺️ How It Works

PoliSense runs a classic RAG (Retrieval-Augmented Generation) pipeline in 6 sequential stages:

flowchart TD
    A([🌐 Input: Document URL + Questions]) --> B

    subgraph INGESTION["📥 Stage 1 — Document Ingestion"]
        B[HTTP GET with progress bar\nrequests + User-Agent header]
        B --> C{Content Type?}
        C -- PDF --> D[pypdf\nPer-page text extraction\nwith page number metadata]
        C -- DOCX / Email / Other --> E[unstructured-io\nAuto-partition → single Document]
    end

    subgraph CHUNKING["✂️ Stage 2 — Text Chunking"]
        D & E --> F[RecursiveCharacterTextSplitter\nchunk_size=1000  overlap=200\nPreserves page metadata]
    end

    subgraph INDEXING["🗄️ Stage 3 — Vector Indexing"]
        F --> G[Google Gemini embedding-001\nGenerates dense vector per chunk]
        G --> H[(Local Qdrant DB\n./qdrant_db\nforce_recreate=True)]
    end

    subgraph QUERY["🔍 Stage 4 — Retrieval"]
        I([❓ User Question]) --> J[Embed question\nvia Gemini embedding-001]
        J --> K[Similarity search\nagainst Qdrant collection]
        K --> L[Top-k relevant chunks\nwith page references]
    end

    subgraph GENERATION["🤖 Stage 5 — LLM Generation"]
        H -.populated by.-> K
        L --> M[Prompt assembly\nContext + Question + Domain]
        M --> N{LLM Choice}
        N -- app.py --> O[Groq API\nllama3-8b-8192\nLow latency]
        N -- test,py --> P[Groq llama-3.1-8b-instant\n+\nGemini 1.5 Pro\nSide-by-side comparison]
    end

    subgraph OUTPUT["📊 Stage 6 — Structured Output"]
        O & P --> Q[JSON response parsing]
        Q --> R["✅ Decision: Approved / Rejected / N/A
💰 Amount: ₹X or N/A
📝 Justification: quoted clause
📄 PageReference: page N"]
    end

    R --> S([🖥️ Terminal output + timing summary])

Loading

Step-by-step breakdown

1. Document Ingestion — load_document_from_url()
   Downloads the file via requests with a spoofed browser User-Agent. Supports optional --no-ssl-verify for self-signed certs. Streams the response and shows a real-time [####----] progress bar.

2. Document Processing — process_document()
   Detects the content type (application/pdf header or .pdf URL suffix) and routes to the right parser:

   • PDF → pypdf.PdfReader, page-by-page, each page stored as a Document with metadata={"page": N}
   • Other → unstructured.partition.auto.partition(), entire file collapsed into one Document

3. Text Chunking — get_text_chunks()
RecursiveCharacterTextSplitter breaks each document into overlapping 1000-character chunks. split_documents() (used in test,py) preserves page metadata across chunk boundaries — so the LLM knows which page each answer comes from.

4. Vector Store Creation — get_vector_store()
   Embeds every chunk using Google Gemini embedding-001 and stores them in a local Qdrant collection (hackrx-local-collection). The DB is re-created fresh on every run (force_recreate=True). Token count is estimated via tiktoken (cl100k_base encoding) for cost tracking.

5. LLM Initialization

   • app.py → single Groq client with llama3-8b-8192
   • test,py → both Groq (llama-3.1-8b-instant) and Gemini (gemini-1.5-pro-latest) initialized in parallel, enabling direct model comparison on the same queries.

6. Retrieval & Generation — create_retrieval_chain()
   For each question:

   • The question is embedded and used to query Qdrant for the top-k most semantically similar chunks.
   • The chunks + question are injected into a structured prompt template (with {domain}, {context}, {input} slots).
   • The LLM is instructed to respond only in JSON with Decision, Amount, Justification, and PageReference.
   • The raw response is parsed; if JSON extraction fails, the raw text is printed as a fallback.

---

🚀 Getting Started

Prerequisites

• Python 3.9+
• pip
• Internet connection (for document downloads + API calls)
• API Keys: Google Gemini and Groq

Installation

# 1. Clone the repo
git clone <your-repo-url>
cd PoliSense

# 2. Create and activate a virtual environment
python -m venv venv
source venv/bin/activate  # Windows: .\venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

Configure API Keys

Create a .env file in the project root:

GEMINI_API_KEY="YOUR_GEMINI_API_KEY_HERE"
GROQ_API_KEY="YOUR_GROQ_API_KEY_HERE"

⚠️ Never commit .env to git. It is already listed in .gitignore.

---

🖥️ Usage

app.py — Production (Groq only)

python app.py <document_url> -q "<question_1>" "<question_2>" [--domain <domain>] [--no-ssl-verify]

test,py — Comparison mode (Groq + Gemini side-by-side)

python "test,py" <document_url> -q "<question_1>" "<question_2>" [--domain <domain>] [--no-ssl-verify]

Arguments

Argument	Required	Description
document_url	✅	URL of the policy PDF or document
-q / --questions	✅	One or more questions (each in double quotes)
--domain	❌	Document domain, default: insurance. E.g. legal, hr, compliance
--no-ssl-verify	❌	Disables SSL verification (use for self-signed certs)

Example

python app.py \
  "https://hackrx.blob.core.windows.net/assets/policy.pdf" \
  -q "Is knee surgery covered for a 46-year-old male with a 3-month-old policy?" \
     "What is the waiting period for pre-existing diseases?" \
  --domain insurance

---

📊 Sample Output

1. Fetching document from URL: https://example.com/policy.pdf...
   Downloading document...
      Progress: [########################################] 100.00%
   Document fetched successfully.
2. Processing document...
   (Processing as PDF using pypdf...)
   Document processed.
3. Chunking text...
   Text split into 248 chunks.
4. Creating vector store with embeddings...
   Initializing embedding model (Google Gemini)...
   Estimated tokens for embedding: 152400
   Setting up local Qdrant database at './qdrant_db'...
   Local Qdrant vector store is ready.
5. Initializing LLM...
   LLM initialized successfully.

--- Answering Questions ---

-> Question: Is knee surgery covered for a 46-year-old male with a 3-month-old policy?

   Answer:
{
    "Decision": "Rejected",
    "Amount": "N/A",
    "Justification": "As per clause 3.1.2: 'A waiting period of 24 months applies to all knee-related surgeries.' The policy has only been active for 3 months, which does not satisfy the minimum waiting period.",
    "PageReference": "Page 12"
}

   (Time taken for this question: 2.15s)
------------------------------

--- Final Summary ---

Step Durations:
  - Document Loading: 1.50s
  - Document Processing: 0.80s
  - Text Chunking: 0.10s
  - Vector Store Creation: 5.20s
  - LLM Initialization: 0.30s

Token Usage:
  - Estimated Embedding Tokens (Gemini): 152400
  - Total LLM Tokens (Groq): 1200

Total Application Execution Time: 8.10s

---

🗂️ Project Structure

PoliSense/
├── app.py          # Main RAG pipeline (Groq LLM only)
├── test,py         # Experimental — dual LLM comparison (Groq + Gemini)
├── requirements.txt
├── .env            # API keys (not committed)
├── .gitignore
├── LICENSE
└── qdrant_db/      # Auto-generated local vector database

---

⚡ Performance Notes

• Groq API (llama-3.1-8b-instant) delivers sub-3s inference latency per question.
• Qdrant local skips network round-trips — similarity search is in-process.
• force_recreate=True means the vector store is rebuilt fresh on each run. For repeated queries on the same document, remove this flag to reuse the existing index and save embedding cost.
• Token costs are tracked via tiktoken (embeddings) and get_openai_callback (LLM tokens).

---

🛣️ Future Improvements

• Persistent vector store — skip re-embedding if the document URL is unchanged
• Cross-encoder reranking — improve retrieval quality with a sentence-transformers reranker
• Query rewriting — expand ambiguous queries with an LLM before retrieval
• FastAPI / web UI — REST endpoint for integration into external systems
• OCR support — unstructured-io hi_res strategy for scanned PDFs
• Batch document ingestion — index multiple policy documents into a single collection

---

📜 License

MIT License — see LICENSE for details.

---

🙏 Acknowledgements

Developed for HackRx 6.0. Powered by LangChain, Qdrant, Groq, and Google Gemini.