🧠 OmniQuery: Document Intelligence Platform

📋 Table of Contents

• Overview
• Features
• Demo
• Installation
• Configuration
• Usage
• Supported File Types
• Technology Stack
• Project Structure
• How It Works
• Contributing
• License
• Contact

🌟 Overview

OmniQuery is an AI-powered document intelligence platform that allows users to upload documents and interact with them through natural language queries. Built with Streamlit, LangChain, and OpenAI's GPT models, OmniQuery extracts, processes, and analyzes information from your documents, providing intelligent answers to your questions in real-time.

✨ Features

🎯 Core Features

• 📂 Multi-Format Support: Upload PDF, DOCX, TXT, and Markdown files
• 🤖 AI-Powered Q&A: Ask questions in natural language and get intelligent responses
• 💾 Vector Store: Efficient document embedding using ChromaDB for fast retrieval
• 💬 Conversation Memory: Maintains context throughout your chat session
• 📊 Feedback System: Rate responses and visualize feedback analytics
• 📥 Export Functionality: Download complete chat history as JSON

🎨 UI/UX Features

• 🌙 Modern Dark Theme: Sleek cyberpunk-inspired interface with gradient effects
• ⚡ Real-time Processing: Live progress indicators during document processing
• 📈 Visual Analytics: Interactive feedback distribution charts using Plotly
• 🎚️ Temperature Control: Adjust AI response creativity with temperature slider
• ✨ Smooth Animations: Hover effects and transitions for better user experience

🔧 Technical Features

• 🔄 Retry Logic: Robust error handling with exponential backoff
• 📦 Chunking Strategy: Smart document splitting for optimal processing
• 🗄️ Session State Management: Persistent state across interactions
• 🔐 Secure API Integration: Environment-based configuration for API keys
• 🐛 SQLite Compatibility: Uses pysqlite3 for ChromaDB compatibility

🎥 Demo

Upload a document, ask questions, and get instant AI-powered answers!

🚀 Installation

Prerequisites

• Python 3.8 or higher
• OpenAI API key
• Git (optional)

Step 1: Clone the Repository

git clone https://github.com/Varsha-1605/Document-Uploader.git
cd Document-Uploader

Step 2: Create Virtual Environment (Recommended)

# Windows
python -m venv venv
venv\Scripts\activate

# macOS/Linux
python3 -m venv venv
source venv/bin/activate

Step 3: Install Dependencies

pip install -r requirements.txt

⚙️ Configuration

Environment Setup

Create a .env file in the root directory or configure Streamlit secrets:

Option 1: Using .env file

OPENAI_API_KEY=your_openai_api_key_here

Option 2: Using Streamlit Secrets Create .streamlit/secrets.toml:

OPENAI_API_KEY = "your_openai_api_key_here"

Getting OpenAI API Key

1. Visit OpenAI Platform
2. Sign up or log in
3. Navigate to API Keys section
4. Create a new API key
5. Copy and use in your configuration

💻 Usage

Running the Application

streamlit run final_document_uploader.py

The application will open in your default browser at http://localhost:8501

Step-by-Step Guide

1. 📂 Upload a Document

   • Click "Browse files" in the sidebar
   • Select a PDF, DOCX, TXT, or MD file
   • Wait for processing to complete

2. 🌡️ Adjust Settings (Optional)

   • Use the temperature slider (0.0 - 1.0)
   • Lower values: More focused/consistent responses
   • Higher values: More creative/varied responses

3. ❓ Ask Questions

   • Type your question in the text input
   • Click "Generate Response" or press Enter
   • View the AI-generated answer

4. 👍👎 Provide Feedback

   • Rate each response as helpful or not helpful
   • View analytics in the feedback visualization chart

5. 📥 Export Chat History

   • Click "Export Chat History" to download as JSON
   • Preserve your conversation for future reference

📄 Supported File Types

File Type	Extension	Loader Used
PDF	.pdf	PyPDFLoader
Word Document	.docx	UnstructuredWordDocumentLoader
Text File	.txt	UnstructuredFileLoader
Markdown	.md	UnstructuredFileLoader

🛠️ Technology Stack

Frontend

• Streamlit: Web application framework
• Custom CSS: Cyberpunk-themed dark UI

AI & ML

• OpenAI GPT-3.5-turbo: Language model for Q&A
• OpenAI Embeddings: Document vectorization
• LangChain: LLM application framework

Data Processing

• ChromaDB: Vector database for embeddings
• Unstructured: Document parsing
• PyPDF: PDF processing
• python-docx: Word document handling

Utilities

• Tenacity: Retry logic and error handling
• Pandas: Data manipulation
• Plotly: Interactive visualizations
• pysqlite3: SQLite compatibility

📁 Project Structure

Document-Uploader/
│
├── final_document_uploader.py    # Main application file
├── requirements.txt              # Python dependencies
├── New_logo.png                 # Application logo
├── .env                         # Environment variables (create this)
├── .gitignore                   # Git ignore rules
├── README.md                    # Project documentation
│
├── .devcontainer/               # Dev container configuration
│   └── devcontainer.json
│
└── temp_docs/                   # Temporary document storage

🔍 How It Works

1. Document Processing Pipeline

graph LR
A[Upload Document] --> B[Load Document]
B --> C[Split into Chunks]
C --> D[Generate Embeddings]
D --> E[Store in ChromaDB]
E --> F[Create Vector Store]

Loading

2. Query Processing

User Query → Agent Executor → Vector Store Search → Retrieve Relevant Chunks → 
LLM Processing → Generate Answer → Stream Response → Display to User

3. Key Components

Document Loading

def load_document(file_path):
    # Automatically detects file type and uses appropriate loader
    # Supports PDF, DOCX, TXT, MD

Text Chunking

• Chunk Size: 1000 characters
• Overlap: 200 characters
• Strategy: RecursiveCharacterTextSplitter

Vector Store

• Database: ChromaDB
• Embeddings: OpenAI Ada-002
• Collection: document_store

Agent

• Type: VectorStore Agent
• Memory: ConversationBufferMemory
• Callbacks: Streamlit streaming handler

📊 Features in Detail

Temperature Control

Controls the randomness of AI responses:

• 0.0: Deterministic, focused answers
• 0.7: Balanced (default)
• 1.0: Creative, varied responses

Feedback Analytics

• Visual pie chart showing response helpfulness
• Real-time feedback collection
• Exportable feedback data

Conversation Memory

• Maintains context across multiple questions
• Stores full chat history in session state
• Exportable as JSON for record-keeping

🤝 Contributing

Contributions are welcome! Here's how you can help:

1. Fork the Repository

   git fork https://github.com/Varsha-1605/Document-Uploader.git

2. Create a Feature Branch

   git checkout -b feature/AmazingFeature

3. Commit Your Changes

   git commit -m 'Add some AmazingFeature'

4. Push to the Branch

   git push origin feature/AmazingFeature

5. Open a Pull Request

Development Guidelines

• Follow PEP 8 style guidelines
• Add comments for complex logic
• Test thoroughly before submitting PR
• Update documentation as needed

🐛 Known Issues & Limitations

• Large documents (>50 pages) may take longer to process
• API rate limits apply based on OpenAI plan
• Temporary files created during upload are automatically cleaned up
• Session state is lost on browser refresh

🔜 Future Enhancements

• Support for more file formats (CSV, Excel, PPT)
• Multi-document upload and querying
• Document comparison features
• Advanced search filters
• User authentication and saved sessions
• Export to different formats (PDF, Word)
• Support for other LLM providers
• Cloud deployment options

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

📧 Contact

Varsha

• GitHub: @Varsha-1605
• Repository: Document-Uploader

🙏 Acknowledgments

• Streamlit for the amazing web framework
• LangChain for LLM orchestration
• OpenAI for powerful language models
• ChromaDB for vector storage

---

⭐ Star this repository if you find it helpful!

Made with ❤️ using Streamlit and LangChain