Arkiv is an intelligent, document-grounded conversational assistant. Built with a Next.js frontend and a FastAPI backend, it leverages Retrieval-Augmented Generation (RAG) to let you upload various document formats and query their contents using natural language.
| FORMAT | LOADER | SUPPORT |
|---|---|---|
| PDF (.pdf) | PyMuPDF4LLMLoader | ✅ Supported |
| CSV (.csv) | UnstructuredCSVLoader | ✅ Supported |
| Text (.txt) | TextLoader | ✅ Supported |
| Markdown (.md) | UnstructuredMarkdown | ✅ Supported |
| JSON (.json) | JSONLoader | ✅ Supported |
| LaTeX (.tex) | TextLoader | ✅ Supported |
| Word (.docx) | UnstructuredWordLoader | ✅ Supported |
| Excel (.xlsx) | UnstructuredExcelLoader | ✅ Supported |
| PowerPoint (.pptx) | UnstructuredPPTLoader | ✅ Supported |
Arkiv uses a decoupled architecture with a Next.js frontend and a FastAPI backend, connected over REST with real-time SSE streaming for LLM responses. The RAG pipeline processes diverse document types, chunks the text, embeds it using an Embedder, and stores it in a Pinecone vector database, ensuring highly accurate, context-aware responses and reducing hallucinations typical of standard LLMs.
| # | COMPONENT | DESCRIPTION | STACK |
|---|---|---|---|
| 1️⃣ | Frontend | Chat interface for querying documents | TypeScript, Next.js, Tailwind CSS, shadcn/ui |
| 2️⃣ | Backend | REST API handling file processing and LLM chat | Python, FastAPI, Uvicorn |
| 3️⃣ | RAG Pipeline | Ingestion, chunking, and embedding logic | Python, LangChain, Pinecone |
| 4️⃣ | Chat Engine | Context-aware chat with real-time SSE streaming | FastAPI SSE, LangChain |
Arkiv/
├── frontend/ # Next.js frontend
│ ├── app/ # Pages
│ │ ├── page.tsx # Home
│ │ ├── chat/ # Chat interface
│ │ └── layout.tsx # Root layout
│ ├── components/ # UI components (shadcn) + custom components
│ ├── lib/ # Utilities
│ ├── hooks/ # Custom React hooks
│ └── public/ # Static assets
├── backend/ # FastAPI backend
│ └── app/
│ ├── main.py # FastAPI app entry point
│ ├── config.py # App configuration
│ ├── routes/ # API route definitions
│ ├── rag/ # RAG pipeline implementations
│ │ ├── loader.py # Document loaders
│ │ ├── chunker.py # Text splitting
│ │ ├── embedder.py # Vector embeddings
│ │ ├── vectorstore.py # Pinecone integration
│ │ ├── cleaner.py # Text sanitization
│ │ └── pipeline.py # E2E processing
│ └── static/ # Static files
├── docs/ # Documentation and reports
├── README.md
└── .gitignore
For detailed setup and usage instructions, refer to the respective README files:
- 🖥️
Backend Instructions- Setting up & running the FastAPI backend - 🌐
Frontend Instructions- Setting up & running the Next.js frontend
Made with ⚡ by Saptarshi Roy & Krishnendu Das