🤖 Local AI Assistant with RAG
AI assistant that runs **100% locally** — no cloud APIs, no data leaks. Built with Ollama, Next.js 15 and LanceDB.
- 100% Local — everything runs on your machine via Ollama
- RAG Pipeline — upload PDFs and TXT files, ask questions about them
- Streaming responses — real-time typing effect (like ChatGPT)
- Modern UI — TailwindCSS + Framer Motion animations
- Docker ready — one-command deployment
| Layer | Technology |
|---|---|
| Frontend | Next.js 15 (App Router) + TypeScript 5.3 |
| AI Runtime | Ollama (tinyllama + nomic-embed-text) |
| Vector DB | LanceDB (local file-based) |
| Styling | TailwindCSS + Framer Motion |
| DevOps | Docker + docker-compose |
- Node.js 18+
- Ollama installed and running
- Docker (optional)
git clone https://github.com/BroccoliFin/AI-Assistant-with-RAG.git
cd AI-Assistant-with-RAGnpm installollama pull tinyllama
ollama pull nomic-embed-textnpm run devOpen http://localhost:3000 in your browser.
Using Other Ollama Models. By default the project uses the lightweight tinyllama.
To switch to a more powerful model (recommended)
ollama pull llama3.2ollama pull phi3ollama pull mistralUpdate the model name in the code (files inside lib/ where Ollama is initialized). (You can later add model selector in the UI.)
- Upload your documents (PDF / TXT)
- Documents are automatically chunked
- Embeddings are created locally (nomic-embed-text)
- LanceDB finds the most relevant chunks
- Ollama generates the answer using your documents
Everything stays on your machine — documents and vector database never leave your computer.
docker-compose up --build- .env.local — environment variables
- Detailed Ollama setup guide → Ollama.md (./Ollama.md)
- Model selector in the UI
- Chat history + persistence
- More file formats (MD, DOCX, etc.)
- Improved chunking and reranking
- Dark/Light theme toggle
- Export/import chats
Pull requests are welcome!
Feel free to open an Issue or submit a PR.
MIT License — see LICENSE file. © BroccoliFin