CodeSage — Codebase Intelligence

A retrieval-augmented generation (RAG) application for asking questions about a codebase. Upload a project ZIP, index the supported source files, and ask questions in plain English. Answers include file references and code snippets to help you inspect the relevant implementation.

Features

• ZIP upload with filtering for supported source files and excluded generated/dependency directories.
• Language-aware chunking with file, function, and line metadata.
• Hybrid retrieval combining keyword matching and semantic search.
• Cross-encoder reranking of retrieved candidates.
• Source-file citations, syntax-highlighted snippets, an architecture overview mode, and follow-up questions.
• A Streamlit interface with a file-tree view.

How it works

1. Extract the uploaded ZIP and filter source files.
2. Split code into chunks while recording source metadata.
3. Embed chunks with sentence-transformers/all-MiniLM-L6-v2 and index them in Chroma.
4. Combine BM25 keyword retrieval with vector MMR retrieval using reciprocal rank fusion.
5. Rerank candidate chunks and pass the retrieved context to a Groq-hosted language model.
6. Display the answer, references, and code snippets.

Technology

Component	Technology
Application	Python, Streamlit
RAG framework	LangChain
Embeddings	Hugging Face sentence-transformers
Vector store	Chroma
Retrieval	BM25, vector MMR, reciprocal rank fusion, cross-encoder reranking
Generation	Groq API; documented model: openai/gpt-oss-120b

Model availability and API quotas depend on the configured provider.

Run locally

git clone https://github.com/mtaimoor1407/CodeSage.git
cd CodeSage
python -m venv .venv

Activate the environment:

# Windows
.venv\Scripts\activate

# macOS/Linux
source .venv/bin/activate

Install dependencies:

pip install -r requirements.txt

Create a .env file in the repository root with your own provider credentials, one variable per line:

GROQ_API_KEY=your_groq_api_key
HF_TOKEN=your_huggingface_token

Run from the repository root:

streamlit run app.py

The initial run may download embedding/reranking models. Indexing time depends on repository size, model downloads, and the host machine.

Try it

Upload a ZIP of a small codebase, select Index Codebase, and ask:

• Where is the application entry point?
• How does authentication work?
• Which function creates the database connection?
• What calls this function, and what context supports that explanation?

Open the cited files to check whether the answer matches the implementation.

Repository guide

Path	Purpose
app.py	Streamlit interface
core/	Code loading, chunking, indexing, retrieval, and generation
models/	Structured response schemas
utils/	Supporting utilities
requirements.txt	Python dependencies
.streamlit/	Streamlit configuration

Evaluation and limitations

Source citations help inspect an answer; they do not guarantee that the answer is correct. The displayed confidence label should not be interpreted as a calibrated probability of correctness.

Language-aware splitting and retrieval may miss relationships that require whole-program analysis. Large repositories, generated code, dynamic behavior, and insufficient retrieved context can reduce answer quality.

A useful next evaluation is a fixed set of codebase questions with expected source files, reporting retrieval Recall@k, citation correctness, answer correctness, latency, and comparisons with keyword-only and vector-only retrieval. Quantitative results should be added after those experiments are run.

Author: Muhammad Taimoor — BS Computer Science, COMSATS University Islamabad, Lahore Campus.