RAGLint

Most RAG systems fail because of bad data.

RAGLint scans your data and tells you if it's actually ready for AI.

RAGLint is a small, CLI-first diagnostic tool that scans a folder of documents and produces an AI Data Readiness Report for retrieval-augmented generation (RAG). It is not a chatbot or a demo RAG app: it helps you find data problems before you spend weeks tuning prompts and rerankers.

What it does

• Ingests .txt, .md, and basic .pdf files from a directory tree
• Chunks text with configurable fixed windows (optional paragraph-aware mode)
• Scores chunk quality, near-duplicate overlap (embeddings + cosine similarity), document structure heuristics, lightweight metadata signals, and a FAISS + sentence-transformers retrieval simulation using synthetic queries
• Recommends concrete next steps (chunk size, headings, deduplication, and so on)

Why it matters

Bad corpora dominate RAG failure modes: oversized chunks, repeated boilerplate, wall-of-text pages, and chunks that never surface on realistic queries. RAGLint gives engineers a fast, repeatable sanity check on raw data.

Requirements

• Python 3.11+
• Dependencies listed in requirements.txt (FastAPI, Typer, FAISS, sentence-transformers, PyPDF, PyYAML, pytest, and so on)

Quick start

cd raglint
python3.11 -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Scan a folder

From the raglint repository root (so app and configs resolve correctly):

python cli/cli.py scan examples/sample_docs

Optional flags:

python cli/cli.py scan examples/sample_docs --verbose
python cli/cli.py scan examples/sample_docs --json
python cli/cli.py scan ./data --config configs/default.yaml

The first run downloads the default embedding model (sentence-transformers/all-MiniLM-L6-v2); on a typical laptop the sample corpus should finish well under two minutes.

Optional API mode

uvicorn app.main:app --reload --app-dir .

Then POST /v1/scan with JSON body:

{ "path": "/absolute/path/to/examples/sample_docs" }

Configuration

Edit configs/default.yaml or pass --config to override thresholds (chunk sizes, duplication similarity, retrieval top_k, model id, and score caps). Avoid hardcoding: the CLI and API both load this file by default.

Example CLI output

----------------------------------
AI Data Readiness Report

Chunk Quality: 6.8 / 10
Duplication:   18%  (chunks 0%, lines 18%)
Structure:     5.5 / 10
Metadata:      7.0 / 10
Retrieval:     7.2 / 10

Issues:
- Large chunks detected (>=1200 chars, ~300+ tokens)
- High repeated-line redundancy across the corpus

Recommendations:
- Reduce target chunk size or enable semantic paragraph-aware chunking
- Remove duplicated boilerplate lines or deduplicate templates before chunking
----------------------------------

(Exact numbers depend on your corpus and config.)

Project layout

raglint/
  app/
    main.py              # FastAPI app
    core/                # scanner, chunking, metrics, retrieval, report
    api/routes.py        # HTTP scan endpoint
    utils/               # config + text helpers
  cli/cli.py             # Typer CLI
  configs/default.yaml   # thresholds + model id
  examples/sample_docs/  # tiny demo corpus
  tests/                 # pytest
  requirements.txt

Development

cd raglint
source .venv/bin/activate
PYTHONPATH=. pytest tests -q

Core logic tests mock the embedding model so they stay fast; a real scan exercises sentence-transformers, FAISS, and your filesystem data.