A comprehensive, interactive pipeline for detecting, analyzing, and categorizing AI-generated hallucinations with evidence-based verification and factuality scoring.
This project demonstrates AI Quality Assurance — specifically the ability to evaluate and validate AI-generated content against ground-truth reference information. Rather than merely using AI, this system focuses on quality control of AI output.
The system implements a complete Hallucination Detection Pipeline that:
- Extracts Claims from AI-generated text using NLP heuristics
- Verifies Evidence by cross-referencing claims against ground-truth data
- Scores Factuality using weighted multi-method similarity analysis
- Categorizes Errors into specific hallucination types with severity levels
- Generates Reports with actionable findings and recommendations
┌──────────────┐ ┌──────────────────┐ ┌────────────────────┐
│ LLM Output │────▶│ Claim Extraction │────▶│ Evidence │
│ (Input) │ │ (NLP Tokenizer) │ │ Verification │
└──────────────┘ └──────────────────┘ └────────┬───────────┘
│
┌──────────────┐ ┌──────────────────┐ ┌────────▼───────────┐
│ Evaluation │◀────│ Error │◀────│ Factuality │
│ Report │ │ Categorization │ │ Scoring │
└──────────────┘ └──────────────────┘ └────────────────────┘
- Sentence tokenization with abbreviation handling
- Factual assertion detection (numbers, proper nouns, dates, factual verbs)
- Claim type classification: quantitative, temporal, attributive, factual
- Jaccard Similarity: Word set intersection over union
- Weighted Term Overlap: Length-weighted content word matching
- N-gram Overlap: Bigram and trigram sequence matching
- Named Entity Matching: Proper noun, number, date, and measurement comparison
- Combined Score: Weighted fusion of all methods
| Error Type | Severity | Description |
|---|---|---|
| Fabrication | Critical | Completely made-up information |
| Contradiction | Critical | Directly conflicts with reference |
| Exaggeration | Major | Inflated numerical values |
| Misattribution | Major | Incorrect entity or source attribution |
| Unsupported | Minor | Cannot be verified against reference |
- Frontend: HTML5, CSS3, Vanilla JavaScript
- Styling: Custom CSS with glassmorphism, animations, and responsive design
- Charts: Canvas API for score gauge, bar charts, and donut charts
- Fonts: Inter (UI) + JetBrains Mono (code/data)
- NLP: Custom JavaScript implementation (no external dependencies)
- ✅ Interactive pipeline visualization
- ✅ 3 pre-loaded example datasets with known hallucinations
- ✅ Real-time claim extraction and verification
- ✅ Animated factuality score gauge
- ✅ Claim-by-claim analysis table with status indicators
- ✅ Error distribution and claim status charts
- ✅ Comprehensive evaluation report generation
- ✅ Downloadable text report
- ✅ Fully responsive design
- ✅ Zero external dependencies
# Clone the repository
git clone https://github.com/developerpratima/ai-hallucination-detector.git
# Navigate to project directory
cd ai-hallucination-detector
# Open in browser (no build step required!)
# Option 1: Open directly
start index.html
# Option 2: Use a local server (recommended)
npx serve .
# or
python -m http.server 8000ai-hallucination-detector/
├── index.html # Main application page
├── style.css # Complete design system & styles
├── pipeline.js # Core NLP analysis engine
├── app.js # UI controller, charts, & rendering
└── README.md # Project documentation
Factuality Evaluation • Hallucination Detection • Error Analysis • AI Quality Assurance • LLM Evaluation • Data Validation • NLP Processing • Text Similarity Analysis • Named Entity Recognition • Claim Verification
MIT License — feel free to use and modify.
Pratima Kumari
GitHub: github.com/developerpratima