Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🔍 AI Hallucination & Factuality Detection System

A comprehensive, interactive pipeline for detecting, analyzing, and categorizing AI-generated hallucinations with evidence-based verification and factuality scoring.

By Pratima Kumari


🚀 Overview

This project demonstrates AI Quality Assurance — specifically the ability to evaluate and validate AI-generated content against ground-truth reference information. Rather than merely using AI, this system focuses on quality control of AI output.

The system implements a complete Hallucination Detection Pipeline that:

  1. Extracts Claims from AI-generated text using NLP heuristics
  2. Verifies Evidence by cross-referencing claims against ground-truth data
  3. Scores Factuality using weighted multi-method similarity analysis
  4. Categorizes Errors into specific hallucination types with severity levels
  5. Generates Reports with actionable findings and recommendations

🏗️ Architecture

┌──────────────┐     ┌──────────────────┐     ┌────────────────────┐
│  LLM Output  │────▶│ Claim Extraction │────▶│ Evidence           │
│  (Input)     │     │ (NLP Tokenizer)  │     │ Verification       │
└──────────────┘     └──────────────────┘     └────────┬───────────┘
                                                       │
┌──────────────┐     ┌──────────────────┐     ┌────────▼───────────┐
│  Evaluation  │◀────│ Error            │◀────│ Factuality         │
│  Report      │     │ Categorization   │     │ Scoring            │
└──────────────┘     └──────────────────┘     └────────────────────┘

🧠 NLP Methodology

Claim Extraction

  • Sentence tokenization with abbreviation handling
  • Factual assertion detection (numbers, proper nouns, dates, factual verbs)
  • Claim type classification: quantitative, temporal, attributive, factual

Similarity Analysis (Multi-Method)

  • Jaccard Similarity: Word set intersection over union
  • Weighted Term Overlap: Length-weighted content word matching
  • N-gram Overlap: Bigram and trigram sequence matching
  • Named Entity Matching: Proper noun, number, date, and measurement comparison
  • Combined Score: Weighted fusion of all methods

Error Taxonomy

Error Type Severity Description
Fabrication Critical Completely made-up information
Contradiction Critical Directly conflicts with reference
Exaggeration Major Inflated numerical values
Misattribution Major Incorrect entity or source attribution
Unsupported Minor Cannot be verified against reference

💻 Tech Stack

  • Frontend: HTML5, CSS3, Vanilla JavaScript
  • Styling: Custom CSS with glassmorphism, animations, and responsive design
  • Charts: Canvas API for score gauge, bar charts, and donut charts
  • Fonts: Inter (UI) + JetBrains Mono (code/data)
  • NLP: Custom JavaScript implementation (no external dependencies)

🎯 Features

  • ✅ Interactive pipeline visualization
  • ✅ 3 pre-loaded example datasets with known hallucinations
  • ✅ Real-time claim extraction and verification
  • ✅ Animated factuality score gauge
  • ✅ Claim-by-claim analysis table with status indicators
  • ✅ Error distribution and claim status charts
  • ✅ Comprehensive evaluation report generation
  • ✅ Downloadable text report
  • ✅ Fully responsive design
  • ✅ Zero external dependencies

📦 Getting Started

# Clone the repository
git clone https://github.com/developerpratima/ai-hallucination-detector.git

# Navigate to project directory
cd ai-hallucination-detector

# Open in browser (no build step required!)
# Option 1: Open directly
start index.html

# Option 2: Use a local server (recommended)
npx serve .
# or
python -m http.server 8000

📁 Project Structure

ai-hallucination-detector/
├── index.html          # Main application page
├── style.css           # Complete design system & styles
├── pipeline.js         # Core NLP analysis engine
├── app.js              # UI controller, charts, & rendering
└── README.md           # Project documentation

🔑 Resume Keywords

Factuality Evaluation • Hallucination Detection • Error Analysis • AI Quality Assurance • LLM Evaluation • Data Validation • NLP Processing • Text Similarity Analysis • Named Entity Recognition • Claim Verification

📄 License

MIT License — feel free to use and modify.

👩‍💻 Author

Pratima Kumari
GitHub: github.com/developerpratima

About

End-to-end NLP-driven AI Hallucination & Factuality Detection pipeline with claim extraction, evidence verification, and factuality scoring.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages