AI-Driven PDF PII Redaction System
**Author:** Esrom Ghebreweldi
**Date:** December 13, 2025
## 🛡️ Project Overview
This project implements an automated, end-to-end pipeline for detecting and redacting Personally Identifiable Information (PII) from PDF documents. It utilizes a **hybrid detection strategy** to ensure high recall and precision across both digital and scanned documents.
### Key Features
* **Hybrid Detection:** Combines **Regex** (for rigid patterns like Email, SSN, Phone) with a **Fine-Tuned DistilBERT NER Model** (for contextual entities like Names, Addresses).
* **OCR Support:** Integrates **Tesseract-OCR** to handle scanned (image-based) PDFs.
* **Secure Redaction:** Uses **Text Replacement** to regenerate PDFs, ensuring sensitive data is permanently removed (not just visually hidden).
* **Interactive UI:** Includes a **Streamlit Web Interface** for easy drag-and-drop testing.
---
## Repository Structure
```text
PDF_PII_Redaction_System/
│
├── README.md # Project documentation
├── requirements.txt # Python dependencies
│
├── data/
│ ├── pdf/ # Place input PDFs here
│ ├── pdf_redacted/ # Redacted PDFs will be saved here
│ └── processed/ # Tokenized dataset (generated by script)
│
├── models/
│ └── pii_ner/ # Saved DistilBERT model files
│
├── plots/ # Evaluation graphs (Loss, Confusion Matrix)
│
└── src/
├── app.py # Streamlit Web Application
├── prepare_dataset.py # Tokenizes AI4Privacy dataset
├── train_ner.py # Fine-tunes the DistilBERT model
├── evaluate_and_plot.py # Generates Confusion Matrix & Loss Curve
├── redactor.py # Core PII detection logic (Hybrid)
├── pdf_utils.py # PDF text extraction & OCR logic
└── tokenization.py # Helper utils for label alignment
Installation & Setup
1. Install Python Dependencies
Run the following command to install the required libraries:
code
pip install -r requirements.txt
2. Install Tesseract OCR (Required for Scanned PDFs)
This project requires the Tesseract engine installed on your system to process scanned documents.
* macOS: brew install tesseract
Usage Instructions
Phase 1: Model Training
Before running the redaction system, you must prepare the data and train the NER model.
1. Prepare the Dataset:Downloads the AI4Privacy PII Masking dataset, filters for English, and aligns BIO tags.
code
python src/prepare_dataset.py
2. Train the Model:Fine-tunes the DistilBERT model. This will run for 4 epochs and save artifacts to models/pii_ner.
code
python src/train_ner.py
Note: Training may take 5-10 minutes depending on hardware (CPU vs GPU).
Phase 2: Evaluation
Generate performance metrics, a Training Loss Curve, and a Confusion Matrix. Images are saved to the plots/ folder.
code
python src/evaluate_and_plot.py
Phase 3: Running the Application
The best way to demo the project is using the Streamlit Web Interface.
1. Navigate to the source directory:code cd src
2. Launch the app:code streamlit run app.py
3. A browser window will open (usually at http://localhost:8501). Drag and drop any PDF file to test the redaction in real-time.
Performance Results
The system was evaluated on a held-out test set (20% split) of the AI4Privacy dataset.
Metric Score Notes
Precision 78.91% Measures avoidance of over-redaction.
Recall 84.52% Critical for privacy; measures how much PII was caught.
F1-Score 81.62% Overall model balance.
Detailed graphs for Training Loss and Confusion Matrices can be found in the plots/ directory.
Limitations
* OCR Dependence: Redaction quality for scanned documents depends heavily on image clarity. If Tesseract misreads a character (e.g., '5' as 'S'), detection may fail.
* Processing Speed: Enabling OCR significantly increases inference time per page compared to native text extraction.
* GPU Recommendation: While the system runs on CPU, a GPU is recommended for faster training and batched inference.
