An enterprise-grade platform for validating, profiling, monitoring, and improving AI datasets used for machine learning and Large Language Model (LLM) applications.
AI Data Quality Validator helps AI engineers ensure that datasets are clean, consistent, complete, and production-ready before model training or evaluation.
The platform automatically detects missing values, duplicate records, inconsistent annotations, schema violations, formatting issues, label imbalance, data drift, and other quality problems.
Designed for AI teams, the application supports structured, semi-structured, and annotation datasets while providing detailed validation reports and quality metrics.
Validate datasets before training or deployment.
Supported checks
- Missing values
- Duplicate records
- Invalid data types
- Schema validation
- Required fields
- Null detection
- Empty values
- Invalid formats
Validate annotation quality for AI datasets.
Supports
- Classification
- Named Entity Recognition
- Object Detection
- Image Segmentation
- OCR
- Text Classification
- LLM Evaluation Datasets
Automatically verify dataset structure.
Dataset
↓
Schema Validation
↓
Field Validation
↓
Data Type Validation
↓
Quality Report
Generate statistics including
- Row count
- Column count
- Missing percentage
- Unique values
- Cardinality
- Distribution
- Outliers
Detect
- Exact duplicates
- Near duplicates
- Duplicate prompts
- Duplicate responses
- Similar records
Visualize label balance.
Dataset
↓
Class Distribution
↓
Imbalance Detection
↓
Recommendations
Compare datasets over time.
Training Data
↓
Production Data
↓
Distribution Comparison
↓
Drift Report
Generate an overall quality score based on
- Completeness
- Consistency
- Accuracy
- Validity
- Uniqueness
- Integrity
Export reports as
- CSV
- Excel
- JSON
Web Dashboard
│
▼
FastAPI Backend
┌─────────────┼─────────────┐
▼ ▼ ▼
Dataset API Validation Engine Reports
│ │ │
└─────────────┼─────────────┘
▼
Data Processing
▼
Quality Analysis
▼
Validation Report
- React
- TypeScript
- Tailwind CSS
- Python
- FastAPI
- Pydantic
- Pandas
- NumPy
- Great Expectations
- PostgreSQL
- SQLite
- Plotly
- Chart.js
- Docker
- GitHub Actions
ai-data-quality-validator/
├── app/
│ ├── api/
│ ├── validators/
│ ├── profiling/
│ ├── reports/
│ ├── analytics/
│ ├── services/
│ ├── database/
│ ├── models/
│ └── utils/
│
├── datasets/
├── reports/
├── tests/
├── frontend/
├── docs/
├── scripts/
└── README.md
Clone the repository
git clone https://github.com/ForwardAva/ai-data-quality-validator.gitNavigate to the project
cd ai-data-quality-validatorCreate a virtual environment
python -m venv .venvActivate the environment
Windows
.venv\Scripts\activateLinux / macOS
source .venv/bin/activateInstall dependencies
pip install -r requirements.txtCreate a .env file.
DATABASE_URL=
JWT_SECRET=
REPORT_OUTPUT=reports/
MAX_UPLOAD_SIZE=100MBRun the backend
uvicorn app.main:app --reloadRun the frontend
npm install
npm run devOpen your browser
http://localhost:3000
- CSV
- JSON
- JSONL
- Excel
- Parquet
- TSV
- Dataset Validation
- Annotation Validation
- Data Profiling
- Schema Validation
- Data Drift Detection
- Duplicate Detection
- Quality Reports
- Analytics Dashboard
- REST API
- Dataset Upload API
- Validation API
- Report API
- Docker
- Testing
- CI/CD
- Deployment
- Roadmap
- Contributing
- License
Run the API server
uvicorn app.main:app --reloadBase URL
http://localhost:8000
Swagger Documentation
http://localhost:8000/docs
ReDoc
http://localhost:8000/redoc
POST /api/v1/datasets/upload
Supported formats
- CSV
- JSON
- JSONL
- Excel
- Parquet
- TSV
Response
{
"dataset_id": "b52d9f41",
"status": "uploaded"
}POST /api/v1/validation/run
Request
{
"dataset_id": "b52d9f41",
"validation_profile": "standard"
}Response
{
"status": "completed",
"quality_score": 96.8
}GET /api/v1/datasets/{id}/profile
Returns
- Row count
- Column count
- Missing values
- Duplicate rows
- Column statistics
- Data types
GET /api/v1/reports/{id}
Returns a complete validation report.
POST /api/v1/reports/export
Supported formats
- CSV
- Excel
- JSON
POST /api/v1/drift/analyze
Compare two datasets.
Request
{
"baseline_dataset":"dataset_a",
"current_dataset":"dataset_b"
}POST /api/v1/annotations/validate
Supported tasks
- Classification
- NER
- OCR
- Image Classification
- Object Detection
- LLM Evaluation
- Question Answering
Upload Dataset
│
▼
Schema Validation
│
▼
Data Profiling
│
▼
Quality Checks
│
▼
Validation Report
│
▼
Export Results
The platform validates
- Missing Values
- Duplicate Records
- Invalid Data Types
- Invalid Labels
- Empty Columns
- Invalid Dates
- Schema Violations
- Label Imbalance
- Data Drift
- Outliers
app/
├── api/
├── analytics/
├── database/
├── profiling/
├── reports/
├── schemas/
├── services/
├── validators/
├── middleware/
├── utils/
└── main.py
Run all tests
pytestRun coverage
pytest --cov=appRun validation tests
pytest tests/validationBuild
docker build -t ai-data-quality-validator .Run
docker run -p 8000:8000 ai-data-quality-validatorDocker Compose
docker compose up -d| Metric | Result |
|---|---|
| Dataset Upload | < 2 sec |
| Validation Speed | 100K+ rows/min |
| Duplicate Detection | < 1 sec |
| API Latency | < 400 ms |
| Concurrent Users | 500+ |
Replace these placeholders with your own screenshots.
docs/images/dashboard.png
docs/images/upload.png
docs/images/profile-report.png
docs/images/validation-results.png
docs/images/data-drift.png
docs/images/analytics.png
- Dataset Upload
- Schema Validation
- Data Profiling
- Duplicate Detection
- Quality Reports
- REST API
- Scheduled Validation
- Batch Processing
- Dataset Versioning
- Team Collaboration
- Email Notifications
- AI Data Cleaning
- Automatic Error Correction
- Active Learning Integration
- Real-Time Monitoring
- Enterprise Workspaces
- Data Lineage
- ML Dataset Registry
- Cloud Storage Integration
Contributions are welcome.
-
Fork the repository.
-
Create a feature branch.
git checkout -b feature/my-feature- Commit your changes.
git commit -m "Add custom validation rule"- Push your branch.
git push origin feature/my-feature- Open a Pull Request.
Never commit sensitive datasets, API keys, or credentials.
Use environment variables for secrets and follow secure data handling practices.
This project is licensed under the MIT License.
See the LICENSE file for details.
Oscar Garcia
AI Quality Engineer • LLM Evaluation Specialist • AI Application Developer
GitHub
- AI-Based Data Cleaning
- Custom Validation Rules
- LLM Dataset Validation
- Prompt Dataset Analysis
- Annotation Consensus Scoring
- Data Version Control
- Automated Dataset Monitoring
- Cloud Storage Connectors
- Model Readiness Score
- Enterprise Audit Logs
If you find this project useful, consider giving it a ⭐ on GitHub.
Bug reports, feature requests, and pull requests are always welcome.
Built with ❤️ using FastAPI, React, Pandas, Great Expectations, and Python.