Fintech Transaction Intelligence: Credit Default Prediction System

A modular, production-ready Machine Learning system for credit default risk estimation and transaction intelligence on large-scale financial data (458,913 customers, 5.5M transactions), based on the American Express Default Prediction benchmark.

---

🏛️ System Architecture

flowchart TD
    Raw["Raw Transactions (5.5M rows CSV / Parquet)"] --> Ingestion["1. Streaming Ingestion & Downcasting<br/>(Polars Streaming, -70% RAM)"]
    Ingestion --> Features["2. Feature Engineering Pipeline<br/>(Last, Mean, Min, Max, Trends, NUnique)"]
    Features --> Store["3. Feature Matrix Store<br/>(1,002 Clean Features in Parquet)"]
    
    Store --> Selector["4. Feature Selection & Ablation<br/>(Group Matching, Importance Pruning)"]
    Selector --> Trainer["5. Model Training & Validation Engine<br/>(LightGBM, XGBoost, CatBoost on CUDA)"]
    
    Trainer --> Tracker["6. Experimentation & Metrics Tracker<br/>(AMEX Metric, Gini, Top-4% Recall)"]
    Trainer --> Exporter["7. High-Speed ONNX Compiler<br/>(models/onnx_exporter.py)"]
    
    Tracker --> Models["8. Serialized Model Artifacts<br/>(artifacts/models/lightgbm.pkl & .onnx)"]
    Exporter --> Models
    
    Models --> API["9. Production Multi-Worker REST API Server<br/>(FastAPI + Gunicorn / Uvicorn: /predict, /predict/batch)"]
    Models --> ONNX["10. Ultra-Low Latency ONNX Inference<br/>(0.025 ms per customer)"]
    Models --> Predictor["11. Production Batch Inference CLI<br/>(458k customers scored in 20s)"]
    Models --> Diagnostics["12. Error Diagnostics & Calibration<br/>(Risk Banding, Confusion Cohort Profiling)"]

Loading

---

⚡ High-Speed ONNX Runtime & Production Serving

Inference Engine	Single Customer Latency	Throughput Speedup	Use Case
Standard Baseline	$797.0\text{ ms}$	$1\times$	Cold start initial call
Warmed Up LightGBM (C++)	$3.8\text{ ms}$	$\mathbf{210\times}$	General batch scoring
ONNX Runtime (C++ Engine)	$0.025\text{ ms}$ (25 $\mu$s)	$\mathbf{31,000\times}$	Real-time card transaction authorization

---

📊 Experimental Results & Leaderboard

The project executed systematic ablation, feature importance profiling, dimensionality reduction, hyperparameter tuning, and multi-seed stability validation:

Rank	Experiment	Core Innovation	Features	AMEX Metric	$\mathbf{\Delta AMEX}$	Gini	Top-4% Recall
🥇 1	EXP-013_seed777	Multi-seed stability test 3	1,002	0.794174	$\mathbf{+0.006460}$	0.923604	0.664745
🥈 2	EXP-012_seed123	Multi-seed stability test 2	1,002	0.793783	$+0.006069$	0.924075	0.663492
🥉 3	EXP-011_slow_learning	lr=0.03, 1000 trees, leaves=45, reg_L2=2.0	1,002	0.791023	$+0.003309$	0.923019	0.659028
4	EXP-010_deep_regularized	depth=8, leaves=63, L1=0.1, L2=1.0	1,002	0.790375	$+0.002661$	0.922433	0.658317
5	EXP-003_no_ratios	Feature ablation: dropped *_ratio_last_mean	1,002	0.788848	$+0.001134$	0.921594	0.656101
6	EXP-008_top_500	Top 500 features (56.8% reduction, 2.3x faster)	500	0.788057	$+0.000343$	0.921643	0.654470
7	EXP-001_baseline	Full raw feature matrix baseline	1,158	0.787714	$0.000000$	0.921375	0.654052

Key Findings:

• Removing noisy ratios (*_ratio_last_mean): Eliminated division-by-near-zero outliers, yielding $+0.001134$ in AMEX score and accelerating training by 25%.
• Top Feature Drivers: P_2_last (Credit score profile) accounts for 60.28% of total tree gain. The last statement group contributes 81.48% of overall predictive power.
• Multi-Seed Stability: Cross-seed mean performance reached 0.792993 ($\sigma = 0.0017$).

---

💾 Dataset & Download Links

The pipeline is trained and evaluated on the American Express - Default Prediction benchmark dataset:

Dataset Resource	Format	Size	Description	Link
Official Kaggle Competition	CSV (Raw)	~50 GB	Raw multi-month statement transaction event logs and binary default labels.	Kaggle Competition Data
Optimized Parquet Mirror (Raddar)	Parquet	~4 GB	Memory-optimized integer/float typed columns for ultra-fast reading.	Kaggle Parquet Dataset

Dataset Characteristics:

• Customers: $458,913$ unique customer accounts.
• Transactions: $\approx 5.5\text{M}$ historical monthly billing statements.
• Features (188 raw):
  • $D_*$ (Delinquency variables: payment delays, delinquencies)
  • $S_*$ (Spend variables: statement amounts, revolving spends)
  • $P_*$ (Payment variables: payments, profile metrics like P_2)
  • $B_*$ (Balance variables: balance amounts, credit card balances)
  • $R_*$ (Risk variables: risk scores, credit indicators)
• Target: Binary credit card default event within an 18-month performance window.

Download via CLI:

# Download official Kaggle competition raw data
python main.py download-data --source competition

# Or download lightweight pre-converted Parquet
python main.py download-data --source dataset --dataset raddar/amex-data-parquet-converter

---

🚀 Universal CLI Entry Point (main.py)

All tasks are accessible through the root main.py entry point:

# 1. Download & Unpack Dataset (Kaggle or Direct URL)
python main.py download-data --source competition

# 2. Start FastAPI REST Server with Multi-Worker Scaling
python main.py serve --host 0.0.0.0 --port 8000 --workers 4

# 3. Export Model to Ultra-Fast ONNX Graph Format
python main.py export-onnx --model artifacts/models/lightgbm.pkl --output artifacts/models/lightgbm.onnx

# 4. Run Preprocessing & Feature Engineering
python main.py build-features --config configs/data_config.yaml

# 5. Train Model
python main.py train --config configs/training_config.yaml

# 6. Run Tracked Experiment with Feature Selection
python main.py experiment --config configs/training_config.yaml --model lightgbm

# 7. Run Batch Inference
python main.py predict --model artifacts/models/lightgbm.pkl --features data/processed/train_features.parquet --output predictions.parquet

# 8. Compare Completed Experiments
python main.py compare --metric amex_metric

# 9. Audit Feature Matrix Health
python main.py audit-features

# 10. Analyze Feature Importance
python main.py feature-importance --model artifacts/models/lightgbm.pkl

---

📁 Repository Structure & Documentation

fintech-transaction-intelligence/
├── main.py                           # Universal CLI entry point
├── Dockerfile                        # Multi-stage production container with OpenMP
├── docker-compose.yml                # Docker compose orchestration
├── configs/                          # Experiment & training YAML configurations
│   ├── experiments/                  # Reproducible experiment configs (EXP-001 - EXP-014)
│   ├── gunicorn_conf.py              # Production Gunicorn master process configuration
│   ├── data_config.yaml              # Ingestion & preprocessing parameters
│   └── training_config.yaml          # Baseline training config (CUDA enabled)
├── data/                             # Raw & processed Parquet feature matrices
├── artifacts/                        # Serialized models (.pkl / .onnx), importance tables
├── reports/                          # Detailed analytical & research reports
├── scripts/                          # Dedicated CLI scripts (see scripts/README.md)
│   ├── score_real_customers.py       # Live evaluation on real bank customer dataset
│   ├── test_api_client.py            # API automated verification client
│   ├── run_experiment.py             # Config-driven experiment runner
│   ├── train.py                      # Model training entrypoint
│   ├── compare_experiments.py        # Leaderboard comparison tool
│   ├── analyze_feature_importance.py # Feature gain & group diagnostics
│   ├── analyze_errors.py             # Error analysis & cohort profiling
│   └── predict.py                    # High-throughput batch inference CLI
├── src/                              # Core package source code (see src/README.md)
│   ├── api/                          # FastAPI serving layer (see src/api/README.md)
│   ├── core/                         # Foundation types, protocols, exceptions
│   ├── data/                         # Polars streaming loaders & memory downcasting
│   ├── dataset/                      # Train/val splitters & data contracts
│   ├── diagnostics/                  # Error analysis & risk band segmenters
│   ├── experiments/                  # Experiment tracking & leaderboard engine
│   ├── features/                     # Aggregators, selectors, RFM, importance analyzer
│   ├── inference/                    # ONNXPredictor & Predictor with auto feature alignment
│   ├── models/                       # LightGBM, XGBoost (CUDA), CatBoost, ONNX Exporter
│   ├── monitoring/                   # Drift detection & PSI metrics
│   └── training/                     # Cross-validation orchestrator
└── tests/                            # Comprehensive Pytest test suite (277 passed)

---

🧪 Testing & Code Quality

# Run complete test suite
python -m pytest -v

All 277 unit and integration tests verify data contracts, feature extraction idempotence, model wrappers, metrics calculation, ONNX export/inference, API routes, and CLI inference.