A production-style commercial real estate (CRE) credit-risk modeling pipeline, built as research-support infrastructure for stress testing under macro and mobility shocks.
The pipeline ingests public data from FRED, Google COVID-19 Mobility, and Boston Public Zoning, persists everything to a SQLite database, fits a binary classifier for elevated-stress conditions with explicit threshold control on false-positive rate and recall, and produces ROC / SHAP / commentary artifacts for review.
Scope and disclaimer. This is a personal portfolio project that demonstrates a reviewable stress-testing workflow, model-evaluation process, and explainable risk analytics using public data. All data sources are 100% public (FRED, Google COVID-19 Mobility, Boston Public Zoning). The repository does not reflect any employer's internal data, methodology, model logic, or assumptions. It is not a deployed credit-risk system and should not be used as one.
| Goal | How it shows up |
|---|---|
| Reproducible | One-command pipeline (make run), env-driven config, deterministic seeds, locked deps |
| Production-style | Numbered pipeline stages, modular src/cre_stress/ package, type hints, logging, pytest |
| Auditable | SQLite persistence with explicit schema, model card, methodology doc |
| Bilingual (R + Python) | Python for ML / ingestion; R for ARIMA-based macro stress-scenario forecasting |
| Documented and reviewable | Methodology doc, model card, explicit limitations; optional commentary module produces a draft summary that a human reviewer must verify |
| Reviewable on GitHub | CI runs pytest on push; figures committed so reviewers can audit without running code |
┌─────────────┐ ┌──────────────┐ ┌──────────────┐
│ FRED API │ │ Google │ │ Boston │
│ (FEDFUNDS, │ │ Mobility │ │ Zoning │
│ UNRATE, │ │ (MA) │ │ CKAN API │
│ GDP) │ │ │ │ │
└──────┬──────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
▼ ▼ ▼
┌──────────────────────────────────────────────────────┐
│ src/cre_stress/ingest.py → data/raw/*.csv │
└──────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────┐
│ src/cre_stress/persist.py → SQLite │
└──────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────┐
│ src/cre_stress/features.py │
│ • Quarterly alignment │
│ • Quantile-based stress label (UNRATE q75 OR │
│ retail mobility q25) │
│ • Optional R/ARIMA-forecasted scenario covariates │
└──────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────┐
│ src/cre_stress/models.py │
│ • Logistic Regression / Random Forest │
│ • RandomOverSampler / SMOTE for class imbalance │
│ • Threshold tuning: FPR ≤ τ, recall ≥ ρ │
└──────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────┐
│ src/cre_stress/evaluate.py │
│ • Classification report, ROC, AUC │
│ • SHAP summary + waterfall │
│ • Outputs: outputs/figures/, outputs/tables/ │
└──────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────┐
│ src/cre_stress/commentary.py (optional) │
│ Draft plain-English summary of the classification │
│ report and top SHAP drivers, for human review │
└──────────────────────────────────────────────────────┘
# 1. Install
make install
# 2. Configure: add your FRED API key (free at https://fred.stlouisfed.org/)
cp .env.example .env
# Edit .env
# 3. Run the full pipeline (ingest → persist → train → evaluate)
make run
# 4. Optional: generate AI commentary
make commentary
# 5. Browse results
make dashboard # Streamlit dashboard at http://localhost:8501cre-stress-test/
├── src/cre_stress/ # Python package: ingest, persist, features, models, evaluate, commentary
├── src/dashboard/ # Streamlit dashboard
├── R/ # R companion: ARIMA forecast of stress-scenario macro inputs
├── sql/ # SQLite schema and seed data
├── scripts/ # Pipeline orchestration entry points
├── tests/ # pytest suite
├── data/ # raw/ and processed/ snapshots (public-data only)
├── outputs/ # figures/ and tables/ from the latest run
├── reports/ # Standalone HTML reports
├── docs/ # methodology.md, data_dictionary.md, model_card.md
├── notebooks/ # Cleaned-up exploratory notebooks
└── .github/workflows/ # CI: pytest + ruff
| Metric | Value | Notes |
|---|---|---|
| AUC | 0.73 | Out-of-sample, stratified test split |
| Recall (positive class) | ~78% | Subject to threshold constraint |
| FPR | ≤ 20% | Hard constraint via post-hoc threshold tuning |
See outputs/tables/ for the full classification report, outputs/figures/ for ROC and SHAP plots, and docs/model_card.md for limitations and intended use.
The Python pipeline does the ingestion, persistence, and classification. The R companion (R/forecast_stress_scenarios.R) uses forecast::auto.arima to project the macro drivers — UNRATE, FEDFUNDS, GDP — forward under baseline / adverse / severely-adverse scenarios. The Python pipeline can then re-score using those projected covariates. This mirrors how policy-research teams typically split work: R for the time-series side, Python for ML, both sharing a common data layer.
- docs/methodology.md — full methodology, target definition, sampling strategy, threshold choice
- docs/data_dictionary.md — every column, type, source, transform
- docs/model_card.md — intended use, limitations, ethical considerations
This project translates recurring patterns from stress-testing and model-evaluation work — scenario construction, classification under class imbalance, explainability, and model documentation — into a public, public-data-only portfolio prototype. It uses only public data sources (FRED, Google COVID-19 Mobility, Boston Zoning) and does not replicate any employer's internal methodology, data, or model logic. The goal is to demonstrate reproducible, reviewable analytics with interpretable outputs.
- Synthetic target. The "stress" label is constructed from quantile cuts
on
UNRATEand Google Mobility, not from observed CRE defaults or delinquencies. The classifier predicts a defined macro/mobility regime, not credit losses. - Sample size. Quarterly alignment of the source series yields a small panel; AUC and recall numbers are illustrative and should not be read as out-of-sample performance estimates for any production use.
- No causal identification. The classifier and SHAP plots describe predictive associations, not causal effects of macro shocks on CRE risk.
- Scenario forecasts are univariate. The R companion fits independent
auto.arimamodels per macro driver; cross-driver dependencies are not modeled. - No live data refresh. Snapshots in
data/processed/are committed as reference fixtures and must be regenerated viamake runto refresh. - Commentary module is a draft. The optional commentary module produces a plain-English summary of the latest run via an LLM API; outputs are framed as drafts that a human reviewer must verify against the source figures and tables before sharing.
- Add a small evaluation harness comparing the draft commentary outputs against a hand-graded reference set, so commentary quality is measurable over time.
- Replace the quantile-based label with a published distress indicator (e.g., aggregated CMBS delinquency rates from public sources).
- Add a temporal cross-validation strategy (expanding-window) so reported metrics reflect realistic deployment conditions.
- Add a Quarto briefing template that renders ROC, SHAP, and commentary into a one-page memo.
- Add a
renv.lockfor the R companion so package versions are pinned alongside the Pythonpyproject.toml.
- Bilingual research workflows — Python for ingestion / persistence / classification, R for ARIMA scenario forecasting, shared SQLite layer between them.
- Production-style code organization — installable Python package under
src/cre_stress/, numbered pipeline stages, type hints, structured logging,maketargets, pytest with deterministic fixtures, CI on push. - Data engineering — three public sources merged through a documented schema; SQLAlchemy ORM persistence; idempotent re-runs.
- Classification under class imbalance — RandomOverSampler / SMOTE, explicit threshold tuning to a target FPR and recall, ROC and SHAP diagnostics committed to the repo for review.
- Model documentation — methodology, data dictionary, and a model card spelling out intended use, limitations, and ethical considerations.
- Responsible LLM use — optional commentary module is framed as a
draft for human review, not a finding;
CLAUDE.mdcodifies code-style and review rules for AI-assisted diffs. AI coding tools may support scaffolding, documentation review, and consistency checks, but the workflow logic, assumptions, validation criteria, and final outputs remain human-reviewed.
The pipeline is laid out the way a recurring research-support workflow would be packaged for review: ingestion, persistence, feature construction, modeling, evaluation, and (optional) AI-assisted commentary, with each stage isolated, tested, and regenerable from one command. The classifier itself is a placeholder; the value of the repository is the scaffolding around it — schema discipline, threshold control, explainability, documented limitations, and a separate channel for AI-assisted narrative that a human reviewer must approve before sharing.
For the reusable AI-assisted review templates (prompts, sample QC reports, human-in-the-loop checklist), see llm-research-workflow-assistant.
MIT — see LICENSE.