PulseForecast AI

Production-style revenue intelligence and forecasting platform for a fitness studio.

SafeX Skills Development Internship · Group 38 · AI/ML

PulseForecast AI turns the official SafeX assignment — AI Sales Forecasting Model for a Fitness Studio — into a portfolio-grade ML/backend/MLOps system rather than a single notebook. The project covers data quality, feature engineering, expanding-window backtesting, champion/challenger model selection, empirical forecast uncertainty, explainability, scenario planning, persistence, monitoring, API delivery, and an operator dashboard.

Data honesty: the bundled dataset is synthetic and reproducible. Every performance number below is measured on that synthetic project dataset; none is presented as commercial client performance.

Product preview

Verified engineering evidence

Capability	Current implementation
Dataset	974 daily synthetic observations · 2024-01-01 → 2026-08-31
Evaluation	5-fold expanding-window / walk-forward backtest, 56-day validation folds
Champion selection	Ridge, Gradient Boosting, HistGradientBoosting, Random Forest, Extra Trees
Selected champion	Ridge, selected automatically by mean walk-forward MAPE
Walk-forward MAPE	2.935% ± 0.099%
Walk-forward MAE	PKR 4,334 ± 165/day
Walk-forward RMSE	PKR 5,238 ± 257/day
Walk-forward R²	0.875 ± 0.056
Seasonal lag-7 baseline	10.297% MAPE
Calibration	280 out-of-fold residuals
Forecast uncertainty	empirical residual calibration with daily + aggregate P80/P95 ranges
Explainability	exact standardized contribution decomposition for Ridge; Tree SHAP path for tree champions
Data quality	100/100, no missing dates or duplicate dates in bundled dataset
Model health	healthy, trend-aware mean PSI 0.080, retraining not recommended
Persistence	SQLAlchemy · SQLite local default · PostgreSQL-ready · Alembic migration
Observability	structured JSON logs · Prometheus metrics · health/readiness · optional OpenTelemetry
Security controls	optional API key · bounded Pydantic inputs · request IDs · security headers · demo rate limiting
Delivery	Dockerfile · Docker Compose · PostgreSQL · Prometheus · Grafana · GitHub Actions CI
Automated verification	97 passing tests · 82% measured code coverage

Architecture

flowchart LR
  A[Daily studio data] --> B[Schema validation + data quality]
  B --> C[Feature pipeline]
  C --> D[5-fold expanding-window backtest]
  D --> E[Champion / challenger selection]
  D --> F[Out-of-fold residual calibration]
  E --> G[Final champion model]
  G --> H[Conditional scenario forecast]
  F --> H
  H --> I[P80 / P95 uncertainty]
  H --> J[Explainability]
  H --> K[FastAPI /api/v1]
  K --> L[(SQLite / PostgreSQL)]
  K --> M[Operator dashboard]
  K --> N[Prometheus / Grafana]
  K --> O[Optional AI advisor]
  B --> P[Trend-aware PSI]
  P --> Q[Model health / retraining decision]

Loading

Why Ridge won

The system does not force a complex model just to sound advanced. Five model families compete under the same chronological backtest. Ridge achieved the lowest mean MAPE on the bundled synthetic dataset, so it becomes the champion. This demonstrates an important production ML principle: select the simplest model that wins on unseen time windows.

Professional ML decisions demonstrated

• Chronological validation instead of random splitting. Future observations never leak into a training fold.
• Champion/challenger evaluation across five model families and five later time windows.
• Explicit seasonal baseline using lag-7 revenue.
• Out-of-fold residual calibration for uncertainty instead of an arbitrary ± percentage band.
• Conditional forecasting: marketing, promotion, discount, and sentiment are future assumptions supplied by the operator, not magically known future facts.
• Explainability follows the champion: exact contribution decomposition for Ridge; SHAP path when a tree model wins.
• Trend-aware drift monitoring so ordinary long-run growth is not automatically mislabeled as data drift.
• LLMs are not the forecaster. OpenAI/Anthropic/LangChain are optional explanation layers; the numerical forecast remains deterministic and testable offline.

Quick start

python -m venv .venv
# Windows
.venv\Scripts\activate
# Linux/macOS
source .venv/bin/activate

pip install -r requirements-dev.txt
python scripts/generate_data.py
python scripts/train_model.py
pytest
python scripts/create_reports.py
uvicorn api.main:app --reload

Open:

• Dashboard: http://127.0.0.1:8000
• OpenAPI docs: http://127.0.0.1:8000/docs
• Health: http://127.0.0.1:8000/api/v1/health
• Readiness: http://127.0.0.1:8000/api/v1/readiness
• Prometheus metrics: http://127.0.0.1:8000/metrics

Production-style local stack

docker compose up --build

This starts:

• FastAPI on :8000
• PostgreSQL
• Prometheus on :9090
• Grafana on :3000 with a provisioned PulseForecast AI - Operations dashboard

Core API

Method	Endpoint	Purpose
GET	/api/v1/health	liveness
GET	/api/v1/readiness	data/model readiness
GET	/api/v1/summary	dataset, model, leaderboard, quality and health
GET	/api/v1/data-quality	completeness, gaps, duplicates, freshness, quality score
GET	/api/v1/model-health	validation performance + trend-aware PSI
GET	/api/v1/history	recent observed business history
POST	/api/v1/forecast	1–90 day scenario forecast with P80/P95 ranges
POST	/api/v1/scenarios/compare	compare 2–6 operating scenarios
GET	/api/v1/explainability	local next-day + global model drivers
GET	/api/v1/forecast-runs	persisted forecast evidence
POST	/api/v1/insights	conservative OpenAI/Anthropic/offline explanation
POST	/api/v1/sentiment	member-review sentiment
POST	/api/v1/similar-periods	business-context retrieval
POST	/api/v1/advisor	retrieval + forecast advisor workflow

Example:

curl -X POST http://127.0.0.1:8000/api/v1/forecast \
  -H "Content-Type: application/json" \
  -d '{
    "horizon_days": 30,
    "persist": false,
    "scenario": {
      "marketing_multiplier": 1.15,
      "promotion_active": true,
      "discount_percent": 10,
      "sentiment_delta": 0.03
    }
  }'

Repository map

api/                         FastAPI application + operational middleware
app/static/                  dependency-free operator dashboard
config/                      Prometheus + Grafana provisioning
migrations/                  Alembic schema migration
data/raw/                    reproducible synthetic dataset
docs/                        architecture, MLOps, security, model card, SafeX pack
notebooks/                   executed EDA / model-evaluation notebook
reports/                     metrics, backtests, health evidence, screenshots
scripts/                     data, training, reports and CLI utilities
src/fitness_forecast/
  backtesting.py             expanding-window validation
  data.py                    dataset generation, validation, fingerprint
  data_quality.py            quality scoring
  drift.py                   trend-aware PSI + model health
  explainability.py          Ridge decomposition / Tree SHAP path
  features.py                calendar + lag + rolling features
  forecast.py                recursive conditional forecasting
  model.py                   champion/challenger training
  persistence.py             SQLAlchemy forecast/model run records
  uncertainty.py             empirical residual P80/P95 intervals
  tracking.py                local + optional MLflow experiment tracking
  observability.py           JSON logs + Prometheus + optional OTEL
  insights.py                OpenAI/Anthropic/offline explanation
  langchain_workflow.py      optional LangChain orchestration
  sentiment.py               Hugging Face/offline sentiment
  vector_store.py            ChromaDB/TF-IDF retrieval
tests/                       97 automated tests

Documentation

• Architecture
• Model card
• MLOps design
• Deployment guide
• Security notes
• Technical write-up
• SafeX submission pack

Optional integrations

pip install -r requirements-ai.txt
pip install -r requirements-mlops.txt
pip install -r requirements-observability.txt

Environment switches:

• PULSEFORECAST_API_KEY=... protects application endpoints through X-API-Key.
• DATABASE_URL=postgresql+psycopg://... switches persistence from SQLite to PostgreSQL.
• ENABLE_MLFLOW=1 enables MLflow experiment logging if installed.
• ENABLE_OTEL=1 enables FastAPI OpenTelemetry instrumentation if installed.

Testing and reproducibility

pytest
pytest --cov=src/fitness_forecast --cov=api --cov-report=term-missing
python -m compileall -q api src scripts tests

The current local verification run produced:

• 97 / 97 tests passed
• 82% measured coverage across src/fitness_forecast and api
• successful model training on the deterministic 974-row dataset
• successful Alembic initial migration against a fresh SQLite database

CI repeats dataset generation, training, linting, tests/coverage, PostgreSQL migration verification, source compilation, and a Docker image build.

SafeX mapping

The project still directly satisfies the SafeX brief:

• working numerical prediction
• clear sample input/output
• bad-input handling
• technical write-up
• more than 10 test cases
• GitHub-ready code and README
• screenshots
• 5–10 minute explanation script

See docs/SAFE_X_SUBMISSION.md and docs/VIDEO_SCRIPT.md.

CV-safe project wording

Built PulseForecast AI, a production-style revenue forecasting platform using Python, scikit-learn, FastAPI and SQLAlchemy; implemented 5-fold expanding-window backtesting across five model families, automatically selecting a Ridge champion with 2.94% mean MAPE and 0.875 R² on a 974-row synthetic fitness-studio dataset, versus 10.30% MAPE for a weekly seasonal baseline.

Added empirical P80/P95 forecast intervals, scenario simulation, exact Ridge contribution explanations, trend-aware data-drift/model-health monitoring, SQLAlchemy/Alembic persistence, Prometheus observability, Docker/GitHub Actions CI and optional AI/RAG modules; verified with 97 automated tests and 82% measured coverage.

Do not remove “synthetic dataset” when quoting the model metrics.

License

MIT — see LICENSE.