⚡ AEP Load Forecasting with XGBoost

Hourly electricity-load forecasting on the AEP grid region using calendar + lag features and XGBoost, benchmarked against strong naive baselines and exported as a next-24-hour forecast with an interactive Streamlit demo.

Overview

A compact, end-to-end time-series pipeline:

1. Feature engineering — calendar features + autoregressive lags + rolling means (past-only, no leakage).
2. Baselines — "same hour yesterday", "same hour last week", and a 50/50 blend.
3. Model — gradient-boosted trees (XGBoost) with a time-based train/validation/test split.
4. Forecast — a recursive next-24-hour forecast exported to CSV + plot.
5. Demo — a Streamlit app to inspect the forecast interactively.

Results

Evaluated on the last 30 days (an exact 720-hour window) as an out-of-time test set:

Model	MAE (MW)	RMSE (MW)
Baseline (blend: yesterday + last week)	921.10	1215.43
XGBoost	142.44	184.41

➡️ ~84.5 % lower MAE than the baseline blend.

Approach

• Target: hourly load AEP_MW.
• Features (src/aep_load_forecasting/make_features.py):
  • Calendar: hour, dayofweek, month, is_weekend
  • Lags: lag_1 (1 h), lag_24 (1 day), lag_168 (1 week)
  • Rolling means (shifted, past-only): roll_24_mean, roll_168_mean
• Split (src/aep_load_forecasting/xgb_eval.py): disjoint, time-based partitions — train (everything before the final 60 days), validation (720 h), and test (720 h).
• Model: XGBRegressor(n_estimators=800, learning_rate=0.05, max_depth=6, subsample=0.8, colsample_bytree=0.8).
• 24-hour forecast (src/aep_load_forecasting/forecasting.py, src/aep_load_forecasting/forecast_24h.py): the final model is fit on all data, then steps hour-by-hour, feeding each prediction back in as the next lag_1. The reusable forecast core validates the hourly history and model feature contract before prediction.

More plots

Raw load (last 14 days)

Baseline forecast (last 7 days of the test window)

Project structure

load-forecasting-xgboost/
├── src/
│   └── aep_load_forecasting/
│       ├── plot_load.py        # EDA: plot the last 14 days of load
│       ├── make_features.py    # build calendar + lag + rolling features
│       ├── sample_data.py      # deterministic no-download dataset
│       ├── evaluation.py       # chronological split utilities
│       ├── reporting.py        # metrics and CSV report persistence
│       ├── forecasting.py      # validated recursive forecast utilities
│       ├── demo_data.py        # forecast CSV validation for the demo
│       ├── baseline_eval.py    # yesterday / last-week / blend baselines
│       ├── xgb_eval.py         # train and evaluate XGBoost
│       └── forecast_24h.py     # export a recursive future forecast
├── streamlit_app.py        # interactive demo (reads assets/forecast_next24h.csv)
├── assets/                 # plots + a sample forecast CSV
├── requirements.txt
└── README.md

data/, reports/, and models/ are git-ignored and created at runtime.

Dataset

This project uses the AEP hourly series from the public Hourly Energy Consumption dataset (PJM regions) on Kaggle: https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption

The data is not included in this repo. Download AEP_hourly.csv (columns Datetime, AEP_MW) and place it at data/AEP_hourly.csv.

Quickstart without downloading the dataset

Run the complete pipeline against a deterministic 90-day sample:

aep-demo

This single command creates the sample data, features, baseline and XGBoost metrics, evaluation plots, trained model, and next-24h forecast under the existing ignored data/, reports/, and models/ directories. Use --output-dir to keep every artifact below a different directory. The module form, python -m aep_load_forecasting.demo_pipeline, remains available after installing the project.

Each successful run also writes reports/sample_run_manifest.json. The manifest records the effective pipeline parameters, Python and dependency versions, and the byte size and SHA-256 checksum of every generated artifact. This makes a saved demo run self-describing and independently verifiable.

The equivalent installed console commands are:

aep-sample-data
aep-make-features `
  --input data\sample_aep_hourly.csv `
  --output data\sample_features_aep.csv
aep-baseline-eval `
  --features data\sample_features_aep.csv `
  --metrics reports\sample_baseline_metrics.csv `
  --plot reports\figures\sample_baseline.png
aep-xgb-eval `
  --features data\sample_features_aep.csv `
  --metrics reports\sample_xgb_metrics.csv `
  --plot reports\figures\sample_xgb_evaluation.png
aep-forecast `
  --input data\sample_aep_hourly.csv `
  --features data\sample_features_aep.csv `
  --output reports\sample_forecast.csv `
  --figure reports\figures\sample_forecast.png `
  --model models\sample_xgb.joblib

The generated load combines daily and monthly seasonality, a weekend effect, a small trend, and seeded noise. It is designed for trying the workflow and must not be used to reproduce the reported benchmark metrics.

Setup

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install --editable .

This installs the training and evaluation dependencies, the Streamlit demo, and the six aep-* console commands. Use python -m pip install . instead when an editable source checkout is not needed.

Usage (Windows / PowerShell)

Run from the repository root:

# 1) create the local data folder and add the dataset
mkdir data
#    -> place AEP_hourly.csv in .\data\

# 2) build the validated feature table
aep-make-features

# 3) evaluate baselines and XGBoost (test = last 30 days)
aep-baseline-eval
aep-xgb-eval

# 4) train the final model and export the next-24h forecast
aep-forecast

The evaluation commands create reproducible artifacts automatically:

Command	Metrics	Plot
aep-baseline-eval	reports/baseline_metrics.csv	reports/figures/baseline_evaluation.png
aep-xgb-eval	reports/xgb_evaluation_metrics.csv	reports/figures/xgb_evaluation.png
aep-forecast	reports/forecast_next24h.csv	reports/figures/forecast_next24h.png

The commands create their output directories automatically. The forecast horizon, evaluation windows, boosting rounds, and every input or output path can also be configured from the command line:

aep-forecast `
  --horizon 48 `
  --estimators 400 `
  --interval-coverage 0.9 `
  --calibration-days 30 `
  --output reports\forecast_next48h.csv `
  --figure reports\figures\forecast_next48h.png

Forecast CSVs include the point estimate plus forecast_xgb_lower_MW and forecast_xgb_upper_MW. The interval width is calibrated separately for every forecast hour using recursive rolling-origin errors from the trailing calibration window. The calibration model sees only earlier observations; after calibration, the exported production model is refitted on all available feature rows.

Demo (Streamlit)

streamlit run streamlit_app.py

The app loads the bundled assets/forecast_next24h.csv by default, or lets you upload your own forecast CSV. Uploads must contain hourly Datetime values and a numeric forecast_xgb_MW column; baseline_blend_MW and the paired prediction-interval columns are optional. Invalid files produce a clear error in the app instead of a chart or metric failure.

Data validation

The feature command validates the required columns, parses timestamps and load values strictly, averages duplicate timestamps, and rejects missing hours before building row-based lags. This prevents an unnoticed time gap from turning lag_24 into something other than the same hour on the previous day.

The demo applies the same principle to forecast files: it rejects invalid or duplicate timestamps, non-hourly sequences, and non-finite forecast values before rendering them.

Custom paths and target columns can be supplied without editing the source:

aep-make-features `
  --input data\AEP_hourly.csv `
  --output data\features_aep.csv `
  --target AEP_MW

Run the synthetic validation tests with:

python -m pip install --editable ".[dev]"
python -m ruff check src tests streamlit_app.py
python -m pytest
python -m build

GitHub Actions installs the project, runs the same lint, test, source-compilation and distribution-build checks, and smoke-tests all six console commands on Python 3.10 and 3.12 for every pull request.

Limitations

• Single region (AEP); interval coverage is empirical and can drift when the load process changes. Time-series dependence means the calibrated intervals are not a formal distribution-free coverage guarantee.
• The 24-hour forecast is recursive, so errors can compound over the horizon.
• Features are calendar + lags only — no weather or holiday signals.
• Reported metrics come from a single 30-day out-of-time test window.

License

Released under the MIT License — see LICENSE. © 2026 Amar Akram.