Adaptive Electricity Load Forecasting (CAISO + Weather)

This project builds an end-to-end, adaptive time-series forecasting system for hourly electricity demand. It combines system load from CAISO with historical weather from Open-Meteo, engineers time-series features, trains a static baseline, and then uses online learning with concept drift adaptation for post-deployment updates. A Streamlit replay demo visualizes live forecasts, drift events, and adaptive vs static performance.

The emphasis is on adaptive forecasting: the model updates online as new data arrives, detects drift, and adapts without full retraining, while remaining stable under non-stationary conditions.

---

Quickstart

1. Build the dataset (CAISO + weather):

   python -m scripts.00_setup_data
   

2. Train static baseline:

   python -m scripts.01_train_static
   

3. Static backtest:

   python -m scripts.02_run_backtest
   

4. Online backtest (adaptive model):

   python -m scripts.03_run_online_backtest
   

5. Streamlit demo:

   streamlit run app/streamlit_app.py
   

---

Data

Sources

• CAISO OASIS: system load actuals + day-ahead forecasts (market_run_id=ACTUAL, DAM).
• Open-Meteo: hourly weather signals (temperature, precipitation, wind).

Dataset Build

The data pipeline lives in src/ingestion/:

• fetch_caiso.py: CAISO API calls (rate-limited, cached, chunked).
• fetch_weather.py: Open-Meteo historical archive.
• build_dataset.py: merges CAISO and weather on hourly timestamps.

Output

The merged dataset is saved to:

data/processed/load_weather.parquet

Key columns (after merge)

• timestamp_utc (UTC, hourly)
• timestamp_local (local time for UI)
• caiso_actual_mw
• caiso_dam_forecast_mw
• temperature_2m, precipitation, wind_speed_10m

Data notes

• All time alignment is done in UTC to avoid DST ambiguity.
• Local timestamps are created only for UI/plots.
• CAISO fetch is cached in data/raw/caiso_cache/ for resumability.

---

Feature Engineering (Phase 2)

Feature generation is in src/features/make_features.py. It is designed for adaptive forecasting and online learning:

Autoregressive features

• Lags: [1, 2, 3, 6, 12, 24, 48, 72, 168]
• Rolling windows: 24h, 48h, 168h (mean/std/min/max)
• EWMAs: 24h, 168h

Time features

• hour, dayofweek, month, weekend
• optional cyclical sin/cos encodings

Exogenous features

• Weather variables (temperature, precipitation, wind)
• Weather lags + rolling stats (past-only)

Target

For horizon h, target is:

y_t = load_{t+h}

---

Models

Static baseline (reference)

• Ridge regression + standardization (scikit-learn)
• Used as a strong non-adaptive benchmark.
• Trained with scripts/01_train_static.py.

Adaptive online models

• Online models are defined in src/models/online_model.py.
• The default production model is DualSpeedEnsemble:
  • Fast model: online MLP (River) updated every step.
  • Slow model: Adaptive Random Forest (ARF) updated every N steps.
  • Predictions are dynamically blended using recent error windows.
  • Stabilized via log-target transform and prediction clipping.

This combination is designed to handle micro-drift (fast MLP) and macro-drift (slow ARF), then blend by recent accuracy.

---

Adaptive System Design

The adaptive path is based on prequential evaluation and drift handling:

1. Predict on the next step.
2. Measure error immediately when the true value is observed.
3. Update the model online (learn_one).
4. Track drift using ADWIN on the error stream.
5. (Optional) Reset/retrain on drift events.

The final system uses dual-speed adaptation (fast + slow) which outperformed other adaptive strategies in MAPE.

---

Experiments

All experiments live in experiments/ and do not affect Streamlit.

Run:

python -m experiments.run_online_mape_experiments

Results (MAPE, MAE, RMSE)

All experiments used:

• horizon = 24
• min_train_size = 168

Variant	MAPE	MAE	RMSE
dual_speed	7.4266	8471.55	11158.48
all_three	15.9383	19191.90	27361.69
drift_plus_blend	16.2410	19506.52	27469.28
no_reset	16.4936	18332.92	22598.49
dma_mlp_arf	16.4936	18332.92	22598.49
buffer_retrain	17.7984	21091.22	25624.85
label_delay_h	17.9384	19946.40	24545.39
drift_retrain	17.9400	21026.18	25534.75
residual_plus_drift	18.1215	21152.54	25701.36
blend_static	30.3199	35834.41	50706.51
reset_only	44.9968	50937.27	69800.60
residual_features	45.1263	51100.98	69916.51

Best MAPE: dual_speed at 7.4266.

Full results are recorded in:

artifacts/experiments/online_mape_experiments.csv

The DualSpeed approach is now the default in both online backtests and the Streamlit demo.

---

Evaluation

Static backtest

• Uses a fixed train/test split (first 80% train, last 20% test).
• Metrics: MAE, RMSE, MAPE.

Online backtest (prequential)

• Predict -> score -> update loop.
• Metrics computed on streaming error.
• Best configuration: dual_speed ensemble.

---

Streamlit Demo

The Streamlit demo (app/streamlit_app.py) provides:

• live forecast vs actual
• drift event markers
• adaptive vs static comparison
• state persistence (resume simulation)

State is saved to:

artifacts/sim_state.pkl

---

Architecture Diagram

flowchart TD
    A[CAISO OASIS] --> B[fetch_caiso.py]
    C[Open-Meteo] --> D[fetch_weather.py]
    B --> E[build_dataset.py]
    D --> E
    E --> F[load_weather.parquet]

    F --> G[make_features.py]
    G --> H[Static Model<br/>Ridge]
    G --> I[Online Model<br/>DualSpeedEnsemble]

    I --> J[Drift Monitor<br/>ADWIN]
    I --> K[Replay Simulator]
    H --> K
    K --> L[Streamlit Demo]

    G --> M[Backtest]
    M --> N[Metrics + Experiments]

Loading

---

Project Structure

app/                  Streamlit UI
data/                 raw + processed data (ignored by git)
experiments/          experimental runners
scripts/              runnable entrypoints
src/
  adaptation/         drift detection + policies
  config/             settings
  evaluation/         backtests + metrics
  features/           feature engineering
  ingestion/          data fetching + dataset build
  models/             static + online models
  serving/            simulator + state persistence

---

Notes

• Defaults (date range, chunking, cache paths) live in src/config/settings.py.
• Online model defaults are in src/models/online_model.py.
• Experiments are fully isolated from Streamlit.