Benchmarking load forecasters for battery peak shaving.
Live report · Full results · Method
When a commercial battery uses a load forecast to cut demand charges, how much money does forecast error actually cost? This repo answers in dollars rather than MAPE. It simulates a year of battery dispatch at 15-minute resolution under day-ahead forecasters, from a seasonal-naive baseline to Google's TimesFM and Mistral's Timestral, and three dispatch policies, on one synthetic office building and six real metered buildings.
The claim under test: most of the value is in a dispatch policy that degrades gracefully when the forecast is wrong, not in a better forecast. The experiment was built so that claim could fail, and in part it did.
Choosing the dispatch policy is worth $20k a year. Choosing the forecaster is worth $2.8k.
| Policy | With equipment spikes | Smooth (no equipment spikes) |
|---|---|---|
| Threshold + ratchet | 62-67% | 43-54% |
| Threshold | 16-24% | 20-25% |
| Window | 32-35% | 67-79% |
Share of perfect-foresight savings each policy captured, across the six forecasters (synthetic building, one year at 15-minute resolution).
- Policy beats forecaster. With the forecaster fixed (Ridge), the three policies are $20k a year apart; with the policy fixed, the forecasters are $2.8k apart (6.9x).
- The obvious policy is the worst one. Threshold captures 16-24% of perfect-foresight savings; threshold + ratchet captures 62-67%.
- Timing error breaks schedules. A perfect forecast 30 minutes late costs the window policy $38k a year (72% of its savings); the meter-driven policies don't move.
- No policy wins everywhere. Without the equipment spikes, window comes out on top (67-79%) and threshold + ratchet drops to 43-54%.
- Real meters. On six metered buildings (UCI, Portugal, 2014), threshold + ratchet was the best policy on six of six, the policy spread beat the forecaster spread on six of six, and a perfect forecast 30 minutes off cost the window schedule 35-70% of its savings.
- Better forecasts, little money. The foundation models (TimesFM 3.0, TimesFM 2.5 and Timestral 209M) moved annual savings by at most $3.0k against ridge. The most accurate forecaster by MAPE, Timestral 209M (4.0% vs 5.7% for ridge), changed threshold + ratchet savings by +$1.3k a year.
Every table, the real-meter breakdown and what the numbers mean: docs/results.md.
Generated by make report from out/results.json. Edit the code, not these numbers.
- Buildings: a synthetic Boston-area office (weekday occupancy, a summer cooling peak, short equipment spikes), plus six real 15-minute meters from UCI's ElectricityLoadDiagrams20112014.
- Tariff: an illustrative Massachusetts-style time-of-use rate with a monthly demand charge on the single highest 15-minute interval.
- Battery: a 2-hour C&I battery with round-trip losses, state-of-charge limits and off-peak charging.
- Forecasters: seasonal naive, 28-day profile and ridge (refit daily), TimesFM 3.0, TimesFM 2.5 and Timestral 209M (zero-shot on CPU), and a perfect-foresight oracle as the ceiling. Each sees only data up to the midnight before the day it forecasts. TimeGPT is wired in too but needs an API key.
- Policies: window discharges on the forecast's schedule; threshold discharges whenever the meter crosses the day's target; threshold + ratchet also never spends energy shaving below a peak already set this month.
The details, the foundation-model setup and what the tests check are in docs/method.md.
pip install -r requirements.txt
python -m src.evaluate # synthetic building -> out/ (about 15 s)
python -m src.real_meters # six real meters -> out/real/ (about 30 s)
python -m src.report # HTML report and the generated results
streamlit run app.py # interactive dashboardNo GPU or torch needed: the foundation-model forecasts are cached in out/cache/, and the latest results are committed, so the dashboard runs straight after the install. With make: make all, then make app.
| Path | What's there |
|---|---|
src/ |
data, forecasters, tariff, battery dispatch, evaluation, charts, report |
app.py |
Streamlit dashboard |
tests/ |
pytest, including a check that no forecaster can see the future |
data/uci/ |
the six cleaned real meters and how they were picked |
out/ |
results, CSVs and the forecast cache (committed) |
docs/ |
the report served by GitHub Pages, results and method |
The main building is synthetic; the six real meters are a check, not a proof. The tariff is illustrative, batteries don't degrade, dispatch is day-ahead only, and there are no confidence intervals, so forecaster differences of a few points are within noise. The full list is in docs/method.md.
Code: MIT. Real meters: UCI ElectricityLoadDiagrams20112014, CC BY 4.0 (attribution). No model weights are included; TimesFM 3.0's are for non-commercial use only and Timestral's are under Mistral's modified MIT licence.