fix: request energy-charts prices on local day boundaries#35
Conversation
|
ah, seems like I was running an older version, I'll rebase the pr. Maybe db97100 already would have fixed the issue I saw today |
The market day runs on local time, but the fetch window was snapped to UTC midnight. For zones east of UTC that cuts the first hours of the day off the request, and energy-charts returns only what was asked for. For BE on 2026-07-23 the store requested start=2026-07-24T00:00:00Z and got 88 quarter hours starting at 02:00 local instead of 96 starting at 00:00. The eight missing quarters were stored as 0.0 and served as real prices. The gap is the size of the UTC offset, so two hours in CEST and one hour in CET. Convert the range to the bidding zone timezone before snapping to midnight. The window can only grow, so nothing that was covered before stops being covered.
504a48f to
656d5d7
Compare
|
Hmm, I do agree that it would probably be better to use local times here, but are you sure this actually fixes your problem? It does, however, explain, why I used 20x4 instead of 24x4 to check for the last days data being 0 - since, as you recognized correctly, accidentally didn't have 24 hours and didn't know why. Not that it really matters. So yes, I think my later patch did fix it. Happy to merge anyway since it makes intentions clearer I think? I assume this is tested properly? |
|
Today I think I first ran into the issue where all actual next day prices where 0's, the bug that was fixed in db97100 . I then tried refetching the data and got this |
|
The 24 -> 25 July day-ahead publication (the first day boundary after deploying) came through cleanly. The window that was broken before now holds real prices: before (unpatched, UTC-midnight fetch): after (this change): No zero blocks in the stored series, and the fetch URL now starts at 2026-07-24T22:00:00Z, i.e. local midnight for the BE zone. |
|
cool, thanks for double checking |

Rebased onto current master.
Problem
fetch_prices_energychartssnaps the requested range to UTC midnight:rstartis already UTC here, but the market day runs on local time. For abidding zone east of UTC this drops the first hours of the day from the request,
and energy-charts returns exactly the window it was asked for.
Reproducible for BE (CEST, UTC+2):
The eight quarter hours from 00:00 to 01:45 local were never fetched. They ended
up in the store as 0.0 and were served by
/pricesas real prices:That is the cheapest slot of the whole horizon, so anything optimising on price
schedules load exactly there while the real price was around 155 EUR/MWh.
This usually repairs itself, because a later and wider fetch covers the same
range before those hours become the present. It only surfaces when the fetched
range starts on the new UTC day. The gap is the size of the UTC offset: two
hours in CEST, one hour in CET.
Relation to #30 /
_is_invalid_zero_dataThis is the same family of problem as #30 (and the zero data mentioned in #34),
but the guard added there only discards a fetch when the last ~20 hours are all
zero, i.e. a full-day blackout. The case here is a partial gap of a few quarter
hours at the day boundary, embedded in otherwise valid data, so the guard does
not catch it. That guard treats the symptom; this change removes the cause, so
those quarter hours are fetched with real prices in the first place. The two are
independent and both useful.
Fix
Convert the range to the bidding zone timezone before snapping to midnight, then
back to UTC for the query.
region.get_timezone_info()is already usedelsewhere in this class. The window can only grow, so nothing that was covered
before stops being covered.
Observed on zone BE, 2026-07-23.