Commit e01f157
Expire the index cache on each provider's publishing cadence
The AMOC cache I added had no expiry, so it would have gone stale silently as
RAPID published new releases. Fixing that properly meant giving every index the
same treatment rather than special-casing one.
Each entry now records how its source updates: monthly for the NOAA indices,
annual for RAPID, none for LCR, which was published with a paper and ends at
2014. That cadence sets how long a cached copy may be reused - a week, a month,
or forever - so a living index re-downloads on its own and a finished one is
never re-fetched pointlessly.
Three things back it up.
The staleness check is on the data rather than the cache, because a fresh
download of a stale file is still stale. If a series ends further behind the
present than its provider's own lag, that is reported with the command to force
a refresh, and the note that if refreshing changes nothing the provider has not
published either.
A failed download with a cached copy on disk returns the cached copy and warns,
rather than erroring. A provider being briefly down should not become an outage
here. Saying so is what keeps it from being mistaken for current.
climate_index_status() reports what is cached and what is due without
downloading anything, so it is safe offline. refresh_climate_index() forces a
re-fetch and skips the indices that cannot change.
Downloading also moves out of the NetCDF reader into cached_index_file(), which
every format now shares. The text indices were previously re-downloaded on every
call and are now cached too. That made the mocking in the AMOC tests
unnecessary: read_rapid_netcdf() takes a path, so those tests pass a fixture
directly instead of intercepting the cache.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>1 parent 556486c commit e01f157
11 files changed
Lines changed: 619 additions & 84 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
| 11 | + | |
10 | 12 | | |
11 | 13 | | |
12 | 14 | | |
| |||
24 | 26 | | |
25 | 27 | | |
26 | 28 | | |
| 29 | + | |
27 | 30 | | |
28 | 31 | | |
29 | 32 | | |
| |||
Large diffs are not rendered by default.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 8 additions & 7 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments