Fix year-boundary slot gap in normalise_dates#33
Open
jnioche wants to merge 1 commit into
Open
Conversation
Split 30 min before year end instead of at Jan 1 00:00. The old boundary produced a cross-year URL (end = Jan 1 00:01Z) that the API silently truncates at Dec 31 23:00. The new boundary keeps chunk URLs within the current year, then restarts cleanly at Jan 1. The Dec 31 23:30 slot (whose 'to' is Jan 1 00:00) remains inaccessible — this is an API limitation, not a library bug. Also switches reqwest to rustls-tls so unit tests compile without requiring libssl-dev. Closes #6 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #6 (the remaining missing 30-minute slot after the original workaround).
Jan 1 00:00, which made the chunk's API URL end atJan 1 00:01Z— still a cross-year request. The API silently truncates those atDec 31 23:00, so theDec 31 23:30slot and (depending on end date) earlier slots were missed.Dec 31 23:30) instead, keeping the URL within the current year. The next chunk then starts cleanly atJan 1 00:00.Dec 31 23:30slot (whosetofield isJan 1 00:00) is permanently inaccessible from the API — confirmed by direct API testing. No request (cross-year or within-year) returns it.reqwesttorustls-tls(default-features = false) so unit tests compile without requiringlibssl-dev.Test plan
normalise_dates_splitting— updated to expectDec 31 23:30as last boundarynormalise_dates_skipping_year— updated to expectDec 31 23:30/Jan 1splitnormalise_dates_year_boundary_slot— new test verifying boundary structure for any cross-year rangenormalise_dates_end_within_current_year— new regression test: ranges that stay within a year are unaffectedcargo test)🤖 Generated with Claude Code