Add new ghcn data access - #74
Open
crvernon wants to merge 8 commits into
Open
Conversation
Collaborator
|
While reviewing this PR I found and fixed a few issues, and ran additional testing:
Testing
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds support for the next-generation NCEI GHCN-hourly (GHCNh) weather data source to the
diyepwpackage, ensuring continued generation of AMY EPW files beyond NOAA's discontinuation of the ISD-Lite product in August 2025. The update includes new data normalization utilities, expanded APIs, and comprehensive documentation on the new data source and station identifier mapping.Major new features and improvements:
Support for GHCNh data and ISD-Lite discontinuation:
wmo_to_ghcn_id(), to resolve differences between ISD-Lite and GHCNh station IDs. Documentation describes how the mapping is built and used.API and CLI enhancements:
create_amy_epw_file()andcreate_amy_epw_files_for_years_and_wmos()now accept asourceargument ("auto","isd_lite", or"ghcnh") and an optionalstation_idfor direct GHCN access. CLI documentation explains usage and fallback behavior. [1] [2]analyze_amy_file,get_ghcnh_file, andwmo_to_ghcn_idindiyepw/__init__.pyfor unified access to both ISD-Lite and GHCNh workflows.Data normalization and analysis:
_ghcnh_normalize.py, a new module for reading and converting GHCNh PSV files into the canonical AMY DataFrame schema, including logic for collapsing sub-hourly observations to hourly and normalizing ISD-Lite data for consistency.analyze_amy_file.py, a new analysis module that works with both ISD-Lite and GHCNh formats, superseding the original ISD-Lite-only analysis.Infrastructure improvements:
_http.pywith HTTP helpers enforcing bounded timeouts, improving reliability when downloading from NOAA/NCEI endpoints.These changes ensure that
diyepwremains functional and robust as NOAA transitions away from ISD-Lite, while providing users with clear documentation and flexible APIs for working with both legacy and next-generation weather datasets.