Add USGS stream gauges: discharge and stage hydrographs - #3
Merged
Conversation
The channel-side counterpart to gauges.py. Where that pulls what fell out of the sky at a point, this pulls what came down the channel -- discharge and gage height for the USGS gauges in the AOI, over the same window, so a hydrograph can be read against the rain that produced it. For post-fire work the pairing is the point: a burned catchment converts rain to runoff at a rate an unburned one does not, and the interesting quantity is the relationship between the two, including how long after the cell the response arrived. Two sources, one interface, because USGS is mid-migration. `nwis` (default) is the legacy Water Services endpoints -- no key, and the format everyone parses, but announced for decommission on an explicitly uncertain timeline. `ogc` is the modernized replacement, which works anonymously at 100 requests/hour (ample: one site query plus a few series per AOI) and reads a free key from $STORMSCAPE_USGS_API_KEY. Legacy stays the default only because the replacement is still served under a /v0/ path; `ogc` is implemented and tested, not a stub. The two were checked against each other rather than assumed equivalent: over the same AOI they return the same 20 gauges and byte-identical values, max |nwis-ogc| = 0.000000 cfs across 289 timestamps. Getting there took work -- monitoring-locations answers "is this a stream site", not "does it measure discharge every 15 minutes", so filtering on it returned 120 sites against NWIS's 20. Querying time-series-metadata for parameter 00060 with computation_period_identifier="Points" fixes that, and its begin/end dates reproduce NWIS's active-site filter (the five remaining extras were records that ended between 1994 and 2017). - streamflow.py: sites, series, per-gauge summary, and a canonical event store mirroring gauges.fetch_gauge_event. - CLI `flow` writes the store, a hydrograph atlas and a map; `--detail` adds a figure per gauge, `--rain` puts the event's rain gauge above it as an inverted hyetograph. `--stream-gauges` overlays the gauges on any map command. - plot.hydrograph / hydrograph_atlas / add_stream_gauges. The atlas shares a time axis so gauges can be compared for timing but keeps independent y-scales, because discharge across an AOI spans five orders of magnitude. Timestamps are normalised to UTC on arrival: NWIS returns the gauge's local time with an offset, so read naively a Pacific hydrograph lands seven hours from the radar that caused it. Missing values arrive as -999999, not null, and the site service returns one row per time series so a gauge appears several times. Units are carried both ways throughout -- discharge_cms/discharge_cfs and stage_m/stage_ft -- because USGS publishes ft3/s while the rest of stormscape is metric. Summaries and figures default to SI; --units cfs reads them the USGS way. `peak_at_edge` flags a gauge whose peak lands on its last observation: the window cut the hydrograph while it was still rising, so the peak reads low. Found by looking at a figure, not by a test, and confirmed on Steamboat Ck at Cleanwater. Same rule as max_wet_hours -- warn, never let a truncated storm pass as complete. Colouring gauges by a value now draws a colour bar; the first render encoded peak discharge with no key at all. Tests: tests/test_streamflow.py, 56 offline cases, including the AST guard that checks every args.<name> the command reads exists in the parser.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The channel-side counterpart to
gauges.py. Where that pulls what fell out ofthe sky at a point, this pulls what came down the channel — discharge and gage
height for the USGS gauges in the AOI, over the same window, so a hydrograph can
be read against the rain that produced it.
For post-fire work the pairing is the point: a burned catchment converts rain to
runoff at a rate an unburned one does not, and the interesting quantity is the
relationship between the two — including how long after the cell the response
arrived.
Two USGS sources, one interface
USGS is mid-migration off the legacy NWIS Water Services, so
--sourcepicks thebackend:
--sourcenwis(default)waterservices.usgs.govogcapi.waterdata.usgs.gov/ogcapi/v0Legacy stays the default only because the replacement is still on a
/v0/path.ogcis implemented and tested, not a stub.The two were checked against each other rather than assumed equivalent: over
the same AOI they return the same 20 gauges and byte-identical values
(max |nwis − ogc| = 0.000000 cfs across 289 timestamps).
Getting there took work.
monitoring-locationsanswers "is this a stream site",not "does it measure discharge every 15 minutes", so filtering on it returned
120 sites against NWIS's 20. Querying
time-series-metadatafor parameter00060withcomputation_period_identifier="Points"fixes that, and itsbegin/enddates reproduce NWIS's active-site filter — the five remainingextras turned out to be records that ended between 1994 and 2017.
What's here
streamflow.py— sites, series, per-gauge summary, and a canonical event storemirroring
gauges.fetch_gauge_event.flowwrites the store, a hydrograph atlas and a map;--detailadds afigure per gauge and
--rainputs the event's rain gauge above it as aninverted hyetograph.
--stream-gaugesoverlays the gauges on any map command.plot.hydrograph/hydrograph_atlas/add_stream_gauges. The atlas shares atime axis so gauges can be compared for timing, but keeps independent
y-scales — discharge across an AOI spans five orders of magnitude.
Things that would have corrupted a hydrograph
NWIS returns the gauge's local time with an offset, not UTC. Read naively, a
Pacific hydrograph lands seven hours from the radar that caused it. Everything is
normalised on arrival. Missing values arrive as
-999999rather than null, andthe site service returns one row per time series so a gauge appears several times.
outputDataTypeCdwithsiteOutput=expandedis an HTTP 400 — the pair looksreasonable and fails. The NWIS site service is also genuinely slow over a bbox
(30–45 s against sub-second for OGC), so it has its own longer timeout.
Units are carried both ways throughout —
discharge_cms/discharge_cfsandstage_m/stage_ft— because USGS publishes ft³/s while the rest of stormscapeis metric. Summaries and figures default to SI;
--units cfsreads them the USGSway. Nothing is lost either direction.
Two things caught by looking at figures, not by tests
peak_at_edge. A gauge whose peak lands on its last observation was stillrising when the window ended, so the peak — and everything derived from it —
reads low. Spotted on Steamboat Ck at Cleanwater, now flagged and named in the
run output. Same rule as
max_wet_hours: warn, never let a truncated storm passas a complete one.
A colour bar for the gauges. The first map coloured them by peak discharge
with no key at all, which is the same flaw as a graduated symbol without a size
legend.
Caveat for the science
On a regulated river a sharp rise may be a reservoir release, not a storm
response.
rise_ratioand drainage area help, but attribution on a dammedchannel is the analyst's call, not the tool's.
🤖 Generated with Claude Code