Skip to content

NGWPC NWM Regionalization, Hindcast, and Coastal Tools 2026-06-09 - #205

Merged
christophertubbs merged 1327 commits into
NOAA-OWP:mainfrom
NGWPC:ngwpc-3.1.2.3.0
Aug 10, 2026
Merged

NGWPC NWM Regionalization, Hindcast, and Coastal Tools 2026-06-09#205
christophertubbs merged 1327 commits into
NOAA-OWP:mainfrom
NGWPC:ngwpc-3.1.2.3.0

Conversation

@cmaynard-ngwpc

Copy link
Copy Markdown
Contributor

Expanded forecasting and hindcasting capabilities through the introduction of a comprehensive hindcast framework, including new models, endpoints, validation logic, data retrieval APIs, configuration options, logging, status tracking, and support for cold starts, calibration workflows, and verification processing. Improved forecast configuration management by adding module property persistence, import/export functionality, enhanced parameter validation, support for additional formulations and data sources, observational data integration, and geopackage handling. Strengthened authentication and security with MFA/2FA support, recovery codes, credential management improvements, and authentication workflow updates. Enhanced operational reliability through extensive logging improvements, performance optimizations, validation refinements, race-condition fixes, AWS profile support, archive handling fixes, and workflow cleanup. Modernized containerization and deployment by integrating EWTS throughout the platform, improving Docker build processes, dependency tracking, CI/CD pipelines, build reproducibility, and customer delivery workflows. Additional updates included forecast and hindcast data management improvements, Hawaii forecast enablement, retrospective workflow updates, verification enhancements, support for new hydrologic formulations, and numerous code quality, configuration, and maintenance improvements.

Additions

  • Integrated EWTS logging support across affected components, including module-specific logging updates, optional logging behavior, Docker/build integration, and unit-test coverage where applicable.
  • Expanded verification and evaluation capabilities with additional metric calculation behavior, observation retrieval support, plotting updates, gage-source integration, and forecast-to-observation alignment.
  • Added state saving, state restoration, reset-time, restart, and hindcast workflow capabilities, including related time-management updates and model restart handling.
  • Added and expanded forcing workflow support, including updated configuration templates, forecast product handling, lagged-ensemble inputs, regional forcing behavior, and package-delivered forcing resources.
  • Added NHF, hydrofabric, geopackage, VPU, and crosswalk workflow capabilities, including utility scripts, oCONUS handling, and domain/gage data integration.
  • Added hydrologic model coupling capabilities, including SAC-SMA and Soil Moisture Profiles integration where present in the commit set.
  • Expanded automated testing coverage, expected-result data, unit-test support, tolerance-based comparisons, serialization checks, and CI-exercised validation paths.
  • Added additional release updates covering Enable Hawaii forecasts, additions for display name, additions for 2fa to cli, additions for code for setting up mfa, and Support multiple AWS profiles.

Removals

  • Removed unused, duplicate, obsolete, and deprecated code paths, including stale helper functions, duplicated definitions, older EWTS references, and no-longer-needed implementation artifacts.
  • Removed additional release updates covering removal of translation, Delete old credentials when setting up, removal of active flag from master file, removal of AWS credentials check Update swagger, and removal of aet_rootzone field.

Changes

  • Updated CI/CD, security scanning, downstream pipeline triggers, and customer-delivery workflow configuration.
  • Updated Docker, build, packaging, dependency, and installed-resource configuration.
  • Refactored and cleaned up implementation details, including naming consistency, type hints, imports, property handling, and internal helper structure.
  • Refactored BMI model interfaces and input-forcing behavior, including model-class selection, getter/setter behavior, bounds handling, and BMI calculation fixes.
  • Improved validation, warning, exception, traceback, permission, and user-facing error-reporting behavior.
  • Corrected unit conversions and size/type reporting across model calculations, serialized buffers, time-step handling, and BMI-facing data exchanges.
  • Refined configuration, realization, template, sample-config, and package-resource handling across supported workflows and deployment environments.
  • Updated hydrologic model outputs and model-specific runtime behavior, including formulation outputs, GIUH/channel-inflow values, ponded-depth variables, and model-specific state/runtime handling.
  • Updated cache, dataset, temporary-file, scratch-directory, and data-access behavior, including lazy loading, explicit dataset closure, retry handling, and local/S3 access refinements.
  • Updated geospatial metadata, regridding, CRS, mesh/geogrid, domain, gage, and crosswalk handling across forcing, verification, and realization workflows.
  • Updated additional release updates covering Disable SR Alaska Extended, Adjust lead times, updates to Hawaii values, Avoid race condition when updating status, and Use different date range for NWM Retrospective.

peterkronenberg-rtx and others added 30 commits January 26, 2026 09:12
remove DEBUG from __local_settings_prod_pw.py because it was overridd…
Merge ngwpc-candidate into development for release 3.1.2.2.0-rc5
…. If we want to do that, we'll need to save it to the cache so all workers see the update
 Change handling of download zip files
Merge ngwpc-candidate into development for release 3.1.2.2.0-rc6
Rename formulation_name to job_name and save it in gage tab
Merge ngwpc-candidate into development for release 3.1.2.2.0-rc7
peterkronenberg-rtx and others added 29 commits May 5, 2026 17:17
Use BMI forcing for all jobs; include observational data on hindcast plots
Merge development into ngwpc-candidate for release 3.1.2.3.0-rc1
addind EWTS to docker compose files, added dependency repo info to Docker OCI label for better traceability
…ndidate

# Conflicts:
#	calibration/util/cloud_util.py
#	calibration/views/calibration_download_views.py
#	calibration/views/calibration_gage_views.py
#	calibration/views/calibration_import_export_views.py
#	calibration/views/calibration_landing_views.py
#	calibration/views/common.py
#	calibration/views/data_services.py
#	cerfServer/settings.py
…didate

 ngwpc release merge from candidate for release 3.1.2.3.0
@christophertubbs
christophertubbs merged commit f508de1 into NOAA-OWP:main Aug 10, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants