Add VPU metadata support

[pratikshac15]

This PR recreates the changes from #90 on a branch within the CIROH-UA repository so the authenticated GitHub Actions can run.

This includes the VPU metadata support changes and the latest review updates.

Replaces #90.

[Copilot]

Pull request overview

Adds VPU-aware handling for multi-GeoPackage / multi-weight forcing runs, including standardized VPU identifiers and new VPU-delineated metadata outputs, while preserving existing global metadata behavior.

Changes:

• Add VPU ID inference/normalization to align naming across weights generation and forcing processing.
• Write new per-VPU metadata outputs (metadata_by_vpu.csv plus catchment stats with a vpu_id column).
• Adjust multiprocessing behavior in weights generation and harden weight-ingestion logging.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
src/forcingprocessor/weights_hf2ds.py	Adds VPU ID inference and uses a spawn multiprocessing context for ProcessPoolExecutor; updates naming logic used to build VPU/catchment mappings.
src/forcingprocessor/processor.py	Adds VPU ID normalization + config handling and writes VPU-level metadata outputs; fixes local file size accumulation and adjusts grid extraction logic.

Comments suppressed due to low confidence (1)

src/forcingprocessor/processor.py:307

• The new condition treats any dataset with south_north/west_east dims like retrospective-2-1 for timestamp parsing, but many NWM files (e.g. nwm.t00z.short_range.forcing...) use those dims while the filename is not YYYYMMDDHH.... This will raise a ValueError in datetime.strptime. Timestamp extraction should prefer the model_output_valid_time attribute when available, and only fall back to parsing the filename when that attribute is missing.

                if "retrospective-2-1" in nwm_file or ("south_north" in nwm_data.dims and "west_east" in nwm_data.dims):
                    data_allvars[var_dx, :, :] = np.flip(np.squeeze(nwm_data[jvar].isel(west_east=slice(x_min, x_max+1), south_north=slice(shp[1] - (y_max+1), shp[1] - y_min)).values),axis=0)
                    t = datetime.strftime(datetime.strptime(nwm_file.split('/')[-1].split('.')[0],'%Y%m%d%H'),'%Y-%m-%d %H:%M:%S')
                else:
                    data_allvars[var_dx, :, :] = np.flip(np.squeeze(nwm_data[jvar].isel(x=slice(x_min, x_max+1), y=slice(shp[1] - (y_max+1), shp[1] - y_min)).values),axis=0)
                    time_splt = nwm_data.attrs["model_output_valid_time"].split("_")
                    t = time_splt[0] + " " + time_splt[1]

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.