Skip to content

Add native modification-aware RAS Mapper terrain export - #321

Merged
gpt-cmdr merged 17 commits into
mainfrom
codex/native-rasmapper-terrain-export
Aug 30, 2026
Merged

Add native modification-aware RAS Mapper terrain export#321
gpt-cmdr merged 17 commits into
mainfrom
codex/native-rasmapper-terrain-export

Conversation

@gpt-cmdr

@gpt-cmdr gpt-cmdr commented Aug 29, 2026

Copy link
Copy Markdown
Owner

Summary

Adds RasTerrain.export_rasmapper_terrain(), a supervised wrapper around RAS Mapper's registered-terrain Export to Single Raster operation.

The API selects a terrain from the project .rasmap, preserves RAS Mapper source ordering, priorities, stitches, masks, and vector modifications, and writes one bounded Float32 GeoTIFF. It supports exact source-derived 1x, 2x, 4x, and 8x nearest-neighbor output grids while allowing RAS Mapper to consolidate mixed-resolution registered sources.

The operation runs out of process on native Windows and through configured Wine on Linux. It uses task-local state, supervises only owned processes, validates the completed raster before promotion, and returns a bool-compatible TerrainExportResult with a machine-readable JSON receipt. The derivative is not registered into the source project.

RasTerrainMod.compute_modified_terrain_raster() is deprecated as of 0.99.2, emits DeprecationWarning, and is scheduled for removal in 1.1. It remains available only for the compatibility window; native failures never fall back to the numerically different row sampler.

Type of Change

  • Bug fix
  • New feature
  • Documentation update
  • Example notebooks
  • Native helper and packaging

Public API and Runtime Behavior

  • DataFrame-first terrain selection through RasMap.list_terrain_layers().
  • str and Path inputs, ras_object multi-project support, and explicit version consistency checks.
  • Bounded, source-grid-aligned output at exact 1x/2x/4x/8x cell size.
  • Nearest-neighbor resampling with resampleTo1RFI=True and optional native resampleVecMods rasterization.
  • Unique same-directory partial output, semantic GeoTIFF validation, and atomic TIFF/receipt promotion.
  • Transaction-safe overwrite rollback; stale-backup cleanup failures cannot delete the newly committed TIFF/receipt pair.
  • Complete cleanup of owned partials and .aux.xml, .ovr, .tfw, and .prj sidecars.
  • Task-local Wine prefix/staging and owned-process-group timeout cleanup; no global wineserver termination.
  • Strict managed reflection of the private nine-parameter GenerateNewRasTerrain contract, including the final by-reference TiffMetadata<float> parameter.
  • Reproducible tracked x86 helper build recipe. The packaged helper remains I386 PE32, .NET Framework 4, ILONLY | 32BITREQUIRED.

HEC-RAS Version Contract

Version Status
6.3 / 6.3.1 Unsupported: no bounded GenerateNewRasTerrain(..., resampleVecMods, ...) contract
6.4.0 Unsupported: unqualified and affected by HEC's documented terrain issue
6.4.1 Supported and qualified on Windows and Wine
6.5 Supported and qualified on Windows and Wine
6.6 Supported and qualified on Windows and Wine
6.7 beta releases Unsupported prereleases
7.0.0 Unsupported because of HEC's documented terrain-modification export defect
7.0.1 Supported and qualified on Windows and Wine
7.1 Forward-open but unqualified until an official Classic release is available

Version checks run before output directories or native work. Explicit terms, RasPrj.ras_version, project executable paths, and resolved native/Wine installation directories reject prerelease labels before final-release canonicalization, including space-, hyphen-, and underscore-delimited beta forms.

Native and Wine Qualification

Native Windows qualification covers every supported release with bounded modification-aware UPGU3 exports and stitched Muncie exports. The HEC-RAS 6.6 mixed-resolution Bald Eagle case produced one 61 x 61 GeoTIFF at exactly 40-foot cells from 36.504512049933-foot and 20-foot registered sources. Enabling native modifications raised 264 cells by 0.15625 to 9.625 feet while 1,769 control cells remained exactly unchanged.

Wine qualification covers every supported release:

  • The exact-input HEC-RAS 6.6 Windows/Wine comparison produced identical Float32 arrays, validity/affected/control masks, grid metadata, source inventory, and receipt semantics; maximum valid-cell difference was 0.0 feet.
  • Independent CLB07 runs for 6.4.1, 6.5, and 7.0.1 each passed stitched Muncie plus mixed-resolution modification-off/on exports.
  • All nine multi-version receipts passed. Arrays and masks were pixel-identical across 6.4.1, 6.5, and 7.0.1, with 264 raised cells and 1,769 unchanged controls in each release.
  • The exact managed invocation contract passed for all supported Wine runtimes.
  • Cleanup checks found no owned helper, Wine, gdalinfo, stage-directory, or partial-output survivor.

Example Notebooks and Visual Evidence

Generated GeoTIFFs, receipts, and large qualification artifacts remain task-local and are not committed.

Independent Claude Code / Fable QAQC

Claude Code 2.1.251 reviewed the full PR using canonical claude-fable-5 at xhigh effort in strict read-only mode. Findings were independently reproduced before changes were made.

Confirmed findings fixed in this branch include:

  • overwrite cleanup ordering that could remove the prior TIFF after a post-promotion cleanup error;
  • incomplete owned .tfw/.prj sidecar cleanup;
  • missing reproducible x86 helper build instructions;
  • shallow helper-response validation and poor malformed-receipt diagnostics;
  • private qualification fixture defaults;
  • prerelease labels that could canonicalize to a supported final-release key; and
  • stale public/agent documentation about native export and Wine qualification.

Fable then reviewed the remediation and the two follow-up prerelease-guard deltas. The final targeted verification result was CLEAN, with no residual runtime beta-label finding.

Final Validation

  • Combined terrain host, public/version, packaging-resource, and five-notebook contract suites: 193 passed in 4.51 seconds.
  • Deprecation and native-export focused regression suites after the final compatibility change: 143 passed in 2.60 seconds.
  • Opt-in native qualification module: all 12 tests collected without execution.
  • With qualification enabled but fixture variables unset: 12 skipped cleanly, proving portable environment-required behavior.
  • Public imports, py_compile, and Ruff: passed.
  • Helper build recipe against HEC-RAS 6.6: passed; produced the required I386 PE32/.NET 4/32-bit-required helper.
  • Source distribution: contains helper source, packaged executable, and build recipe.
  • Notebook audit for 316/612/920/930/931: zero stored exceptions; only retained pre-existing warnings in 316 and 612.
  • Notebook metadata/index: 135 of 135 entries, zero errors.
  • Production-equivalent MkDocs build: passed in 49.61 seconds; retained repository-wide revision-date/missing-link warnings only.
  • Full branch diff, cached diff, and worktree checks: clean.
  • GitHub validate: passed.

Previously recorded broader evidence remains in the committed qualification report, including the opt-in native Windows matrix and the full repository run of 2,393 passed, 62 skipped, with eight documented baseline/environment-order failures outside this feature.

Qualification Boundaries

  • Per operator direction, the dedicated triangular-nose/minimum-Y regression fixture was skipped; no result is claimed for that fixture.
  • HEC-RAS 7.0.0 remains rejected based on HEC's published defect notice.
  • Stable HEC-RAS 7.1 is forward-open but remains unqualified until an official binary is published; prerelease 7.1 labels are rejected.
  • Direct UNC output shares are not qualified because helper GDAL junction staging requires a local Windows volume. UNC project/input paths with a local output retain supported path handling.
  • Fully qualified helper response/output paths beyond the legacy .NET 260-character limit are unqualified; use a short task-local output root.
  • Terrain registration, RAS Mapper UI automation, hydraulic simulation, and reimplementation of terrain-modification mathematics remain out of scope.

Review Checklist

  • Static public API and logging conventions followed
  • pathlib.Path used while accepting str and Path
  • DataFrame-first terrain selection and ras_object multi-project conventions preserved
  • Deprecated row-sampler retained for the documented compatibility window
  • Source project and terrain registration remain immutable
  • No model, input, output, installer, dependency, or executable hashes added
  • Notebook outputs and visual evidence retained where execution was safe
  • Independent Fable QAQC findings fixed and re-reviewed

LLM Attribution

Tools used: Codex and Claude Code/Fable

@gpt-cmdr

Copy link
Copy Markdown
Owner Author

Final Linux/Wine qualification update (6f6eac5):

  • Independent CLB07 CT212 run passed on Ubuntu 24.04, Wine 11.0, and HEC-RAS 6.6.
  • Bounded two-source Muncie 2x export passed (16 x 23, 10-ft cells), and the original mixed-resolution Terrain50 consolidated its 36.504512-ft and 20-ft sources to one 61 x 61 GeoTIFF at 40-ft cells.
  • Exact-input parity used the same notebook-316 modified project/HDF on native Windows and CLB07 Wine. Off/on Float32 arrays, validity and affected/control masks, grid/raster metadata, normalized source inventories, validation fields, and receipt semantics matched exactly; maximum valid-cell difference was 0.0 ft. The shared semantic mask contained 264 raised cells and 1,769 unchanged controls.
  • Fixed the Linux diagnostic label for conflicting Windows-drive and UNC ras_exe_path values. Runtime rejection behavior was already correct.
  • Corrected static mkdocstrings targets for the RasProject APIs; the CI-equivalent non-strict docs build now passes locally.
  • Added permanent environment-gated Linux qualification coverage for Muncie, mixed-source consolidation, and optional exact Windows-reference pixel parity.
  • CT212 cleanup passed: it is stopped with onboot=0; no owned Wine/helper/gdalinfo processes, stage directories, or partial outputs remain.

Remaining qualification gaps: Wine runs for HEC-RAS 6.4.1, 6.5, and 7.0.1; the exact triangular-nose/minimum-Y fixture; and HEC-RAS Classic 7.1 once published. The PR remains open and unmerged.

@gpt-cmdr

Copy link
Copy Markdown
Owner Author

Fable QAQC follow-up

Claude Code 2.1.251 / Fable independently reviewed the native terrain-export implementation. The accepted follow-up commits (46dd5ffa, 66bfb169, 16469a39) confirm and harden these areas:

  • helper inspect/export responses now receive deep extent, source-inventory, dimension, cell-size, and finite-number validation before use;
  • semantic rejection removes every owned partial-sidecar spelling;
  • overwrite promotion treats the TIFF and receipt transactionally, restores the prior pair if receipt promotion fails, and retains a committed success while warning if only stale-backup deletion fails;
  • real-runtime qualification fixtures are environment-only and skip explicitly when unset—no private machine-default project paths remain;
  • stable supported keys cannot be reused by Beta/prerelease project versions, executable folders, discovered Windows installs, or Wine runtime directories; Beta matching is token-bounded to avoid false positives inside unrelated words;
  • the packaged PowerShell build recipe reproducibly targets I386 PE32 / ILONLY | 32BITREQUIRED against the HEC-RAS 6.6 managed references, and the source distribution includes the recipe, helper source, and helper executable.

Final CLEAN verification after those fixes:

  • host/public/version/package-resource/notebook-contract suite: 193 passed in 4.51 s;
  • opt-in qualification module: 12 tests collected in 0.78 s without executing runtimes;
  • opt-in with every fixture unset: 12 skipped in 0.79 s as designed;
  • imports, py_compile, and Ruff: passed;
  • five-notebook audit: zero stored exceptions; retained pre-existing warnings only in notebooks 316 and 612;
  • notebook metadata/index: 135 notebooks, zero errors; checks passed;
  • helper rebuild and sdist-content checks: passed;
  • production-equivalent non-strict MkDocs build: passed in 49.61 s with existing revision-date and missing-notebook-link warnings;
  • worktree, cached diff, and full branch diff checks: clean.

Qualification boundaries remain explicit: Windows-drive and UNC project/input paths are covered, but direct output to a UNC share and helper response paths beyond the legacy .NET 260-character limit remain unqualified. Per operator direction, the triangular/minimum-Y defect fixture was skipped; HEC-RAS 7.0.0 remains rejected. HEC-RAS 7.1 remains forward-open behind the managed-contract check but unqualified until an official Classic release is available.

@gpt-cmdr
gpt-cmdr merged commit 46864af into main Aug 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant