Skip to content

Add optional forecast model processing - #680

Closed
FahrenheitResearch wants to merge 4 commits into
dpaulat:developfrom
FahrenheitResearch:feature/model-processing
Closed

Add optional forecast model processing#680
FahrenheitResearch wants to merge 4 commits into
dpaulat:developfrom
FahrenheitResearch:feature/model-processing

Conversation

@FahrenheitResearch

Copy link
Copy Markdown

Summary

This branch adds an optional forecast-model workflow to Supercell Wx through a dedicated View > Forecast Models dock. Users can discover runs, download and process selected forecast hours, render direct and derived products as georeferenced overlays, and generate point soundings from stored model fields.

The host integration remains C++/Qt. Model acquisition, decoding, derivation, and rendering run in an optional child executable through a versioned JSON-lines protocol; no Rust ABI, Rust library, or Rust toolchain is added to a normal Supercell Wx build. If the helper is absent, existing radar and alert workflows continue normally.

Architecture and review path

flowchart LR
    subgraph SCWX["Supercell Wx - C++ / Qt"]
        Dock["Forecast Models dock"]
        Manager["ModelManager / QProcess"]
        Records["Typed host records"]
        Map["MapWidget / ModelLayer"]
        Sounding["Sounding viewer"]

        Dock --> Manager
        Manager --> Records
        Records --> Map
        Records --> Sounding
    end

    subgraph Helper["Optional child executable"]
        Bridge["Versioned JSONL bridge"]
        Pipeline["Discovery, download, decode, derive"]
        Renderers["Overlay and sounding renderers"]

        Bridge --> Pipeline
        Bridge --> Renderers
    end

    subgraph Data["Provider-owned data"]
        Sources["Remote model sources"]
        Cache["Raw GRIB cache"]
        Store["Decoded run store"]
        Artifacts["Georeferenced PNGs"]
    end

    Manager -->|"CLI arguments"| Bridge
    Bridge -->|"JSONL events"| Manager
    Manager -.->|"cancel / terminate"| Bridge
    Bridge -.->|"stderr / exit"| Manager
    Sources --> Pipeline
    Pipeline <--> Cache
    Pipeline --> Store
    Store --> Renderers
    Renderers --> Artifacts
    Artifacts --> Map
    Artifacts --> Sounding
Loading

The architecture document identifies the main review entry points, the typed provider boundary, failure isolation, and the relationship to ProductDatastore PR #643. It also compares the current optional-helper approach with a built-in C++ provider and a future provider-plugin interface. The proposed low-risk convergence step is to extract a ModelProvider interface from ModelManager; the current JSONL adapter can then be one implementation without coupling the dock or map layer to its storage format.

Main changes

  • Dedicated dock rather than additional Radar Toolbox controls.
  • Host-owned model/run/product/frame/sounding records and signals.
  • Optional CMake packaging of a prebuilt processor and its license.
  • Transparent Web-Mercator overlays with search, favorites, visibility, opacity, playback, and optional timeline synchronization.
  • Responsive sounding preview and separate resizable sounding window.
  • Progress, cancellation, cache paths, and staged processor writes outside the UI process.
  • Bounded latest-run probing: Find latest confirms the requested forecast hour instead of scanning the model's full horizon. Exhaustive publication discovery remains an explicit processor operation.

The probe delay was caused by a second, unnecessary all-hour availability pass after the requested hour had already been verified. On HRRR extended cycles that meant 49 serialized NOMADS requests; models with longer horizons could issue hundreds. The bounded path uses the same model-specific cycle and cadence registry for every supported model.

Validation

  • Windows x64 Release target built successfully at d6390a5326.
  • Two bridge probe tests passed.
  • Two Supercell Wx fast/legacy probe-contract tests passed.
  • Live HRRR/NOMADS latest-run discovery improved from 125.8 seconds to 5.58 seconds; GFS/NOMADS completed in 0.57 seconds.
  • The preview package contains the architecture/user documents and the processor's license and third-party notices.

Known limitations

  • Overlays are viewport- or explicit-bounds raster snapshots rather than automatically tiled products.
  • Soundings require the selected hour to contain the sounding-profile fields.
  • One processor operation runs at a time; heavy ECAPE diagnostics remain an explicit opt-in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant