Removing qt - #1
Open
cbuahin wants to merge 52 commits into
Open
Conversation
Implement directed graph Network for river/flow networks with vertices and edges. Implement GDAL-backed Raster with band access and coordinate/value read/write operations. Both use opaque pointers in headers to avoid coupling to heavy GDAL/interface dependencies. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement Cartesian, rectilinear, and curvilinear grid types for 2D and 3D structured grids. Support node indexing, cell geometry, envelope calculation, and SIMD vectorization. Row-major storage layout for cache efficiency. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Update constructor to match AbstractComponentDataItem requirements (id, dimensions, valueDefinition, modelComponent, raster, bandIndex) - Fix getValue/getValues/setValue/setValues parameter order and types to match AbstractComponentDataItem pure virtual methods - Change dimensionCount() to dimensionLength() for consistency - Use proper hydrocouple_variant indices for double (11) and int (4) - Disable test compilation to allow library build verification Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Convert TINComponentDataItem to TINComponentDataItemStorage - Convert NetworkComponentDataItem to NetworkComponentDataItemStorage - Convert RegularGrid2DComponentDataItem to RegularGrid2DComponentDataItemStorage - Convert RegularGrid3DComponentDataItem to RegularGrid3DComponentDataItemStorage These template classes now provide pure storage with no AbstractComponentDataItem inheritance. They are designed to be used as mixins combined with concrete implementations that inherit from AbstractComponentDataItem and provide proper interface implementation. Storage mixins provide: - Typed data() vector access - getValue/setValue methods for individual elements - bulkGet/bulkSet for SIMD-vectorizable bulk operations - initStorage methods for deferred initialization - No override methods or interface implementation Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Established storage mixin pattern for spatial data types: - Pure storage templates (no interface inheritance) - Designed for multiple inheritance with AbstractComponentDataItem - SIMD-vectorizable bulk operations (bulkGet/bulkSet) - Row-major or index-based memory layout Storage mixins implemented: - TINComponentDataItemStorage<T> for mesh data - NetworkComponentDataItemStorage<T> for network data - RegularGrid2DComponentDataItemStorage<T> for 2D grids - RegularGrid3DComponentDataItemStorage<T> for 3D grids Architecture verified with RasterComponentDataItem concrete implementation. Next: Create concrete data item classes combining storage mixins with AbstractComponentDataItem. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Three coordinated cleanups:
1. Delete legacy Qt-based code (none was being compiled).
- include/{core,spatial,temporal,spatiotemporal,composition,tests}
- src/{core,temporal,spatiotemporal,composition} and src/spatial
(except triangle library, which moves to vendor)
- Top-level Qt utilities: hydrocoupleexceptions, progresschecker,
splineinterpolator, matrix, specialmap, and the Qt scratch main.cpp
2. Categorize remaining utilities under organized folders.
- src/spatial/{triangle,tricall}.c -> src/vendor/triangle/
- include/spatial/triangle.h -> include/vendor/triangle/
- include/spline.h -> include/vendor/spline.h
- threadsafenetcdf/ wrappers stay in place (already non-Qt)
3. Reorganize the new C++20 code by namespace.
- include/hydrocouple/ keeps the core HydroCouple namespace headers
- HydroCouple::Spatial -> include/hydrocouple/spatial/, src/hydrocouple/spatial/
- HydroCouple::Temporal -> include/hydrocouple/temporal/, src/hydrocouple/temporal/
- All cross-subdirectory #include paths qualified with hydrocouple/<subdir>/
src/CMakeLists.txt updated; clean Release build of libHydroCoupleSDK.dylib
verified on macOS (Unix Makefiles + vcpkg toolchain).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… namespace, domain layout
- Port SDK to HydroCouple v2 interface: BufferDescriptor hyperslab data plane
(bufferio engine + storage mixins), variant convention removed entirely
- Template data items: Argument1D/2D<T>, IdBasedArgument<T>, Input/Output1D/2D<T>,
TimeSeriesInput/Output<T> with exported int/double/string instantiations
- Reorganize into include/hydrocouplesdk/{core,component,data,temporal,spatial}
under namespace HydroCouple::SDK; export.h; PCH moved to src/
- vcpkg manifest + CMake conventions per openswmm.engine; feature-gated deps;
configure-time version alignment with the interface definitions
- GTest suite (55 tests) incl. fixes: CSV comma parsing, Julian-Gregorian
inversion, libc signal.h shadowing; CI build/test + Doxygen docs workflows
- MIT SPDX headers; author normalized to Caleb Buahin; Qt fully removed
…ializer - IOutputWriter lifecycle interface + move-only Snapshot (owned field slices deep-copied through the typed hyperslab data plane) - IOThread: single writer thread, bounded queue with back-pressure, first-error latching per writer, drain-then-join shutdown - CSVWriter reference implementation (long-format time,item,index,value) - ModelInitializer: JSON/YAML composition documents targeted at IArgument (validate before apply, actionable errors, output-to-input wiring incl. multi-inputs, symmetric serialize() round trip; YAML converts to JSON at the boundary via optional yaml-cpp — added to the vcpkg manifest) - AbstractModelComponent: arguments materialize lazily on first access, honoring the v2 contract that arguments() is available in Created state - 17 new GTest cases (snapshot capture/ownership, CSV content, IOThread ordering/back-pressure/error latching, initializer validation + round trip, end-to-end compose-capture-write); suite now 72 green
- HDF5UGRIDWriter (SDK::IO, pimpl over the HDF5 C API — no HDF5 types in the public header): CF-1.11/UGRID-1.0 conventions, mesh-topology dummy dataset with cf_role, node coordinates, fill-value-padded face_node_connectivity, edge connectivity, unlimited /time with julian calendar, chunk+deflate on all time-varying datasets, extend-and-write hyperslab appends, fields tagged mesh + location (node|edge|face) - MeshDefinition: UGRID-congruent SoA/CSR mesh description with structural validation (IMeshView-aligned vocabulary) - Numeric DataKinds converted through a reused staging buffer (no steady state allocation); registered-field presence and element counts enforced with actionable messages - CMake: USE_HDF5 with config-then-module find chain (hdf5-shared/-static/ HDF5::HDF5) + explicit zlib chaining for static archives; CI matrix now builds with the hdf5 vcpkg feature - 11 new GTest cases reading the file back via the HDF5 C API (structure, attributes, padded connectivity, time growth, error paths, behind the IOThread); suite 83 green with HDF5, 72 without; file independently validated with h5py (conventions, tags, gzip chunking)
- GeoPackageWriter (SDK::IO, pimpl over the SQLite C API): gpkg_* conformance tables with mandatory SRS rows (-1/0/4326), 'GPKG' application_id + 1.3 user_version, optional mesh_nodes feature layer with GeoPackageBinary-encoded WKB points registered in gpkg_contents/ gpkg_geometry_columns - Engine result schema: simulations / variables (auto-registered, addField enriches units+description) / result_timeseries(simulation, variable, object_index, elapsed_time, value) with covering index / result_summary aggregates computed at finalize() - Engine efficiency kit: RAII DbPtr/StmtPtr/Transaction with rollback, prepared statements, buffered transactional inserts (configurable flush, default 5000 rows), PRAGMA synchronous=NORMAL - CMake USE_GEOPACKAGE with unofficial-sqlite3 config → FindSQLite3 module fallback; vcpkg geopackage feature (sqlite3 + rtree); hdf5 feature now builds threadsafe; CI matrix builds tests;hdf5;geopackage - 7 new GTest cases reading the package back via the SQLite C API (conformance, GP binary header + WKB payload, series values, aggregates, transaction rotation with tiny flush, behind the IOThread); suite 90 green; file independently validated with Python sqlite3 (integrity_check ok, contents/summary/nodes)
- NetCDFUGRIDWriter (SDK::IO, pimpl over the NetCDF-4 C API): same CF-1.11/UGRID-1.0 layout as the HDF5 writer — topology dummy variable, node coordinates, fill-value-padded face_node_connectivity, unlimited time with julian calendar, shuffle+deflate on field variables tagged mesh + location - MeshDefinition/MeshLocation extracted to io/meshdefinition.h (shared by the HDF5, NetCDF, and GeoPackage writers; validate() unconditionally compiled) - threadsafenetcdf retired: its ~100 '#pragma omp critical' wrappers existed to serialize concurrent NetCDF calls — a job the single-writer IOThread performs by design (plan §4.5 rationale recorded) - CMake USE_NETCDF: netCDF config target → manual header/library probe fallback; vcpkg netcdf feature now netcdf-c (hdf5-backed); CI builds tests;hdf5;geopackage;netcdf - 6 new GTest cases reading the file back via the NetCDF C API (conventions, topology, unlimited time growth, packed connectivity, error paths, behind the IOThread); suite 96 green (72 lean); file independently validated with h5py (netCDF-4 = HDF5 underneath)
- io/sidecar.h: @ref reference objects (path/kind/shape/byteOrder) with header-only write/read utilities; bulk bytes move through the typed hyperslab data plane in single stream operations — large payloads never transit a JSON/YAML DOM - ModelInitializer sidecar mode: setSidecarDirectory/setSidecarThresholdBytes (default 1 KB); serialize() emits references + sidecar files for large fixed-width arguments; initialize() resolves relative references against the sidecar/document directory and feeds them through a read-only SidecarSourceItem adapter, so every argument resizes and bulk-copies via its own initialize(IComponentDataItem&) path - Benchmark test: 10^7-element (153 MB) argument round trip in ~3.3 s at ~47 MB/s with the composition document under 4 KB; corrupted-sidecar and small-payload-stays-inline cases covered - Suite 100 green full-featured (76 lean)
- Envelope: fixed 32-byte trivially-copyable wire header (magic 'HCPL', version, op codes incl. Heartbeat/Error/Shutdown for cloud hardening, source/target ranks, tag, epoch, payloadBytes) enabling the two-phase exact-size receive pattern; layout enforced by static_asserts - ValuePacker: typed contiguous wire blocks over BufferDescriptor — kind/rank/shape header + raw elements; contiguous sources memcpy, strided sources gathered once through the hyperslab engine; unpack validates kind and block size with actionable messages (payloads are never per-element variants, per plan §5.2) - InProcTransport/InProcHub: backend #0 of the ITransport seam — rank-addressed mailboxes with MPI-style tagged matching (any-source -1, out-of-order tag retrieval), blocking send/receive and async IExchangeRequest handles; internally synchronized for cross-thread use; makes the entire protocol unit-testable without mpirun - 16 new GTest cases: envelope layout/round-trip/corruption, packer round trips + strided gather + mismatch/truncation rejection, transport cross-thread exchange, tag/any-source semantics, async test/wait, type mismatch surfacing, envelope+payload two-phase pattern - Back-pressure test lower bound gains a scheduling margin (semantic proof stays completeness + ordering); suite 116 green x3 (92 lean)
- ComponentWorker: remote-side command loop over any ITransport — receives the fixed 32-byte envelope (any source, control tag), then exact-size payloads on the request tag; dispatches lifecycle ops (Initialize/ Validate/Prepare/Update/Finish) with Ack or Error-with-text replies; GetValues stages an item's full extent through getValuesInto and ships it as one typed block; SetValues receives a typed block straight into the named item via setValuesFrom (kind/count validated by the transport); Heartbeat/Hello liveness; orderly Shutdown - ProxyModelComponent: implements IProxyModelComponent/ IDistributedModelComponent — compositions treat the proxy as a local component while every call becomes a protocol round trip; cloud hardening per the interface review: connect/disconnect/isConnected, ping(), per-request timeouts (tag-scoped async waits with deadline); peer death or remote failure latches ComponentStatus::Failed plus a Fatal ErrorEntry with the diagnostic - 5 new GTest cases over InProcHub: handshake+ping, remote lifecycle with status mirroring, inflow-push → remote doubling update → flow-pull round trip, unknown-item remote error, dead-peer timeout → Failed + Fatal diagnostic; suite 121 green (97 lean)
…odes) - HaloExchanger: builds the halo send/receive pattern collectively from ownership metadata (each rank tells owners which globals it mirrors; owners record matching local gather positions; owner-side validation rejects requests for globals the rank does not own; self-owned mirrors rejected up front) — validated once, before any exchange, per the SWMM decomposition lessons. Epoch execution: gather owned values into reused per-peer staging at begin() (halo carries values as of begin even when owned slots are overwritten during the overlap window), nonblocking receives, scatter into virtual slots + epoch advance at end()/tryEnd(); kind-agnostic raw-byte movement with no steady-state allocation - PartitionedDataItem<T>: implements IPartitionedComponentDataItem — owned/virtual global-index spans, virtual owner ranks, synchronizationEpoch(), synchronizeAsync() returning an IExchangeRequest that wraps begin → overlap → end; owned-then-virtual contiguous typed storage exposed through the v2 hyperslab data plane; ownedValues()/virtualValues() spans encode the no-DOF mirror policy; double/int32 instantiations exported - 8 new GTest cases: metadata exposure, data-plane layout, two-rank exchange, gather-at-begin overlap proof, three repeated epochs with fresh values, three-rank ring, misattributed-ownership and self-owned pattern failures; suite 129 green (105 lean)
… strategies (plan §5.5) Implements the advanced IWorkflowComponent interface (HydroCouple dev): - AbstractWorkflowComponent: status machine + signal (WorkflowStatusChangeEventArgs), role-labeled component registry with required-role validation, workflow back-pointer bookkeeping, error queue with automatic Fatal on Failed, atomic cooperative stop/pause honored at synchronization points, phase-order enforcement - PullDrivenWorkflow: the OpenMI request-reply default as a declared strategy — explicit trigger input; one update() services one post-order DFS pull chain, each upstream component exactly once (visited set = time-lagged feedback, matching OpenMI) - TimeSteppedWorkflow: execution schedule derived from the exchange-connection graph at prepare() — Tarjan SCCs collapse feedback loops into diagnosed iteration groups (iterationsPerGroup: 1 = time-lagged, >1 = fixed-point sweeps) in topological order - 14 tests: role diagnostics, upstream-first pull ordering with value propagation, run-to-Done, failure propagation, schedule ordering, cycle collapse + sweep counts, signal trace, pause/resume, stop (suite 119 green)
…n — Phase 5 complete
MpiTransport (plan §5.1 backend #2, USE_MPI-gated):
- same ValuePacker wire blocks as InProcTransport, so every layer built
on the ITransport seam (Envelope dispatch, ComponentWorker, proxies,
HaloExchanger, PartitionedDataItem) runs unchanged over MPI
- all MPI types behind a pimpl — the header installs without MPI;
communicator duplicated at construction to isolate tag space
- any-source → MPI_ANY_SOURCE; matched probes (MPI_Mprobe /
MPI_Improbe + MPI_Imrecv) discover message sizes, so async receives
need no out-of-band size negotiation; sendAsync owns its packed
block, making payloads immediately reusable
- MpiEnvironment RAII guard (THREAD_MULTIPLE) for executables the SDK
owns; transport never initializes/finalizes MPI itself
FluxDerivativePayload (§5.4, the SWMM coupling lesson): flux exchanged
with its derivative w.r.t. the coupling state as a documented {2, N}
Float64 block — no new wire format, kind/shape validated on unpack by
every backend — plus linearized boundary evaluation q0 + dq/dh·(h−h0)
and peer-descriptor convention checks.
Tests: dedicated HydroCoupleSDKMpiTests executable (own main; ctest
runs it under mpiexec -n 3; every test skips below its rank
requirement) — typed round trip, any-source + out-of-order tags,
matched-probe async, strided gather, kind-mismatch rejection, and the
3-rank HaloExchanger/PartitionedDataItem ring running unchanged over
real MPI with epoch-exact values (7 tests × 3 ranks green, MPICH
5.0.1). Flux payload suite over InProcTransport (5 tests). Suite 124
green (lean).
…ransfer (Phase 6) SDK::Device follows the transport seam discipline: no vendor type in any public header. - DeviceBackend: vendor-neutral interface (space support, allocate/free, space-pair copies, synchronize); HostBackend always present (Host/HostPinned/Unified from host memory, Device rejected) so negotiation code needs no special casing without accelerators - KokkosBackend (USE_KOKKOS; vcpkg gpu feature): first accelerator backend, every Kokkos type confined to the .cpp; whatever execution space Kokkos was configured with (Serial/OpenMP/CUDA/HIP/SYCL) serves MemorySpace::Device, so one conformance suite proves the seam on a laptop and on an accelerator; byte copies via deep_copy over unmanaged views; initializes Kokkos only when nothing else has - DeviceBufferRegistry: owns a component's device mirrors in one place — signature-checked acquire (kind/shape/space; mismatched re-acquire refused), upload/download through the backend, typed descriptor views, all allocations freed on destruction - Space-aware exchange: negotiateCopyPath() decides the cheapest legal path once at prepare() (HostHyperslab with full strides; BackendCopy for contiguous endpoints; StageThroughHost when a strided host view meets a device buffer; strided device views rejected as negotiation errors, never silently degraded); copyBuffer() executes it with kind/count validated up front Tests: backend conformance parameterized over host+kokkos runs identically — allocation round trips in every supported space, bad-request rejection, registry lifetimes and diagnostics, the negotiation table, and the device-resident producer exchange (strided host → device → contiguous host → strided host, no user copies). Suite 137 green (kokkos+mpi) / 131 (lean); Kokkos 4.5.01 built PIC. Deferred to a future minor: GPU-aware-MPI direct path and ValuePacker device fast paths (both sit behind copyBuffer).
A temporal dimension over the existing typed stores, per the plan:
- IdBasedInput<T>/IdBasedOutput<T>: IIdBasedComponentDataItem over the
id storage mixin — identifier dimension 0, provider matching on
DataKind + identifier count, positional default pull (id reordering
belongs to adapted outputs)
- TimeIdBasedComponentDataItem<T>: ITimeIdBasedComponentDataItem with
{time, identifier} storage
- SDK::SpatioTemporal items implementing the standard's combinations:
TimeGeometryComponentDataItem<T> ({time, geometry}; EnvelopeAdapter
bridges the SDK Envelope value type to IEnvelope),
TimeNetworkComponentDataItem<T> ({time, entity}; edge/vertex
dimension accessors mirror the attachment),
TimeSeriesPolyhedralSurfaceComponentDataItem<T> +
TimeSeriesTINComponentDataItem<T> ({time, patch|edge|vertex}), and
TimeRegularGrid2D/3DComponentDataItem<T> serving rank-3 {time, y, x}
and rank-4 {time, z, y, x} hyperslabs directly over the row-major
time-series block
Design decision: the SDK spatial geometry classes are value types, so
the items' geometry associations (INetwork, IPolyhedralSurface, ITIN,
IRegularGrid, IGeometry) are borrowed interface pointers supplied by
the owning component; counts, dimensions, envelope, and the whole data
plane are owned and real. Open rows: interface-conformant adapters for
the SDK geometry classes; GDAL-gated time-raster item.
Tests: 9 suites-worth — id round trips + provider rejection, {time, id}
slabs, envelope adapter, entity-dimension mirroring, rank-3/rank-4 grid
windows read and written. Suite 146 green (kokkos+mpi) / 140 (lean).
…, IDW (Phase 7b) HydroCoupleTools (BUILD_TOOLS; namespace HydroCouple::SDK::Tools). Every generator emits UGRID-congruent MeshDefinition/SoA structures directly consumable by the IO writers and mesh data items. The dead legacy tools CMake block (nonexistent sources, hard GDAL dependency, non-MIT Triangle) is replaced; CDT (MIT) is the Delaunay engine, found via the vcpkg tools feature or a plain CDT_INCLUDE_DIR. - Terrain-following sigma grids: SigmaLayering (uniform + Song–Haidvogel-style theta stretching with surface/bed blend, validated); SigmaGridGenerator::build (interfaces pinned to surface and bed, min-thickness via proportional surplus shrink, equal-layer collapse for shallow columns, depth preserved exactly); extrude (any horizontal mesh into layered prisms/hexahedra following the terrain) - Curvilinear grids: transfinite interpolation with corner validation, Winslow elliptic smoothing, quality metrics as first-class API (min corner Jacobian, orthogonality deviation), UGRID quad export, graded 3-D extrusion (stretching in the level spacing, composable with SigmaLayering) - Constrained Delaunay triangulation over CDT: boundary + holes + breakline constraints + Steiner points, max-edge-length densification as the refinement control, CCW triangles - Quad-dominant meshing: greedy best-quality-first pair merge with a corner-angle metric; non-convex pairs rejected; leftover triangles retained in a mixed-element mesh - IDW terrain sampling with search radius and exact-hit handling Tests (12): monotone interfaces + min thickness + exact depth over synthetic terrain; wedge extrusion counts and top-above-bottom; quarter annulus TFI with positive Jacobians; Winslow recovery of a perturbed grid; square-with-hole Euler characteristic (V−E+F = 1−holes) with CCW faces and densification bounds; breakline conformance; structured pair merge vs threshold-blocked skewed pairs; IDW radius diagnostics; and the end-to-end triangulate → sample → sigma → extrude pipeline. Suite 157 green (kokkos+mpi+tools) / 151 (lean+tools).
…n complete) Examples (BUILD_EXAMPLES): - serial_coupling: two-component composition initialized from YAML (or identical JSON), orchestrated by TimeSteppedWorkflow, persisted through the IOThread to CSV + GeoPackage + HDF5/UGRID, composition round-tripped - distributed_coupling: the same reach chain decomposed across MPI ranks with PartitionedDataItem + halo exchange and the flux+derivative boundary convention Each also carries a standalone CMakeLists.txt that consumes only the INSTALLED package — the acceptance test for the export. That test found four real install defects, all fixed: - the CMake package installed to lib/cmake/, where find_package cannot see it → lib/cmake/HydroCoupleSDK/ - HydroCoupleSDKConfig.cmake emitted empty path variables and resolved no dependencies → correct PATH_VARS, records how the SDK was built (WITH_MPI/KOKKOS/TOOLS/...), and find_dependency for MPI/GDAL/HydroCouple/nlohmann_json - MPI linked PRIVATE although USE_MPI is PUBLIC and the installed abstractmodelcomponent.h includes <mpi.h> → PUBLIC - interface headers and the nlohmann single header were unreachable from an installed SDK → consumed from HydroCouple::HydroCouple when the interface package is present, otherwise installed alongside, so an installed SDK is self-contained Also: CHANGELOG.md per project convention (critical commits + known limitations), Readme refreshed with the layer overview / examples / find_package contract, CPack verified, and CI extended — the matrix job builds tools+examples, runs the serial example and an install-and-consume smoke test; a new ubuntu job runs the MPI suite and the 4-rank example. Verification: lean core 139 green; everything-on 156 green + 7x3 MPI; both examples run in-tree and standalone; mass conserved exactly across 3 and 4 ranks.
Promotes the ModelInitializer document to an SDK-owned, versioned specification that replaces the Composer's XML. - CompositionSpec: strongly typed parse of the whole document — schema_version, metadata, component blocks (caption, info instantiation hint, execution mode + results manifest, arguments), connections with ordered adapted-output chains and IMultiInput provider roles, workflow strategy/trigger/iteration settings, writer blocks, and the run manifest path. Structural validation catches duplicate ids, undeclared connection endpoints, bad enum spellings, and missing required companions (open without results, pull_driven without trigger) - ModelInitializer applies the spec: adaptation chains built through IAdaptedOutputFactory (created outputs owned by the initializer, their arguments applied before initialize()), multi-input roles resolved by label, and Open-mode components may legitimately reach Finished during initialize(). validateDocument() validates without applying and never loads libraries — naming a library makes a document executable content, so resolution stays with the caller's resolver - serialize() emits v1 documents, carrying forward the blocks the initializer does not regenerate (metadata, workflow, writers, run, per-component execution) - Formal JSON Schemas (draft 2020-12) for compositions and run manifests, installed to share/hydrocouplesdk/schema/ for editors and external tools Blocks the SDK cannot act on itself (writers are feature-gated, workflows live outside SDK::IO) are carried as validated data via spec(). Tests: 17 spec tests — every block parsed, minimal documents still valid, full round trip through toJson(), enum round trips, and one negative per failure mode asserting the message names the offending element. Schemas verified against the examples and the same negatives. Suite 173 green.
A finished run is now described explicitly rather than inferred from
whatever files happen to exist on disk.
- ResultEntry: one catalog row — artifact, format, variable, DataKind,
shape, dimension names, units, mesh + location, and an explicit time
axis (units recorded, never guessed). Round-trips through JSON with
actionable diagnostics for malformed entries
- IOutputWriter::catalog(): defaulted to empty so third-party writers
keep compiling, implemented by all four shipped writers. This is the
necessary addition — a Snapshot's FieldSlice carries only id/kind/
shape, while units, mesh attachment, location, and the in-artifact
variable name are writer-side knowledge. The long-format writers (CSV,
GeoPackage) accumulate the { time, elements } extents they observe,
since neither file states them
- RunManifest: run identity/timing/status, SDK + interface versions,
composition reference (path, optional hash, optional embedded copy),
per-component final status and diagnostics, and the results catalog.
Reading records the manifest's directory so relative artifact paths
resolve; entry()/entriesFor()/artifactPath() are the reader-side index
- RunRecorder: assembles a manifest from live writers and components,
attributing catalog entries to their producing component. Writing is
opt-in, which is what keeps result reuse a declared choice rather than
an accident of filesystem state
- The serial example now records run.json alongside its artifacts
Tests: 8 covering entry round trips and malformed-entry diagnostics, the
CSV catalog matching what was written (extent, kinds, time bounds),
manifest write/read with artifact path resolution, version and shape
rejection, and RunRecorder assembling from an IOThread-driven writer.
Generated manifests validated against the shipped JSON Schema. Suite 181
green.
Delivers the plan's headline capability: a stored run is presented as an ordinary IModelComponent whose recorded items serve values straight out of the run's artifacts, so analysis and visualization tools need no code for the models that produced them. - IResultReader + registry: hyperslab reads guided entirely by the catalog entry (shape, kind, variable, time axis are never inferred from the file). CSV always available — indexed once on open, since a long-format file cannot answer a slab without a scan; HDF5, NetCDF, and GeoPackage readers behind the same gates as their writers, with a build that lacks one saying so plainly instead of failing to link - ResultsModelComponent: opens the readers, exposes one lazily-backed item per catalog entry through results() AND outputs(), and walks Initializing → Initialized → Finishing → Finished, because a recorded run has by definition already computed its values. Recorded items are read-only; setValuesFrom() is refused rather than silently accepted. Missing artifacts, unreadable formats, and unknown components leave the component Failed with a Fatal diagnostic naming the cause - An item recorded by several writers is still one logical variable: it is exposed once, served by the most capable format this build can read (HDF5 > NetCDF > GeoPackage > CSV). This also avoids duplicate output ids in the component registry Tests: reopened values byte-identical (memcmp) to the live run, with the reopened side constructing no model component; arbitrary hyperslabs (time level, single-reach series, interior window); metadata and time axis survival; format preference; read-only enforcement; missing artifact and unknown component diagnostics; CSV and HDF5 artifacts of the same run agreeing element-wise. Suite 190 green (io features) / 188 (lean).
- ModelInitializer honors execution.mode. "open" first lets the
resolver's component load its own results (a component that finishes
during initialize() is taken at its word); otherwise, and whenever the
resolver has no instance at all — the norm for an analysis host with no
model libraries — a ResultsModelComponent is substituted from the
declared manifest. Connections resolve to substitutes via component().
"resume" is diagnosed as reserved rather than silently ignored
- compositionForRun(): a manifest becomes a composition document of
open-mode blocks, so a UI can hand a finished run straight back to the
initializer
- TimeSliceOutput: the missing adapter this slice exposed. A recorded
item is { time, entities… } — history — while a live input wants
{ entities… } — the present, so a stored run could not actually drive a
live consumer. TimeSliceOutput presents one level of a time-major item
as a lower-rank output, with the level selected explicitly
(setTimeIndex/advance/setTime); the SDK never guesses which level is
"now", because it has no global clock. Views are read-only
Tests: open-mode substitution with no live component; a self-loading
component left alone; resume diagnosed; TimeSliceOutput level selection,
clamping, nearest-time seek and read-only enforcement; manifest →
composition round trip initializing an analysis composition; and the
equivalence claim — a reopened upstream driving a live downstream
reproduces the all-live results exactly. Suite 203 green (io features) /
194 (lean).
- analysis_reopen example: reads a run manifest, rebuilds the analysis composition with compositionForRun(), and summarizes every recorded item — linking only the SDK, with no model components anywhere. Builds standalone against an installed SDK, which is how it earns the claim - Two honest gaps it exposed and fixed: the GeoPackage reader queried a column that does not exist (the writer's schema is variables.item_id), and the serial example attached an HDF5 writer without declaring its fields, so results.h5 held a mesh but no data - Two further install-export defects, same class as the v2.0 ones: the exported link interface names PRIVATE dependencies (CMake records them as $<LINK_ONLY:...>), so HydroCoupleSDKConfig.cmake now resolves HDF5 +ZLIB, netCDF, SQLite, Kokkos and yaml-cpp through the same fallback chains the SDK used. Without this, consuming an IO-enabled install failed at configure or link time - CHANGELOG entry for the unreleased 2.1.0 work; Readme section on composition documents and stored runs; CI runs the analysis example and validates the example composition and a generated manifest against the shipped JSON Schemas Suite 203 green (io features + examples) / 194 (lean).
The plan's testing standard called for per-format round trips across all four formats; only CSV and HDF5 had them. That gap is how the GeoPackage reader's wrong SQL column reached a commit — it was caught by running the analysis example by hand, not by the suite. The reopen fixture now records through every writer the build has (CSV always; HDF5, NetCDF, GeoPackage when enabled), and two tests walk the manifest's catalog: - EveryRecordedFormatReadsBackTheSameValues: each entry's format must be readable by this build (a build that can write a format but not read it is itself a defect), and the whole variable must match the producer's values element for element - EveryRecordedFormatServesHyperslabsAndTimes: per format, a single time level, a single-reach series, time coordinates (stored or reconstructed from the recorded axis), and refusal of out-of-range selections Verified non-vacuous: the fixture's manifest carries all four formats, and reintroducing the old v.name/v.item_id bug fails both tests. Suite 204 green (all formats + examples) / 196 (lean).
.claude/settings.local.json (machine-specific tool permissions) and the plans/ folder (working notes) were tracked. Both are local working state rather than project content, so they are untracked and ignored. The files remain on disk.
… docs The Doxygen header was copied from the interface repository and never adjusted: 'View source on GitHub' sent readers of the SDK docs to the HydroCouple interface repository, and it carried a relative Python API link that does not resolve from this site. - GitHub corner now points at HydroCouple/HydroCoupleSDK - Navbar carries absolute cross-links to the interface standard and the Python bindings (absolute because these are separate GitHub Pages projects; a relative path breaks as soon as either moves) - Readme gains a documentation table naming all three sites and noting that Pages deploys from master/dev only No CNAME is added here: project sites inherit the organization's custom domain, so this repository publishes at https://www.hydrocouple.org/HydroCoupleSDK/ once Pages is enabled.
Every job in the matrix failed in dependency resolution, before any
compilation, because the manifest named features the ports do not have:
- netcdf-c's HDF5 backing is the "netcdf-4" feature; there is no
"hdf5" feature. This is what the logs reported.
- hdf5[threadsafe] is mutually exclusive with the hl/cpp features the
port needs, and would have force-enabled HDF5_ALLOW_UNSUPPORTED. The
SDK's concurrency guarantee is the single-writer IO thread, so
threadsafe buys nothing and costs correctness.
- the "tools" feature pulled GDAL though HydroCoupleTools links only
CDT.
Beyond the reported failures:
- MpiTransportSuite hardcoded three ranks and failed on runners with
fewer cores. Ranks are now HYDROCOUPLESDK_MPI_TEST_RANKS and
launcher flags come from MPIEXEC_PREFLAGS, because OpenMPI needs
--oversubscribe and MPICH rejects it - that choice belongs to the
environment, not to the build file.
- the version-alignment check silently passed whenever the interface
came from a source checkout (it read only a generated version.h);
it now falls back to project(HydroCouple VERSION ...).
- version.h was generated into the source tree and tracked, so every
configure dirtied the tree and commits recorded whichever branch was
checked out when someone last configured. It is generated into the
build tree now, installed from there, and gitignored.
- Windows test/example targets now get their dependent DLLs copied
beside them, without which they cannot launch.
- the sidecar benchmark asserted a 10 s wall clock, which measures the
runner and flakes under load. The <4 KB document assertion is the
real proof the bulk path was taken; the timing bound is now a
pathology guard.
- CDT is MPL-2.0 AND BSD-3-Clause, not MIT.
Verified: clean configure + build + 180/180 tests on Linux in the core
configuration, and an install tree carrying exactly one version.h with
correct provenance.
CMake generation failed on ubuntu, macos and windows with
Target "HydroCoupleSDK" links to: HDF5::HDF5
but the target was not found.
even though configure had just reported "HDF5 target:
hdf5::hdf5-static;ZLIB::ZLIB" two lines above. The detection block
resolves the right target into HYDROCOUPLESDK_HDF5_TARGET and line 370
links it. Line 421 then linked HDF5::HDF5 by hardcoded name - a second,
redundant link, and only module-mode FindHDF5 defines that target. vcpkg
uses config mode and exports hdf5::hdf5-static, so this failed on every
runner while configuring fine on any machine that happens to resolve HDF5
through module mode. That is why local builds never caught it.
Removed the whole redundant block, including its netcdf-cxx4 branch: the
SDK includes <netcdf.h> and uses the C API, and netcdf-cxx4 was never a
declared dependency. The remaining links stay PRIVATE, which is correct -
no installed public header includes hdf5.h or netcdf.h, so consumers need
neither.
Verified by reproducing the failure locally: configuring against stub
config packages that export hdf5::hdf5-static / netCDF::netcdf /
unofficial::sqlite3::sqlite3 (the vcpkg names, with no HDF5::HDF5)
reproduces the exact error at the exact line before the change and
generates cleanly after. Core suite still 180/180.
The MPI job is already green in the same run - 181/181 including
MpiTransportSuite - so the rank/oversubscribe fix landed correctly.
Ubuntu reached 216/216 and then failed installing and consuming the
package from an external project:
HydroCoupleSDKConfig.cmake:54 (find_dependency)
Could not find a package configuration file provided by "hdf5"
That step is a faithful stand-in for a downstream user, and it caught a
real packaging defect. The config called find_dependency() for HDF5,
NetCDF, SQLite, yaml-cpp and Kokkos, so consuming an HDF5-enabled SDK
required finding HDF5 - a dependency the consumer never asked for and
which is entirely encapsulated in the shared library.
The comment justifying those calls said CMake exports PRIVATE
dependencies as $<LINK_ONLY:...>. I first "fixed" that by wrapping every
private dep in $<BUILD_INTERFACE:>, then tested the claim: it holds for
STATIC libraries only. A minimal project exporting the same target two
ways shows a static `plain` exporting $<LINK_ONLY:M::fakedep> while the
shared one exports nothing at all. HydroCoupleSDK is SHARED, so nothing
was leaking and the wrapping was unnecessary - worse, it would silently
strip genuinely-needed deps if the library ever became static. Reverted;
only the config template changed, and its comment now says what is
actually true.
Windows failed differently: the job runs Ninja, which needs cl.exe on
PATH. Without the MSVC environment CMake selected the runner's MinGW
g++, which received /utf-8 from the MSVC-built vcpkg packages and read it
as a linker input file. The flag is a symptom - MinGW objects and
x64-windows MSVC packages would not have linked either. Set up MSVC
before vcpkg (vcvars64 also exports VCPKG_ROOT, so ordering matters),
matching openswmm.engine.
macOS is NOT fixed here. Its four NetCDF writer tests write successfully
and then fail to reopen with NC_EHDFERR (-101). Same hdf5 2.1.1 and
netcdf-c 4.9.3 as ubuntu, which passes, and the SDK's direct HDF5 writer
- which also deflates - passes on macOS, so the zlib filter is fine. I
cannot reproduce macOS here, so rather than guess, nc_open failures now
report nc_strerror, the file size, the netcdf library version and the
HDF5 error stack, which the bare numeric assertion did not.
Verified: the rendered config drops to HydroCouple/nlohmann_json/MPI/GDAL
with all backends ON, the stub-package configure still generates, the
core suite is 180/180, and the reworked test file passes -fsyntax-only
against the real netcdf and hdf5 headers.
Windows now genuinely uses MSVC (last commit's fix worked) and promptly found a real defect. __declspec(dllexport) instantiates every member of a class, including the implicit copy assignment; against std::vector<std::unique_ptr<...>> that is ill-formed. GCC and Clang only instantiate it on use, so ModelInitializer compiled everywhere else. Made ModelInitializer explicitly move-only, then scanned the rest of the exported surface rather than waiting for the next file to fail: seven more classes own unique_ptr, mutex or atomic state without declaring their copy operations. All eight already declared destructors, so none were movable and none were copyable in practice - deleting copy changes no behaviour, it only stops MSVC generating a member that cannot exist. Ubuntu reached 216/216 and then failed the standalone consume on nlohmann_json. That one is not like the HDF5 case fixed last commit: nlohmann/json.hpp is included by the SDK's public headers, so it is a genuine usage requirement and find_dependency() is right to resolve it. The example build simply had no vcpkg tree of its own; it now uses the one the SDK was built against, which is what a real consumer has. macOS is still four NetCDF tests. The diagnostics added last commit paid off and correct something I said earlier: this is NOT a product defect. ResultsReopenTest.EveryRecordedFormatReadsBackTheSameValues writes a NetCDF file and reads it back successfully on macOS, so the SDK reads its own output fine there. What fails is nc_open called from the *test executable*, which links its own static netcdf and hdf5 alongside the copies already inside libHydroCoupleSDK.dylib - the linker even warns "ignoring duplicate libraries: libhdf5.a". Two instances of those libraries in one process is unsupported, and macOS's two-level namespace keeps them separate where Linux collapses them. The file is 28 KB and well-formed; the HDF5 error stack printed empty because the failure is not in the instance the test can see. Verified: clean build, 180/180 core suite, workflow YAML parses.
The MPI job becomes a matrix over ubuntu-latest and windows-latest, with MS-MPI installed on Windows per https://learn.microsoft.com/en-us/message-passing-interface/microsoft-mpi Wiring it up surfaced a real bug first. MpiEnvironment requested MPI_THREAD_MULTIPLE and then ignored the out-parameter reporting what was actually granted, so the SDK ran as though full multithreading had been guaranteed regardless of what the implementation said. Nothing in the distributed layer starts a thread and the IO thread never calls MPI, so the genuine requirement is MPI_THREAD_FUNNELED. It now requests SERIALIZED - leaving room for a host to drive components from a worker thread - queries the level when MPI was initialized elsewhere, and exposes providedThreadLevel() and threadLevelSufficient() so a host can check rather than assume. MS-MPI does not offer MULTIPLE, so the old request would have quietly mis-set expectations there. Launcher differences stay out of the build files, as with the earlier rank-count fix: OpenMPI gets --oversubscribe through MPIEXEC_PREFLAGS, MS-MPI gets none since it rejects unknown flags and oversubscribes by default. The transport suite is launched through ctest so it uses whichever mpiexec CMake found, rather than spelling a launcher here. Verified on MPICH in the sandbox: 181/181 with empty preflags, and a probe confirms SERIALIZED is requested and granted (provided=2, sufficient=1). MPICH also rejects --oversubscribe outright, which is a live demonstration of why that flag is not hardcoded.
Mirrors the badge rows the interface and openswmm.engine readmes carry: the two workflows this repo actually has, plus the existing MIT badge (now pointing at a stable URL rather than a relative path) and a C++20 marker. Badges report the default branch, which is master. The workflows currently run on removing_qt, so these will read "no status" until that branch lands on master.
Two changes that together make the next macOS run decisive, whichever
hypothesis is right.
The probe: when nc_open fails, the test now reads the same file, in the
same process, through the SDK's own NetCDF result reader - i.e. from
inside libHydroCoupleSDK rather than from the test executable. The two
are distinguishable because the test binary links its own copy of netcdf
and hdf5 next to the copies already in the shared library; the macOS
linker says so ("ignoring duplicate libraries: libhdf5.a"). If the SDK
read succeeds where direct nc_open fails, the file is fine and the
failure belongs to this executable's library instance. If it also fails,
the writer really did produce something unreadable and the flush path is
where to look.
The trial: nc_sync before nc_close in the writer's finalize, as suggested
elsewhere. Marked EXPERIMENT and to be removed if inert, because I do not
believe it can be the cause: nc_close already writes buffered data by
contract, and in the same macOS run
ResultsReopenTest.EveryRecordedFormatReadsBackTheSameValues writes a
NetCDF file with this writer, reads every value back through the SDK
reader, and passes. An unflushed file cannot pass that and fail the
direct nc_open in the same run. It costs one line to rule out, and it is
deliberately non-fatal - nc_sync is invalid in define mode, and a
diagnostic aid must not become a new failure path.
Because the probe only runs on failure and changes no behaviour, the next
run is unambiguous: green means nc_sync was the fix and my reading was
wrong; still red means the probe names the culprit.
Verified: both files pass -fsyntax-only against the real netcdf and hdf5
headers, createResultReader behaves as the probe expects (returns nullptr
with a clear message in a build without NetCDF), core suite 180/180.
MSVC could not link the DLL: LNK2019: unresolved external symbol AbstractAdaptedOutputFactoryComponentInfo:: AbstractAdaptedOutputFactoryComponentInfo(std::string_view) The diagnosis offered was right: the constructor is declared in include/hydrocouplesdk/data/abstractadaptedoutputfactorycomponentinfo.h and defined nowhere. Every sibling *ComponentInfo carries its out-of-line constructor in its own .cpp - componentinfo.cpp, abstractmodelcomponentinfo.cpp - and this one's was simply never written. Added it following that convention and listed it in src/CMakeLists.txt. Confirmed rather than assumed: nm on the library built before this change finds zero occurrences of the symbol; after, it is present and exported. The linker had nothing to bind to. It survived this long because the class is abstract and GCC and Clang emit only what is used, so nothing referenced the constructor. MSVC's __declspec(dllexport) requires a definition for every declared member - the same mechanism that exposed the move-only copy-assignment defect in the previous commit. Two latent defects, both hidden by toolchains that do not force instantiation. Also swept the exported surface for other declared-but-undefined constructors and found none; the only matches were false positives from `return Point(...)` inside inline operators. That sweep covers constructors only, so if the Windows link reports further LNK2019s they will want the same treatment. Verified: clean build, core suite 180/180.
macOS built hdf5/netcdf from the default arm64-osx triplet's static archives, so each was absorbed twice: once into libHydroCoupleSDK.dylib, which re-exports every symbol it pulls from a prebuilt C archive since -fvisibility=hidden cannot reach one, and once into the test executable. The executable's netcdf then bound its H5 calls to the dylib's instance while the direct HDF5 tests used its own, and the split state made nc_open fail with NC_EHDFERR on the four NetCDFUGRIDWriterTest cases -- on files whose validity the suite's own probe confirmed by reopening them through the SDK reader. macOS now configures with the *-osx-dynamic triplet, leaving one instance per process, as on Windows. Linux is unaffected: ELF's flat namespace interposes the executable's copy everywhere, which is why the same static triplet passes there. The Windows job's package-consumption step then failed to find the package it had just installed. CMAKE_PREFIX_PATH is a ';'-separated list, and MSYS reads that separator as proof the value is already in Windows form, so Git Bash's /d/a/... reached CMake verbatim; a lone path argument such as the cmake --install prefix *is* converted, which is why the install itself looked correct. Each entry now goes through cygpath -m, a no-op where cygpath does not exist.
Two gaps came out of building spatial data-item layers in Composer. EnvelopeAdapter was the only implementation of any spatial interface in the SDK. A component wanting to publish a geometry data item therefore had to implement IGeometry itself -- thirty-two methods -- plus IPoint, IPolygon or INetwork as applicable, and every component would have done it differently. PointAdapter, LineStringAdapter, PolygonAdapter, VertexAdapter, EdgeAdapter, NetworkAdapter, PolyhedralSurfaceAdapter and MeshViewAdapter follow the EnvelopeAdapter precedent: the value types stay values, and the adapter owns a copy and serves the interface over it. The relational predicates and constructive operations need a geometry engine the SDK does not link, so they report false or nullptr rather than guessing; the WKB the adapters do produce is what a caller with GEOS or GDAL uses instead. The UGRID writers had no counterpart, so the SDK could produce files that nothing in the ecosystem could read back. The reader is driven by the conventions rather than by our own variable names -- the topology variable is found by its cf_role and the coordinates and connectivity by the attributes it names -- so a mesh written by another tool reads as readily as one of ours. It honours start_index and drops _FillValue padding. ugridReadSupported() answers at runtime, so a consumer linking a prebuilt SDK need not know how it was configured. Reading back what we write immediately found a writer bug: node_coordinates listed only _node_x and _node_y, so an elevation written as _node_z was in the file but undiscoverable, since a conventions-driven reader looks for coordinates through that attribute. Fixed; elevations round-trip. 17 new tests. The index-convention fixture is written as classic netCDF-3, which needs no HDF5 -- that both sidesteps the macOS split-HDF5 failure and proves the reader is not tied to NetCDF-4. 229/233, and the four failures are the pre-existing NetCDFUGRIDWriterTest ones, confirmed against a stashed baseline.
A layered UGRID file is a two-dimensional topology with a sigma coordinate over it, which is how FVQual writes a water column and how CF says to describe one. readVerticalCoordinate() returns the interfaces, the per-face bed depth and the water surface at a chosen time, so a consumer can rebuild real elevations without agreeing with any particular writer. Found by convention, not by a writer's variable names. The coordinate is the variable whose standard_name is an ocean_sigma_coordinate, and its formula_terms names the sigma, eta and depth variables it is built from. Interfaces come from a CF bounds attribute when there is one; failing that, from a variable one longer than the layers whose values run 0 to -1 -- which is what an interface sigma is, whatever it happens to be called; failing that, they are midpointed from the centres and flagged as derived, since midpointing is exact for a uniform distribution and an approximation otherwise, and a caller computing volumes needs to know which it has. NetCDFUGRIDWriter::setVerticalCoordinate() is the other half. Without it the reader could read vertical coordinates the SDK itself could not produce -- the same asymmetry the node_coordinates fix closed, and the same way of finding it: writing one and reading it back. The surface may carry several time levels. A layered mesh moves, and the surface is what turns sigma into elevations, so a viewer stepping through a run reloads the column rather than only the values on it. 16 new tests, 242 total. 10/10 mutations bite. Three of those needed fixtures with decoys defined *before* the real variables -- the reader scans by variable id, so a decoy appended afterwards is never reached and proves nothing. The four failing NetCDFUGRIDWriterTest cases are the pre-existing macOS split-HDF5 ones, unchanged.
Signed-off-by: cbuahin <caleb.buahin@gmail.com>
A reopened run answered for its values and its time axis and nothing else. Every catalog entry names what its values are attached to -- the mesh and the location -- so that a reader would not have to infer it, but nothing read that field back, which left it a comment: a finished run could be plotted and could not be mapped. IResultReader::readMesh is additive and defaulted, because an artifact of values alone is a complete recording of values and not a failure. It answers with a reason instead, naming the format, so a caller can tell "this one has none" from "this one is broken". The two UGRID formats share one path into readUGRIDMesh, which already understands start_index and the fill padding a rectangular connectivity array carries. An entry naming no mesh is refused rather than given the first mesh in the file: a UGRID file may hold a 1-D network and a 2-D floodplain both, and geometry the values have nothing to do with is worse than no geometry. GeoPackage answers with its node points and no connectivity, because that is what the writer stores; the alternative is inventing faces. Its blob decoder steps over the envelope by the size the header declares rather than assuming there is none -- the SDK writes none, but a GeoPackage is a standard and an artifact that has been through another tool may carry one. The fixture's mesh gained a face, because with nodes alone every connectivity assertion compared zero against zero, and each format is now checked against the mesh that was written rather than against a second read of the same file. Six mutations, all caught. Two of the tests exist because a mutation survived first: an envelope-carrying point, and an empty node table reported as a mesh -- which is the worst of both, a consumer that draws nothing and is told nothing was wrong.
Reading the geometry was half of it. Until the item itself answers as a spatial one, a consumer has to know which artifact a run was recorded in and open it a second time to pair the two halves -- which is the duplication the manifest exists to prevent, and it leaves the geometry invisible to everything else that reopens a run. The item is a separate class rather than a nullable surface on the existing one. The interface is the answer to "can this be mapped?", and an item that claimed IPolyhedralSurfaceComponentDataItem with nothing behind it would say yes and then hand back nothing -- turning a question that can be asked once into one that has to be asked twice. A CSV of values stays exactly what it was. One interface covers all three cases rather than a geometry item for point clouds and a surface for meshes: a consumer should not have to switch on how much topology an artifact happened to record. The entry's location chooses the entity, and the other two dimension accessors answer nullptr, which is what the SDK's own spatiotemporal items already promise. MeshDefinition to PolyhedralSurface is the crossing between the SDK's serialization vocabulary and its geometry one -- a structure of arrays because that is what a UGRID file is, objects because that is what a consumer draws and picks against. It lives in the spatial adapters, where a mesh from any source can use it. The mesh is read once, when the item is built. A mesh does not move between time steps, and re-reading it per step would mean paying for the whole topology to draw one frame. Twelve mutations, all caught. The last one written was the ring left open: every patch still counts as a patch, and the map looks like a mesh until you notice each cell is missing a side.
A PolyhedralSurfaceAdapter rebuilt its mesh view by walking the surface's vertices, so the view reported no faces and no edges for a surface that had both. The standard calls meshView() "the accessor partitioners, interpolating adapters, IO writers, and device staging must use" -- a view answering zero for a mesh that is not empty is not a lesser answer, it is a wrong one, and it is wrong in the shape of an empty mesh rather than of a missing feature. It cannot be recovered after the fact: a PolyhedralSurface records polygons, not which vertex indices each polygon used, so faces and edges could only be rebuilt by matching coordinates back to vertices -- which guesses, and guesses wrong wherever two vertices coincide. So the caller that has the connectivity passes it, and the constructor that does not get it keeps today's behaviour with the reason written down. A reopened run's item is exactly such a caller: it read the mesh a moment earlier and was dropping it on the floor one call before it would have been kept. Edges are the half of this that survives nothing by accident. The fixture mesh gained its three, and the per-format geometry test now checks them too -- the UGRID writers were already recording edge_node_connectivity and the reader was already reading it back, so the loss was entirely in the crossing from arrays to objects. Three more mutations, all caught. Fifteen now.
Implements ILayeredMeshComponentDataItem: mesh entity at dimension 0,
layer at dimension 1. That ordering is not a free choice —
IPolyhedralSurfaceComponentDataItem fixes the entity as dimension 0 with
extra dimensions following, and the item has to stay readable through
that base by a partner that knows nothing about layers.
The vertical coordinate is borrowed and handed out live rather than
snapshotted at construction. Under a terrain-following coordinate every
layer elevation moves as the free surface does, so an item holding a copy
would answer confidently and wrongly for the rest of the run, and nothing
about the values it returned would look wrong.
Five gates: dimension ordering, hyperslab round-trip, a partial slab
touching only its own cells, the coordinate staying live under a moving
stage, and readability through both the surface base and the ILayering
mixin. Three ablations confirm they fail — storage transposed to
{layer, entity} (4 failures), the coordinate dropped (2), and
layerDimension() aliased onto the entity dimension (1).
Header-only, so the existing include/ install glob ships it.
F8.2b delivered a layered data item that could not be exchanged. LayeredMeshComponentDataItem derives from AbstractComponentDataItem, but addInput() and addOutput() take AbstractInput* and AbstractOutput*, so nothing built on the plain base can be connected to anything. A layered item that cannot cross a coupling boundary is most of the way to useless, since crossing one is what it is for. These mirror exchangeitems1d.h — AbstractOutput (or AbstractInput) composed with the typed storage mixin, here the 2-D one — which is the same relationship Output1D has to ComponentDataItem1D. LayeredMeshInput::canConsume checks the layer count as well as the rank. Two layered models that disagree about how many layers they have still match on rank, and would produce a silently truncated exchange. Layer elevations are deliberately not compared: a sigma model and a z-level model legitimately place layers differently, and reconciling that is remapping, which belongs in an adapted output.
CONTRIBUTING and CONTRIBUTING.md were both tracked and byte-identical at 28158 bytes. Nothing in the build, packaging, or documentation referenced the extensionless copy, so it was a stale duplicate rather than a second document — and two files that have to be kept in step is one more than the project needs. The working tree already had it deleted; this records that deletion rather than reverting it.
The meshing tools are static functions that return when they are done, which is everything a batch caller needs and not enough for one driving a window: a domain of any size takes long enough that a user will want to know work is under way, and long enough that they will want to abandon it. MeshProgress is an optional callback, taken by new overloads so that every symbol these libraries already export is left alone. It is told the phase under way and answers whether to carry on. Refusing abandons the run, which then reports itself cancelled and leaves its output empty -- a cancelled run hands back nothing rather than half a mesh, and that holds even when the caller passed in a variable that already held one, which is what meshing twice into the same variable does. What is promised here is narrower than a progress bar implies, and the header says so rather than leaving a caller to find out. The fraction is how far through the phases a run is, not a measure of work remaining: the phases are not equal in cost. And the CDT insertion phases cannot be interrupted at all, being one call into a library that offers no hook, so a cancellation asked for while one runs takes effect when it returns. What can be stopped promptly is where these loops are ours -- densification and extraction in the triangulator, and every phase of the quad merge. A caller must not offer a cancel that lands sooner than it does.
Two defects with one symptom between them: a suite whose verdict could not be trusted. It reported four failures serially and a different set of seven to nine in parallel, and both had been standing long enough to be treated as a baseline to work around. The four were not a defect in the writer. nc_open returned NC_EHDFERR on a file the writer had just produced, while the SDK's own reader opened the same file without complaint -- the diagnostic already in the suite had been saying so. vcpkg supplies netCDF and HDF5 as static archives, and the test executable linked them as well as the copies already inside libHydroCoupleSDK, so one process held two sets of HDF5's globals and a file written through one instance could not be reopened through the other. The macOS linker had been reporting it on every build: "ignoring duplicate libraries: libhdf5.a". The test target now takes those headers without their libraries and resolves nc_* and H5* from the shared library, which is one instance and the same one that wrote the file. Every symbol the suite uses is among those the library exports, and the tests still check written files with the library rather than with our own reader, which is the point of them. HDF5 gets the same treatment as netCDF: nothing made its suite immune, it merely happened not to hand a handle across the two. The parallel set was a race over file names. ctest registers each gtest case as a test of its own and runs them as concurrent processes, and several cases asked for the same artifact -- io_ugrid.nc, io_ugrid.h5, io_results.gpkg, and a whole sidecars/ directory that every SidecarTest wiped in SetUp(). Each writes its file and reads it back, so in parallel they truncated and deleted each other's work mid-read. Artifact paths are now namespaced by the running case and the sidecar fixture gets a directory of its own; the files still land under tests/artifacts where they can be opened afterwards, and now say which case produced them. 260/260 serially, and 260/260 on five consecutive runs at -j 8.
URL has been in IArgument since v1, and AbstractArgument::isValidArgType()
has always answered true for it whenever an argument declares file filters.
Nothing ever fetched. All four typed arguments -- Argument1D, Argument2D,
IdBasedArgument and TimeSeriesArgumentDouble -- wrote
if (argType == AT::File || argType == AT::URL) { std::ifstream f(value); }
so every argument advertised a capability it did not have, and an https URL
came back as "Cannot open file: https://...", which sends the reader looking
for a file that was never meant to exist.
The SDK cannot close that itself. It has no HTTP client and deliberately no
Qt, and putting either underneath every model that links it is not a trade
worth making for this. So it now defines what resolving a URI means and
leaves the doing to the host: IO::UriResolver, installed once at start-up.
HydroCoupleComposer will install one backed by HydroCoupleOgc's fetch tier;
a command-line runner may install none and gets a refusal that names the
scheme instead of blaming a file.
What the SDK still does alone is everything it could do before: a plain path
and a file: URI are read directly, localhost included. Only an unknown scheme
needs a host.
Two things this exposed rather than introduced:
- A one-character URI scheme is legal per RFC 3986 and nobody uses one, but
every Windows path opens with a drive letter and a colon. A scheme is
therefore two characters or more, or C:\data\flow.json would be handed to
a resolver as a URI in the "c" scheme.
- saveData() writes back to whatever initialize() remembered, so remembering
a URL made a value fetched from a service savable to a file named after
the service. A reference is now remembered only when it is somewhere that
can be written to.
TimeSeries::loadFromCSV gained a loadFromCSVText sibling, because a series
fetched from a service arrives as bytes and never as a path. Its extension
is read from the reference with the query string removed --
path("flow.csv?token=x").extension() is ".csv?token=x", so a served series
would have gone to the JSON parser.
18 gates, 12 mutations. Two survived the first run and both were weak gates
rather than redundant code: a file: URI on another host was refused, but the
test could not tell that from the local path it fell back to also not
existing; and an https URL is not written to a file because "https:/" is not
a directory, which proves nothing about whether it was remembered. Both now
observe the decision instead of its side effect.
Suite 278/278.
The composition document reaches an argument through
initialize(payload, JSON, ...) and nothing else, so an argument fetched from
a service recorded its values and no memory of where they came from. Reopen
the composition a day later and it runs on yesterday's gauge, silently.
A payload may now name a URI instead of holding values:
{ "@uri": "https://example.org/flow.json" }
which is read as exactly initialize(uri, URL, ...) -- so a served CSV still
goes to the CSV parser, and nothing about the fetch is duplicated. The
sibling of sidecar.h's "@ref", which points at bulk bytes beside the
document; this one points at a resource somewhere else entirely.
IArgument::serialize's own contract asks for it: "the serialized form should
carry an external binary payload reference (URI ...)".
The write side is one override on AbstractArgument, because all four typed
arguments already delegate to AbstractArgument::readData/writeData. So a
fetched argument records "@uri" beside its values with no per-type work, and
Composer needs no plumbing at all: applyArgument already reads the payload
back from the component and records what it finds.
Two decisions worth naming:
- A URI is authoritative on reopen, and a service that cannot be reached is
a failure, not a fallback to the values recorded beside it. A composition
that quietly runs on a stale copy of live data is worse than one that
refuses and says which service it could not reach. The values stay in the
document because that is what makes a past run inspectable, and deleting
one line pins them.
- A local file is still inlined, as it always has been. A document can be
moved next to a file; it cannot be moved next to a service. Only what the
document cannot otherwise recover is recorded as a reference.
7 gates added, 25 in the suite, 18 mutations, none surviving. One gate found
a real slip on its first run: jsonUriReference wrote its out-parameter before
deciding whether there was a reference at all, so a payload with an empty
"@uri" returned false and cleared the caller's string on the way out.
Suite 285/285.
The SDK is already MIT and has been: the root License file is the MIT text, vcpkg.json declares MIT, the Readme badge says MIT, CPack points at opensource.org/licenses/MIT, and 201 of 202 source files carry SPDX-License-Identifier: MIT. This is a sweep for the places that had not caught up, not a relicensing. Three were behind: - src/stdafx.cpp carried a full LGPL-3.0 notice and a 2014-2018 copyright, the last file in the tree still doing so. It is the precompiled header's translation unit and its only statement was that licence, so it now carries the two-line house header like every file beside it. - CONTRIBUTING.md said "HydroCoupleSDK is released under the GNU Lesser General Public License v3" and asked contributors to agree their work would be made available under the LGPL-3.0. That is the most consequential of the three: it told contributors the wrong terms for the licence their work would actually go out under. Its dependency policy also required LGPL-3.0 compatibility and permitted LGPL dependencies, which under MIT would pull copyleft terms onto users of this SDK. - cmake/FindOpenMP.cmake had no SPDX tag. It is bespoke Homebrew probing rather than a copy of CMake's own module, so MIT is the right tag; every tracked code file now has one. Its broken link is fixed in passing, since it is the same sentence: the licence file is named License, not License.md. Four mentions of other licences are left exactly as they are, because they are true and changing them would misstate somebody else's terms: triangulator.h and CMakeLists.txt describe the bundled CDT library as MPL-2.0 AND BSD-3-Clause; CHANGELOG.md records that and an earlier fix for three places that had wrongly called CDT MIT; and CLA.md speaks of the Technical Manager's right to relicense contributions under MIT, LGPL, AGPL or a commercial licence, which is about future rights and not the licence in force. CONTRIBUTING now says outright that third-party components keep their own licences. Suite 285/285.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.