Add resample pipeline to @itk-wasm/downsample (transform + selectable interpolator)

[thewtex]

Summary

Adds a new resample pipeline to @itk-wasm/downsample / itkwasm-downsample. It wraps itk::ResampleImageFilter to resample a moving image onto a reference image's grid, with an optional transform and a selectable interpolator, across the full ITK-Wasm target matrix (WASI, Node, browser/Emscripten, and Python sync + async / WASI + Emscripten).

This was built in five phases: a working C++ prototype, VectorImage support + interpolator ctest coverage, regenerated TypeScript/Python bindings, test data + independent baselines, and the Node/browser/Python test suites.

What changed

C++ pipeline — packages/downsample/resample.cxx (new)

• Positional input (moving image), reference-image (geometry-only — an empty pixel buffer is accepted, since only origin/spacing/direction/size are read), and output.
• -t,--transform optional transform mapping output-grid points into moving-image space (defaults to identity).
• -i,--interpolator selectable from six methods: linear (default), nearest_neighbor, label_image, b_spline, windowed_sinc, gaussian.
• Supports 2D/3D/4D images over all scalar pixel types and itk::VectorImage (multi-component) types.

Build — packages/downsample/CMakeLists.txt

• Registers the resample executable and adds the required ITK modules (ITKImageFunction, ITKTransform) plus native transform IO (ITKIOTransformInsightLegacy, ITKIOTransformHDF5).
• Adds self-contained ctests: a baseline resample smoke test plus one per interpolator (identity resample of cthead1.png), and a label_image test on 2th_cthead1.png — all needing no new test data.

Bindings — regenerated TypeScript + Python

• TypeScript: resample / resampleNode with ResampleOptions (transform?: TransformList, interpolator?: string) and ResampleResult, exported from the package entry points.
• Python: sync + async resample across the wasi and emscripten sub-packages.

Test data & baselines

• New independent baselines and refreshed dam test-data-hash in package.json / test:data:download.

Tests

• Node: ava tests for the resample bindings.
• Browser: a demo-app controller + sample-input loader and a Playwright resample.spec.ts.
• Python: WASI test_resample.py.
• Adds @itk-wasm/transform-io (TS devDep) and itkwasm-transform-io (pixi) so transform inputs can be exercised in tests.

Why

The downsample package could shrink images but had no general way to map an image onto an arbitrary output grid with a transform and a chosen interpolation method — the standard "resample onto a reference geometry" operation. This adds that as a first-class, fully-bound pipeline.

Implementation notes

• Transform type: the option is read as itk::wasm::InputTransform<itk::AffineTransform<double, N>> rather than the abstract itk::Transform<double, N, N>. The abstract base won't compile — the memory-IO reader calls TTransform::New(), and itk::Transform has no itkNewMacro. The concrete double-precision AffineTransform is-a Transform<double, N, N> and still feeds SetTransform polymorphically.
• Geometry-only reference: the filter uses SetReferenceImage(...) + UseReferenceImageOn(), so the reference image contributes only its geometry.
• VectorImage path: ResampleImageFilter has no native multi-component support, so (mirroring downsample.cxx) each component is extracted (VectorIndexSelectionCastImageFilter), resampled through shared wiring, then recomposed (ComposeImageFilter). Per-component filters are kept alive until a single final compose update, because itk::DataObject only holds a weak pointer back to its producer.
• Shared wiring: reference-geometry / transform / interpolator setup lives in one MakeResampleFilter<TImage> helper reused by both the scalar and per-component vector paths.

[thewtex]

@sedghi @vboussot please take a look

This adds a pipeline to resample according to the sampling grid defined by a reference image. Another PR will add a pipeline to resample according to direct specification of the grid.

[sedghi]

I thought this would be a new /resample package, but it looks like you're putting it under downsample. Is that intentional?

---

These are the things Fable found

Findings (most severe first)

1. packages/downsample/resample-to-reference.cxx:170 — The --transform option is bound to InputTransform<itk::AffineTransform<double, N>> while the public API advertises a generic TransformList, and the deserializer never checks the transform parameterization. WasmTransformToTransformFilter's non-composite path validates only precision and dimensions, then raw-copies parameters into the affine. A Euler2DTransform from @itk-wasm/transform-io (3 params, dims 2→2, float64) passes every guard and silently produces a wrongly-resampled image; a BSpline transform (more params than the affine's N×(N+1)) makes std::copy write past the parameter buffer — wasm heap corruption; a composite list hits a null dynamic_cast and crashes; a multi-element non-composite list silently keeps only the last entry. Using itk::CompositeTransform<double, N> as the input type (as the core transform-read-write test pipeline does) would deserialize any factory-registered parameterization correctly and support chains.
2. packages/downsample/python/itkwasm-downsample-wasi/pyproject.toml:36 — All three Python packages keep itkwasm >= 1.0.b145, but the new bindings unconditionally do from itkwasm import ... TransformList at import time, and TransformList only exists in itkwasm ≥ 1.0b180. An environment holding itkwasm 1.0b145–b179 (which pip considers satisfied) gets ImportError on import itkwasm_downsample, breaking every pre-existing function, not just the new one. Same stale floor in itkwasm-downsample/pyproject.toml:37 and itkwasm-downsample-emscripten/pyproject.toml:36; bump to ≥ 1.0b180 (or transform-io's b185).
3. packages/downsample/python/itkwasm-downsample-wasi/itkwasm_downsample_wasi/resample_to_reference.py:73 — Interpolator validation is interpolator not in ('linear,nearest_neighbor,...') — a single comma-joined string, so in does substring matching, not membership. interpolator='near', 'sinc', or 'gauss' passes the check and dies deep in the wasm pipeline with an opaque CLI error instead of the intended ValueError. Root cause is in bindgen (wasi-function-module.js:237 splits choices on ', ' but they serialize without spaces) — this PR is the first artifact to hit that path, so fix the generator and regenerate.
4. packages/downsample/resample-to-reference.cxx:157 — The reference image is declared as InputImage using the moving image's exact pixel/component type, but the help text (propagated into every TS/Python docstring) promises "only the metadata is used, so an empty pixel buffer is acceptable." A routine cross-type call — float32 moving + uint8 reference — throws "Unexpected component type" at deserialization. The metadata-only claim is also never tested: the test-data tarball even packs cthead1-resample-reference-metadata-only.json "to exercise the empty-buffer path," yet nothing references it, and in Python data=None marshals through np.asarray(None).tobytes() into garbage bytes. Either relax the reference type, or fix the docs and exercise the fixture.
5. packages/downsample/resample-to-reference.cxx:90 — SelectInterpolator ends in a bare else that silently returns linear for unknown names, and the six-name interpolator list is hand-maintained in three places (the two CLI::IsMember lists at lines 183 and 254 plus this chain). Unreachable today because CLI11 gates input, but any future drift (a seventh name added to the CLI lists only) silently yields linear-interpolated output instead of an error — invisible to the run-success-only tests. Prefer a single name→factory table and an itkExceptionMacro on unknown names.
6. packages/downsample/typescript/README.md:26 — The bindgen-generated README (the published npm docs for @itk-wasm/downsample) was not regenerated: it documents the five pre-existing functions but has no resampleToReference/resampleToReferenceNode section, even though all other TS artifacts were regenerated. The new API ships invisible on npm, and the next full bindgen run will produce a surprise README diff.
7. packages/downsample/resample-to-reference.cxx:228 — The VectorImage functor re-declares the scalar functor's entire ~35-line option surface verbatim, including the duplicated CLI::IsMember list and multi-sentence help strings. Since dispatch picks a functor by input pixel type at runtime, drift between the copies would fork the CLI surface between scalar and vector inputs silently. This exceeds the package convention (downsample.cxx duplicates ~11 trivial lines with no enum constraint); hoist the declarations into a templated helper alongside MakeResampleFilter.
8. packages/downsample/CMakeLists.txt:80 — The resample-to-reference smoke ctest is argument-for-argument identical to resample-to-reference-linear (same executable, same inputs, same explicit --interpolator linear; only the output basename differs) — pure duplicate CI work. Additionally, none of the six per-interpolator tests compares output against anything (exit-code-0 only), so a broken interpolator that still writes an image passes the whole matrix. Drop the duplicate and consider a foreach() with a baseline comparison.

[thewtex]

I thought this would be a new /resample package, but it looks like you're putting it under downsample. Is that intentional?

@sedghi yes, that was the original intention. However, at least for now, we have the system downsample python package on conda-forge, and I am vendoring these with downsample.

[sedghi]

Sounds good to me let's do it

[thewtex]

Thanks for the thorough review (via Fable). I went through all eight findings — disposition below. Fixes are in 158d8158a; Node (4/4) and Python wasi (16/16) suites pass on it.

Addressed in 158d8158a

#2 — itkwasm version floor. Correct. The bindings import TransformList (needs itkwasm ≥ 1.0b180) but all three Python packages pinned ≥ 1.0.b145. Bumped to ≥ 1.0.b185, matching @itk-wasm/transform-io (the sibling that provides TransformList); pixi.lock re-synced.

#3 — substring interpolator validation. Confirmed, fixed at the root. The wasi generator (bindgen/python/wasi/wasi-function-module.js) split the CLI11 choice list on ', ' (comma-space) while CLI11 serializes it without spaces, so it produced interpolator not in ('linear,nearest_neighbor,…') — a single string, hence substring matching. The TS generator already splits on ','; I made wasi match and regenerated. Verified at runtime that 'sinc' / 'near' / 'gauss' now raise ValueError instead of failing opaquely inside the wasm.

#5 — silent linear fallback. SelectInterpolator's bare else now matches "linear" explicitly; an unknown name throws itkGenericExceptionMacro rather than silently substituting linear, so future drift between the CLI list and the helper fails loudly. (I kept the two CLI::IsMember lists as the single enforced gate — collapsing all three lists into one name→factory table is a good follow-up but a larger change.)

#4 (docs) — reference-image help text. Reworded the over-promise "…so an empty pixel buffer is acceptable" → "Only the geometry (origin, spacing, direction, size) is used; the pixel values are ignored," which is accurate (the reference still has to deserialize as the moving image's type). Propagated into all TS/Python docstrings + the demo.

#6 — README. The bindgen README is create-once, so it never gained the new sections. Added resampleToReference / resampleToReferenceNode, spliced from a fresh bindgen so the hand-maintained Live Demo / Documentation links are preserved (a naïve full regen drops them — worth knowing for the generator).

#8 — duplicate smoke ctest. Dropped; it was argument-for-argument identical to resample-to-reference-linear.

Deferred / for discussion

#1 — generic transform vs AffineTransform. Real, and the most substantive. Switching to itk::CompositeTransform<double,N> isn't a drop-in here: the in-memory path the TS/Python bindings use (--memory-io) currently mis-handles a TransformList (marshaled as a JSON array but parsed as a single object → throws), and itk::wasm::InputTransform<T> needs a concrete, New()-able T. The affine path is what's tested (affine .h5 from @itk-wasm/transform-io) and round-trips correctly. The BSpline heap-overwrite / composite-crash / multi-entry-drops-last cases you flagged are exactly why I'd rather broaden this deliberately (generic/chained support plus an explicit reject for non-affine parameterizations in the interim), with tests, in a dedicated follow-up rather than fold it into this PR. @thewtex for a call on scope.

#7 — vector functor duplication. Agreed; it mirrors the existing downsample.cxx pattern in this package. Hoisting the shared option surface into a templated helper next to MakeResampleFilter is a clean no-behavior-change follow-up.

#4 (fixture/type). Accepting a genuinely metadata-only / cross-type reference needs an ImageBase-style input the memory-IO reader doesn't support yet — deferred with #1. The unused cthead1-resample-reference-metadata-only.json lives in the pinned test-data tarball (not tracked in git), so it's harmless for now; it'll be wired up or dropped when the reference-type work lands.

[thewtex]

Update on finding #1 (transform bound to AffineTransform, unchecked parameterization) — now addressed in 5278a76e4, going further than the interim reject I'd proposed.

Generic single-transform support. Since itk::ResampleImageFilter::SetTransform() takes the abstract itk::Transform<double, N, N> base, the resample never needed a concrete affine. The new resampleReadInputTransform.h reconstructs any single transform parameterization (translation, rigid, affine, B-spline, …) into that base via the ITK object factory — sizing the parameter buffer to the transform's own parameter count. So the dangerous cases you flagged are fixed at the source:

• B-spline → reconstructed as a real BSplineTransform (correct parameter count) — no more heap overrun; it just works.
• Euler/rigid/translation → applied as the real transform, not silently coerced into an affine's first slots.
• Composite / multi-entry → explicitly rejected (chained transforms are the remaining follow-up), rather than the old null-dynamic_cast crash / silent last-entry-wins.

Binding surface unchanged. --transform is now a std::string (the memory-store index / path) with ->type_name("INPUT_TRANSFORM") kept, so bindgen emits the identical transform?: TransformList API — regenerating produced a zero-line binding diff.

Tests. Node + Python assert a Translation transform resamples identically to the equivalent identity-matrix affine (the old reader produced a shear here); Python asserts a multi-transform list is rejected rather than silently reduced to its last entry. Node 5/5, Python wasi 18/18, C++ CTests 11/11.

One caveat worth flagging for the framework: in the shipping Release builds, ITK_WASM_CATCH_EXCEPTION doesn't actually catch (exception catching is only enabled in the emscripten-Debug link flags), so the rejection — like every itk-wasm pipeline error in Release — surfaces as a hard pipeline abort (a wasm trap under WASI / a process abort under Node) rather than the descriptive message. The message is retained for Debug builds. Happy to revisit if you'd prefer the rejection to short-circuit in the generated binding layer instead.

Full chained/composite (CompositeTransform) support remains the deferred piece — still glad to scope that separately.

[thewtex]

Finding #1 is now fully resolved — 4e0d2b07e adds the last deferred piece: chained and composite transform support.

What changed. resampleReadInputTransform.h now composes multi-entry and composite TransformLists instead of rejecting them:

• In-memory (TS/Python bindings): itkwasm serializes an itk::CompositeTransform as a Composite marker entry followed by its components — the marker carries no parameters. The reader skips markers, reconstructs each component via the ITK object factory (as before for single transforms), and composes several into an itk::CompositeTransform; a bare multi-entry list composes the same way.
• Filesystem: a composite .h5 is returned by ITK's reader as the fully-populated CompositeTransform at the list front (components repeated behind it) — used directly. A multi-transform file without a composite wrapper composes in list order.
• Semantics (verified against ITK source, itkCompositeTransform.hxx): the queue [T0, T1, …, TN-1] maps a point as T0(T1(…TN-1(x))) — the last list entry is applied to the point first, i.e. [A, B] == A ∘ B — matching exactly what itkWasmTransformToTransformFilter produces for the same composite JSON. The --transform help text and all regenerated docstrings document this.
• A component-free composite (marker with no components) is rejected.

Tests (Node 7/7, Python wasi 21/21, C++ CTests 11/11):

• chain [scale2, translate(10,6)] ≡ single affine 2x+(20,12) — all values exactly representable, so the comparison is exact (Node + Python);
• the wrong-order single affine 2x+(10,6) does not match, locking the composition order (Python);
• a composite-marker list composes identically to the bare chain (Node + Python);
• transformwrite → .h5 → transformread → resample round-trip composes identically — validating the marker convention against transform-io's real serialization, not just my reading of it (Python);
• a component-free composite raises (Python).

With this, all four hazard cases from the original finding are closed: B-spline (generic reconstruction, 5278a76e4), non-affine mis-parameterization (5278a76e4), composite crash (now composes), and multi-entry last-wins (now composes).

[thewtex]

Finding #7 addressed in e45894709.

resample-to-reference.cxx: the scalar and vector PipelineFunctor templates each re-declared the full ~35-line option surface (input, reference-image, transform, interpolator + its six-name CLI::IsMember list, output) verbatim. Hoisted it into a ResampleOptions<TImage> struct + addResampleOptions() helper next to MakeResampleFilter, exactly as the finding suggested — both functors now declare options once and read them via options.. The struct owns the option targets on the functor's stack, so CLI11's by-reference binding stays valid through ITK_WASM_PARSE.

downsample.cxx: brought into the same convention (the finding noted resample mirrors this file's pattern, so I didn't want to leave it as the lone duplicative outlier). Its smaller shared surface — shrink-factors, crop-radius, downsampled — is now a DownsampleOptions<TImage> + addDownsampleOptions() used by both DownsampleScalarImage() and the vector functor; the input option and all per-path processing are untouched. Since that surface was only ~11 lines the file is a touch longer, but the option surface is single-source now. Happy to drop the downsample.cxx part if you'd rather keep this PR scoped strictly to resample — it's an independent commit-hunk.

No behavior change, verified: the option names, order, type_names, help text, and interpolator constraint are byte-identical, so regenerating the TS + Python bindings, the README, and the demo produced a zero-line diff. Full rebuild + regen done. Tests green: C++ CTests 11/11, Node 12/12 (including the downsample and resample vector paths), Python wasi 21/21.

[vboussot]

@thewtex I went through the whole pipeline, really nice work.

It's clean and well structured: resampling onto a reference image's grid, with an
optional transform and a selectable interpolator, and it handles both scalar and
vector images (extract → resample → recompose). Sharing MakeResampleFilter and the
option surface between the two paths keeps everything consistent, and reading the
transform generically into itk::Transform (any parameterization plus
composites/chains) at double precision is the right call.

My only real point was that you always need a reference image for the grid, but you
said the direct-grid version is coming in a follow-up, so that's fine by me.

Background value: everything outside the moving image takes the filter's default of
0, with no way to change it (MakeResampleFilter never calls SetDefaultPixelValue,
which ITK exposes with ResampleImageFilter::SetDefaultPixelValue). In CT for example
you'd want -1024 (air) rather than 0. This PR or a follow-up.

Nit: addressPrefixLength = 35 in resampleReadInputTransform.h hardcodes the address
prefix and the store index 0, a bit fragile, but fine given the current itk-wasm
conventions.

One last thing, totally out of scope: in medical image synthesis, where images are
aligned and resampled onto a common grid before being compared, resampling really
isn't implementation invariant, and almost nobody accounts for it. MAE is the
headline metric for evaluating synthesis, and I benchmarked torch grid_sample, ITK
and transformix with the same interpolator: the resampler alone already shifts things
by more than 5 MAE. So it bites exactly where you'd never think to look: train a
synthesis model on images aligned and resampled one way, then evaluate with another,
and you get a silent distribution shift that inflates or wrecks the MAE without
reflecting the real synthesis quality.

[sedghi]

Background value: everything outside the moving image takes the filter's default of
0, with no way to change it (MakeResampleFilter never calls SetDefaultPixelValue,
which ITK exposes with ResampleImageFilter::SetDefaultPixelValue). In CT for example
you'd want -1024 (air) rather than 0. This PR or a follow-up.

oh great find yes this will be an issue on CT

[thewtex]

Background value — addressed in 0cac68f58 (in this PR).

MakeResampleFilter now calls ResampleImageFilter::SetDefaultPixelValue, exposed as a --default-value option (defaultValue in TS, default_value in Python), defaulting to 0 so existing behavior is unchanged. Your CT example works: resampleToReference(moving, reference, { defaultValue: -1024 }).

Design notes:

• The option is a double at the CLI, static_cast to the image's PixelType at the filter. That keeps the generated binding uniform (number / float) across every dispatched pixel type and the VectorImage per-component path, rather than varying the binding type per pixel type — the cast to the actual pixel type happens where ITK does it.
• It lives in the shared ResampleOptions struct / addResampleOptions() helper (from the finding Ensure that Docker is available and working when running npm run build #7 refactor), so the scalar and vector functors both pick it up from one place.
• The generated bindings use the falsy-guard if (options.defaultValue), so an explicit 0 is dropped — but that coincides with the C++ default, so every value (including 0 and negatives like -1024) resamples correctly. I kept scope here and didn't touch the shared generator.

Tests (C++ CTests 12/12, Node 8/8, Python wasi 22/22):

• Node + Python: a large translation maps the entire output grid outside the moving image, and with nearest-neighbor every output pixel equals the background — asserted 200 when set and 0 when unset.
• New C++ CTest resample-to-reference-default-value exercises the --default-value flag on the filesystem/CLI path.
• README: added the defaultValue row to both resample option tables.