fix(python): handle non-row-major and zero-size input arrays

[d-v-b]

🤖 AI text below 🤖

Fixes the silent data corruption behind zarr-python#4237, plus a debug-build panic on zero-size arrays that property-based testing found along the way.

Bug 1: non-row-major input is misread or rejected

The bindings handed the input array's raw buffer to the conversion kernels via numpy::PyReadonlyArray::as_slice. That guard accepts column-major arrays as well as row-major ones:

fn is_contiguous(&self) -> bool {
    check_flags(&*self.as_array_ptr(),
        npyffi::NPY_ARRAY_C_CONTIGUOUS | npyffi::NPY_ARRAY_F_CONTIGUOUS)  // either flag
}

The kernels then wrote those elements into a freshly allocated row-major output of the same shape, silently transposing the data:

>>> base = np.arange(12, dtype=np.float32).reshape(3, 4)
>>> cast_array(base.T, target_dtype="uint16", rounding_mode="nearest-even",
...            out_of_range_mode="clamp")
array([[ 0,  1,  2],      # should be [[0, 4, 8],
       [ 3,  4,  5],      #            [1, 5, 9],
       [ 6,  7,  8],      #            [2, 6, 10],
       [ 9, 10, 11]],     #            [3, 7, 11]]
      dtype=uint16)

No exception, no warning. Arrays that were neither C- nor F-contiguous took the other branch and were rejected with Input array must be contiguous. Both cases occur in routine Zarr pipelines: the transpose codec hands the next codec a transposed view — column-major in 2-D, strided in higher dimensions. The ValueError reported in zarr-python#4237 was the lucky branch; 2-D arrays corrupted quietly.

Fix: the Python wrapper normalizes the input with np.asarray(arr, order="C") — a no-op for row-major arrays, a copy for anything else, and 0-d-preserving (unlike np.ascontiguousarray). The binding keeps a strict backstop for direct callers of the private extension module, rejecting non-row-major input instead of misreading it. cast_array_into gains a Python wrapper (it was previously re-exported raw) so both entry points normalize identically.

An earlier revision of this PR normalized on the Rust side with ndarray's as_standard_layout. The property tests falsified it: ndarray's raw-view stride assertions panic on layouts numpy considers legal (e.g. a zero-size slice with negative strides), so numpy's own normalization is the authoritative place to do this.

Bug 2: zero-size arrays panic in debug builds (pre-existing on main)

numpy gives every zero-size array strides of 0, and ndarray's debug-build stride assertions reject a 0-stride dim of size > 1 as self-overlapping. So this panics on any maturin develop build of main:

>>> cast_array(np.zeros((4, 0)), target_dtype="uint8", rounding_mode="nearest-even")
pyo3_runtime.PanicException: The strides must not allow any element to be referenced by two different indices

Release builds compile the assertion out, which is why published wheels seemed unaffected. Zero-size arrays now skip the conversion block entirely — there is nothing to convert, and the ndarray view over the output buffer is never constructed.

Also corrects the cast_array_into output error message, which said "contiguous" where it meant row-major.

Tests

Example-based: the memory-layout tests are parametrized over 8 layouts (row-major, column-major, 3-D transpose, strided, negative-stride, sliced view, 0-d, empty) × one dtype pair per conversion path (float→int, int→int, float→float, int→float) plus a float16 source, for both cast_array and cast_array_into — 80 cases. Dedicated cases pin the SIMD clamp fast path and scalar-map matching on non-contiguous views, and a backstop test pins the private module's strict rejection. test_non_contiguous_input is removed: it asserted the rejection that was itself the bug.

Property-based (new hypothesis test dependency): layout invariance — casting an arbitrarily-strided view must behave exactly like casting its C-contiguous copy (same values, same shape, or the same error), and cast_array_into must agree with cast_array — sampled over the full 11×11 dtype grid and random transpose/slice views. These properties falsified the earlier Rust-side revision of this fix and found bug 2, which the hand-picked empty-array case missed ((0, 3) happens to slip past the assertion; (4, 0) does not).

Kill-test results: with the source fix reverted, 56 of 152 tests fail — corruption caught by AssertionError, not just ValueError, across every conversion path. With only the wrapper fix applied against the old Rust code, exactly 4 fail, one per Rust-side change, so no part of the fix is untested.

Verification

• pytest — 152 passed; property stress run at 2000 fresh-seed examples clean
• cargo test -p zarr-cast-value — 38 passed; cargo fmt --check and cargo clippy clean
• End-to-end: a wheel from this branch installed into zarr-python at main with no zarr-side patch — all transpose/order/filter-order combinations round-trip correctly, 0-d arrays work, tests/test_codecs/ 774 passed

Downstream

zarr-python#4238 works around bug 1 with np.ascontiguousarray on the zarr side. Its regression test passes against unpatched zarr-python once this ships, so the workaround can be dropped — and should be either way, since np.ascontiguousarray promotes 0-d arrays to shape (1,) and breaks 0-d Zarr arrays. zarr-python will want a minimum cast-value-rs bump once this is released.

Sibling repo cast-value.py needs no change: its numpy implementation is stride-safe, and CastValueRustV1 forwards to cast_array, so it inherits this fix.

[codspeed-hq]

Merging this PR will not alter performance

✅ 51 untouched benchmarks
⏩ 21 skipped benchmarks¹

---

_{Comparing fix/non-row-major-input (8d161f9) with main (b27451d)}

Footnotes

1. 21 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩