fix(cast_value): reject fill values that do not survive a round trip

[d-v-b]

🤖 AI text below 🤖

The cast_value codec now rejects a fill value that does not survive a round trip through the codec, as the cast_value spec requires. Previously out_of_range="clamp" or "wrap", or rounding, could encode the fill value to something that decodes to a different value, so the unwritten part of a partly written chunk read back as that value instead of the fill value. For example, a float32 array with fill value 1000.0 and an int8 target with clamping read 127.0 there. Array creation now raises a ValueError that names the fill value, both data types, and the value it decodes to.

Spec

From the data_type section of the cast_value spec:

The fill value of the output array MUST be cast to the target data type using the same casting semantics as elements. … Implementations SHOULD validate at metadata construction time that the fill value can be successfully cast in both the forward (encode) and inverse (decode) directions using the codec's configuration. If the fill value cannot survive a round-trip cast, implementations MUST treat this as an error.

CastValue.resolve_metadata cast the fill value forward only, with the encode scalar_map, and never checked the inverse.

Reproduction (before this change, cast-value-rs 0.4.2)

import zarr
from zarr.storage import MemoryStore
a = zarr.create_array(MemoryStore(), shape=(4,), chunks=(2,), dtype="float32", fill_value=1000.0,
                      filters=[{"name": "cast_value", "configuration": {"data_type": "int8", "out_of_range": "clamp"}}],
                      compressors=None)
a[0] = 5.0
a[:]  # array([   5.,  127., 1000., 1000.], dtype=float32)

Element 1 sits in a partly written chunk, so it is stored as the encoded fill (127) and decodes to 127.0. Element 2 and 3 are in a chunk that was never written, so they read the fill directly. After this change, create_array raises:

ValueError: The fill value 1000.0 does not survive a round trip through the cast_value codec: it encodes from float32 to 127 in int8, which decodes back to 127.0 in float32. ...

Fix

The check lives in CastValue.resolve_metadata. validate does not receive the fill value, and ArrayV3Metadata.__init__ already calls evolve_from_array_spec, validate, then resolve_metadata for each codec, so resolve_metadata runs at metadata construction.

• The encoded fill is cast back to the source dtype with the decode scalar_map and the same rounding and out_of_range.
• A new helper _same_scalar compares the result with the original fill. NaN matches NaN, whatever the payload. Otherwise the bytes must match, so the sign of zero must be preserved, as the spec requires. This rejects -0.0 with an integer target, since it comes back as +0.0.
• If the forward or inverse cast itself fails, the backend error is wrapped in a ValueError that says it is the fill value and names both dtypes.

resolve_metadata is also called when chunks are read and written, so each call now does two one-element casts instead of one.

Tests

In tests/test_codecs/test_cast_value.py:

Test	Case	Result
test_fill_value_roundtrip_accepted[nan-through-scalar-map]	float32 NaN → int8, encode NaN→0, decode 0→NaN	accepted, stored 0
…[clamp-in-range]	float64 5.0 → int8, clamp	accepted, stored 5
…[wrap-in-range]	float64 -3.0 → int8, wrap	accepted, stored -3
…[int-to-float]	int16 300 → float32	accepted, stored 300.0
…[integral-float-to-int]	float64 2.0 → int8, towards-zero	accepted, stored 2
…[exact-float-narrowing]	float64 1.5 → float32	accepted, stored 1.5
…[signed-zero-float-to-float]	float32 -0.0 → float16	accepted, stored -0.0
test_fill_value_partial_chunk_without_check_reads_wrong_value	float32 1000.0 → int8, clamp	with the check patched out, the unwritten element reads 127.0; with it, creation raises
test_fill_value_clamp_out_of_range_rejected	float32 1000.0 → int8, clamp	raises, comes back 127.0
test_fill_value_wrap_out_of_range_rejected	int16 300 → int8, wrap	raises, comes back 44
test_fill_value_lost_to_rounding_rejected	float64 1.5 → int8, nearest-even	raises, comes back 2.0
test_fill_value_signed_zero_lost_rejected	float32 -0.0 → int8	raises, comes back 0.0
test_fill_value_nan_to_int_without_scalar_map_rejected	float32 NaN → int8, no map	raises, forward cast fails
test_fill_value_cannot_decode_rejected	int32 0 → float64, encode 0→NaN, no decode map	raises, inverse cast fails

The accepted cases write one element of a two-element chunk, then check that the stored chunk holds the encoded fill and that the unwritten element reads back with the same bytes as the fill. All seven error tests and the regression test fail without the change to cast_value.py.

Results with hatch run test.py3.13-optional:pytest (cast-value-rs 0.4.2):

• tests/ (full suite, -n auto): 12663 passed, 1275 skipped, 4 xfailed
• tests/test_codecs/ plus every other test file that mentions cast_value: 2447 passed, 6 skipped, 2 xfailed
• test.py3.13-minimal (no cast-value-rs), test_cast_value.py and test_codec_pipeline.py: 42 passed, 33 skipped
• prek (ruff check, ruff format, mypy, codespell, numpydoc, towncrier-check, and the rest) passes on the changed files

Bookkeeping

• Based on main at bd0dc84.
• No related issue or PR was found (searched cast_value fill in issues and PRs).

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.72%. Comparing base (bd0dc84) to head (e58c19b).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4510   +/-   ##
=======================================
  Coverage   94.71%   94.72%           
=======================================
  Files          94       94           
  Lines       13663    13681   +18     
=======================================
+ Hits        12941    12959   +18     
  Misses        722      722           

Files with missing lines	Coverage Δ
src/zarr/codecs/cast_value.py	98.86% <100.00%> (+0.12%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.