Add Array.prepend and the regular_bounded chunk grid

[rhoadesScholar]

Prototype for zarr-developers/zarr-extensions#79, using the regular_bounded chunk grid proposed in zarr-developers/zarr-extensions#81.

Array.prepend(data, axis) adds data before index 0 of an axis, the way append adds it after the end. It grows by any number of elements and never touches an existing chunk:

z = zarr.create_array(store, shape=(6, 4), chunks=(4, 4), dtype="int32")
z[:] = 1
z.prepend(np.full((3, 4), 2, dtype="int32"))
z.shape   # (9, 4)
z.domain  # ((-3, 6), (0, 4))
# keys: c/-1/0 is new; c/0/0 and c/1/0 are unchanged, byte for byte

The first prepend switches the chunk grid from regular to regular_bounded, a regular grid with an origin and a lower_bound, so readers without that grid refuse the array rather than misread it. Indexing stays zero-based: index i is coordinate lower_bound + i, and Array.domain reports the coordinate range.

To try it: pip install "zarr @ git+https://github.com/zarr-developers/zarr-python@refs/pull/4516/head"

What growing in front costs. Two chunk rows in front of an array of 64³ uint8 chunks, one machine (NVMe), wall seconds:

chunks	rename every chunk file	store get/set/delete	copy to a new array	prepend's metadata step
512	0.03	0.76	0.32	0.001
4 096	0.13	5.9	3.1	0.002
32 768 (8 GiB)	1.0	44	26	0.002

Script: https://github.com/rhoadesScholar/zarr-chunk-offset/tree/main/bench

How it works

• RegularBoundedChunkGridMetadata subclasses RegularChunkGridMetadata, so code that checks for a regular grid keeps working.
• A regular dimension gains a shift, the position of the first element inside the first chunk. A new buffer_offset(chunk) gives where a chunk's data starts inside its buffer, which is 0 except for a partly filled first chunk. The indexers add it at the six places they convert array positions to chunk-local ones. A partly filled first chunk is then handled like a partly filled last chunk, write path included.
• Chunk keys add the grid index of the first chunk, so they can be negative. Decoding rejects -0.

Tests. 22 new cases. Every selection type is read and written against NumPy on an array whose first chunk is partly filled, and removing the buffer offset at any one of the six indexer sites fails a test. Also covered: prepending by part of a chunk, whole chunks and part of a shard; repeated prepends followed by resize; reopening; and the grid's validation. The full suite passes (12,618 tests), and mypy and ruff are clean.

Not covered here: rectilinear grids, Zarr v2 arrays, and switching an array to regular_bounded without prepending.

Refs zarr-developers/zarr-specs#9, zarr-developers/zarr-extensions#30.

[codecov]

Codecov Report

❌ Patch coverage is 95.83333% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.72%. Comparing base (bd0dc84) to head (fb48e95).

Files with missing lines	Patch %	Lines
src/zarr/core/chunk_grids.py	88.88%	2 Missing ⚠️
src/zarr/core/metadata/v3.py	95.45%	2 Missing ⚠️
src/zarr/core/array.py	97.77%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4516      +/-   ##
==========================================
+ Coverage   94.71%   94.72%   +0.01%     
==========================================
  Files          94       94              
  Lines       13663    13767     +104     
==========================================
+ Hits        12941    13041     +100     
- Misses        722      726       +4     

Files with missing lines	Coverage Δ
src/zarr/core/chunk_key_encodings.py	98.64% <100.00%> (+0.07%)	⬆️
src/zarr/core/indexing.py	96.58% <100.00%> (+<0.01%)	⬆️
src/zarr/core/array.py	98.41% <97.77%> (-0.03%)	⬇️
src/zarr/core/chunk_grids.py	96.18% <88.88%> (-0.18%)	⬇️
src/zarr/core/metadata/v3.py	96.75% <95.45%> (-0.17%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[rhoadesScholar]

Ready for review; the registry entry is zarr-developers/zarr-extensions#80. cc @d-v-b

[d-v-b]

thanks for this PR! I really want to see this feature added to zarr! But I think we want to be sure that we support negative chunk indices consistently across the ecosystem, so I'll hold off on reviewing until we get alignment from the other implementers on the zarr-extensions PR.

[rhoadesScholar]

Converting to draft while the design is worked out in zarr-developers/zarr-extensions#79. This branch is a prototype and will be reworked to match.