Skip to content

Add free-threaded tests with pytest-run-parallel #70

Description

@kylebarron

Follow up from #69


Claude:

Follow-up: free-threaded (3.14t) CI testing

PR #69 adds the wheel builds for free-threaded CPython 3.14 (cp314-cp314t)
and scaffolds the abi3t (3.15+) wheel. It intentionally does not add CI that
tests zarrista on a free-threaded interpreter — this issue tracks that work,
which turned out to need a test-suite restructure rather than a CI one-liner.

Why it was deferred

Spiking the 3.14t test job surfaced three blockers:

  1. numcodecs re-enables the GIL. numcodecs.blosc hasn't declared that it
    can run without the GIL, so importing it on 3.14t silently turns the GIL back
    on:

    RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'numcodecs.blosc' ...

    Since nearly all tests pull in zarr-python (→ numcodecs), running the current
    suite on 3.14t gives a false free-threaded signal — the GIL is on.

  2. Missing free-threaded wheels → slow / degraded. icechunk and
    numcodecs compile from source on 3.14t (no cp314t wheels yet), and
    google-crc32c (via icechunk) falls back to its pure-Python implementation:

    As the c extension couldn't be imported, google-crc32c is using a pure python implementation that is significantly slower.

  3. Tests use zarr-python as the fixture writer. 8 of 10 test files call
    zarr.create_array / zarr.open_group to produce fixtures, then read them
    back with zarrista. These are really interop/integration tests. We have
    almost no coverage that exercises zarrista's own read+write round-trip
    (ArrayBuilder.create / store_chunk / store_metadataArray read).

Proposed plan

  • Split the suite into own tests (zarrista-only read/write) vs
    integration tests (zarr-python / icechunk / arro3 interop), via markers
    or a tests/integration/ directory.
  • Add zarrista-only round-trip tests (create + store chunks + read back,
    compared against numpy) so there's real own-code coverage.
  • Split dependency groups so the free-threaded job installs only GIL-safe
    deps (avoid zarr/numcodecs/icechunk).
  • Add a 3.14t CI job that runs only the own tests on the free-threaded
    interpreter.
  • Revisit pytest-run-parallel for genuine thread-safety stress — note it
    runs the same test in N threads sharing one fixture, so any test that
    mutates shared store state flakes; concurrency tests must use per-thread
    isolated stores.
  • Re-enable integration tests on 3.14t once numcodecs/icechunk ship
    free-threaded wheels and declare GIL-safety.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions