Repository navigation

Add Pyodide support and CI jobs for Zarr - #1903

Open

agriyakhetarpal wants to merge 138 commits into

zarr-developers:mainfrom

agriyakhetarpal:emscripten

Open

Add Pyodide support and CI jobs for Zarr#1903
agriyakhetarpal wants to merge 138 commits into
zarr-developers:mainfrom
agriyakhetarpal:emscripten

Commits on May 23, 2024
- Add CI job to test out-of-tree Pyodide builds
  agriyakhetarpal
  committed
  9f5a110
- Add `[msgpack]` dependency for `numcodecs`
  agriyakhetarpal
  committed
  29282fc
Commits on May 27, 2024
- Bump to Pyodide 0.26.0, update comments
  agriyakhetarpal
  committed
  d465742
- Try to run tests without async
  agriyakhetarpal
  committed
  cdf0bb2
- Move shared file to rootdir, outside v2 and v3
  agriyakhetarpal
  committed
  dfe0321
- Move `fasteners` import inside ThreadSynchronizer
  agriyakhetarpal
  committed
  b100ec9
- Make the tests directory importable, fix `_shared`
  agriyakhetarpal
  committed
  b0dddca
- Import list of greetings from `numcodecs`
  agriyakhetarpal
  committed
  d227728
- Skip some tests that use threading
  agriyakhetarpal
  committed
  fdb2bef
- Skip some tests that use `fcntl`
  agriyakhetarpal
  committed
  621077a
- Skip tests that require `dbm`
  agriyakhetarpal
  committed
  7ae9a97
- Move `IS_WASM` logic to internal `zarr` API
  agriyakhetarpal
  committed
  22eb6da
- Skip a few tests trying to import `multiprocessing`
  agriyakhetarpal
  committed
  6836947
- Skip tests that use async and threading code
  agriyakhetarpal
  committed
  fe3bf27
- Improve `asyncio_tests_wrapper`, fix test imports
  agriyakhetarpal
  committed
  08997ec
- Skip entire `test_codecs.py` file
  agriyakhetarpal
  committed
  9bfc860
- Skip yet another test that requires threads
  agriyakhetarpal
  committed
  9bcb350
- xfail test where array's fill values are different
  agriyakhetarpal
  committed
  9985abb
- xfail test because Emscripten FS
  agriyakhetarpal
  committed
  7ea12ef
- Skip last test that tries to run threads
  agriyakhetarpal
  committed
  a6565de
- Another test that tries to run threads
  agriyakhetarpal
  committed
  85f621c
- xfail another array's differing `fill_values` test
  agriyakhetarpal
  committed
  1a64255
- Skip entire sync file under WASM, no threading
  agriyakhetarpal
  committed
  c8cb38b
Commits on May 28, 2024
- Restore pytest config options, remove when needed
  agriyakhetarpal
  committed
  eb36d40
Commits on May 26, 2025
- Merge main
  agriyakhetarpal
  committed
  e3bf365
- Bump Emscripten, Pyodide xbuildenv, Node.js versions
  agriyakhetarpal
  committed
  b3a5b8a
- Running on `ubuntu-latest` should be fine
  agriyakhetarpal
  committed
  42d2792
- Don't persist credentials with git clone
  agriyakhetarpal
  committed
  27068e2
- Don't pin the version of `pyodide-build`
  agriyakhetarpal
  committed
  aff9b18
- Use same xbuildenv for building and testing
  agriyakhetarpal
  committed
  bb2c136
- Temporarily build numcodecs for WASM as well
  agriyakhetarpal
  committed
  710195a
- Use Pyodide 0.28.0a2 for now
  agriyakhetarpal
  committed
  07e5bc9
Commits on May 27, 2025
- Use `fetch-depth: 0` to bring correct versions
  agriyakhetarpal
  committed
  403fbb0
- Skip `test_multiprocessing` for WASM
  agriyakhetarpal
  committed
  24dbc77
- Skip all sync tests
  agriyakhetarpal
  committed
  ecea615
- xfail `test_array_roundtrip` for now
  agriyakhetarpal
  committed
  d919bd7
- Skip `test_group_members_performance[memory]` for now
  agriyakhetarpal
  committed
  0bb7d47
- Set concurrency and max workers as 1
  agriyakhetarpal
  committed
  fb59eba
- Update `zarr.config` tests to match
  agriyakhetarpal
  committed
  4c6bed6
- Move WASM check to resolve circular import
  agriyakhetarpal
  committed
  6754131
- Mark Blosc `test_typesize` as a known failure case
  agriyakhetarpal
  committed
  f426ed7
- Skip `async.concurrency` config override test case
  agriyakhetarpal
  committed
  405d247
- Mark some indexing tests as flaky on WASM
  agriyakhetarpal
  committed
  e93073a
- Oops, fix a config test
  agriyakhetarpal
  committed
  d862953
- Fix another config test
  agriyakhetarpal
  committed
  cbd1d4d
- Hook into Pyodide WebLoop
  agriyakhetarpal
  committed
  3e8bdef
- Move `zarr.constants` to `zarr._constants`
  agriyakhetarpal
  committed
  4ee492e
- Bump to Pyodide 0.28.0a3
  agriyakhetarpal
  committed
  d94970e
- Fix typo
  agriyakhetarpal
  committed
  cd3424c
Commits on May 28, 2025
- `asyncio_mode = "auto"` works now, clean it up
  agriyakhetarpal
  committed
  5044a22
- Restore `test_group_members_performance`
  agriyakhetarpal
  committed
  e131867
- Disable SIMD when building numcodecs
  agriyakhetarpal
  committed
  c7c22dc
- Oops, disable AVX2, SSE2 at the right place
  agriyakhetarpal
  committed
  e62c933
- Debug improper numcodecs version
  agriyakhetarpal
  committed
  d3bcf56
- Debug numcodecs version again
  agriyakhetarpal
  committed
  e4b7379
- Fetch tags manually for now
  agriyakhetarpal
  committed
  35cecc1
- Force Zarr to install
  agriyakhetarpal
  committed
  ceb70c7
- Install `numcodecs` with `crc32c`
  agriyakhetarpal
  committed
  1ea992f
- Install the rest of the missing dependencies
  agriyakhetarpal
  committed
  f51ddd1
- Escape wheel filename correctly
  agriyakhetarpal
  committed
  0ec47b9
- Remove extra install line
  agriyakhetarpal
  committed
  e1617c0
- Fix misquoted end
  agriyakhetarpal
  committed
  23e34bf
Commits on May 29, 2025
- Remove lenience for performance test, skip it instead
  agriyakhetarpal
  committed
  22e6795
- Undo async.concurrency to 1, improve performance
  agriyakhetarpal
  committed
  da8bfc7
- Add `slow_wasm` marker, skip orthogonal indexing tests
  agriyakhetarpal
  committed
  81f5df3
- Move WebLoop patch to `conftest.py`
  agriyakhetarpal
  committed
  bad9920
- Mark more indexing tests as slow in WASM
  agriyakhetarpal
  committed
  2217455
- Fix condition for slow WASM tests
  agriyakhetarpal
  committed
  86f8785
- Clearer skip message for slow WASM tests
  agriyakhetarpal
  committed
  3230892
- Merge main
  agriyakhetarpal
  committed
  07b5645
- Add release note for Pyodide/WASM support.
  agriyakhetarpal
  committed
  ef70cbd
- Ignore WASM code paths Codecov doesn't know about
  agriyakhetarpal
  committed
  9212c0e
- Bring back numcodecs version check
  agriyakhetarpal
  committed
  86323eb
- Revert "Bring back numcodecs version check"
  agriyakhetarpal
  committed
  9c6af32
Commits on May 30, 2025
- Add guidance highlighting JSPI requirement
  agriyakhetarpal
  committed
  090c62b
- Merge branch 'main' into emscripten
  agriyakhetarpal
  authored
  3f2d41d
Commits on Jun 3, 2025
- Drop `shutdown_asyncgens` fixture
  agriyakhetarpal
  committed
  b89f682
- Clarify docs for JSPI availability and usage
  agriyakhetarpal
  committed
  700aae4
- Merge branch 'main' into emscripten
  agriyakhetarpal
  authored
  93680d2
- Fix lint error
  agriyakhetarpal
  committed
  401311e
- Update release note with JSPI info
  agriyakhetarpal
  committed
  dd25a36
Commits on May 16, 2026
- Merge main
  agriyakhetarpal
  committed
  f888142
- Modernise emscripten workflow
  agriyakhetarpal
  committed
  9161ea9
- Ignore some test dependencies on emscripten
  agriyakhetarpal
  committed
  3533283
- Add hatch scaffolding for easier Pyodide testing
  agriyakhetarpal
  committed
  4f31bc2
- Ignore sharded Morton indexing benchmark
  agriyakhetarpal
  committed
  f9e3b92
- Ignore `test_v2_dtype_regression` for Emscripten
  agriyakhetarpal
  committed
  ddbaae4
- numcodecs version check can be brought back
  agriyakhetarpal
  committed
  a1bbcf8
Commits on May 17, 2026
- Use `os.rename` for Emscripten
  agriyakhetarpal
  committed
  6874bab
- Blosc size mismatch no longer occurs!
  agriyakhetarpal
  committed
  531585f
- Skip `test_scripts_can_run`, uses subprocesses
  agriyakhetarpal
  committed
  5038b0c
- Allow running a portion of tests if specified
  agriyakhetarpal
  committed
  ed8eaf0
- Add monkeypatch for pytest-asyncio to use WebLoop
  agriyakhetarpal
  committed
  9b3e15e
- Patch `asyncio.Runner.__enter__` also
  agriyakhetarpal
  committed
  524e962
- Fix Emscripten os.rename silently overwriting files + corrupting group attrs
  agriyakhetarpal
  committed
  5344cee
- Stop-gap `test_typesize`'s size mismatch (for now)
  agriyakhetarpal
  committed
  0b2a565
- Skip hypothesis problems because IDK what to do
  agriyakhetarpal
  committed
  548f7cf
- Remove some lint
  agriyakhetarpal
  committed
  7e4f3b1
- Better typing for asyncio monkeypatch
  agriyakhetarpal
  committed
  19f79bf
- Update CHANGELOG file format, JSPI default enabled
  agriyakhetarpal
  committed
  1f4db1b
- Make Pyodide hatch env run on Python 3.14
  agriyakhetarpal
  committed
  cff0f08
- Merge branch 'main' into emscripten
  agriyakhetarpal
  authored
  978870f
- Add some tests for `_safe_move`
  agriyakhetarpal
  committed
  786e593
- Reduce verbosity in GHA test logs, too hard to scroll
  agriyakhetarpal
  committed
  cd3a59b
- Fix lint
  agriyakhetarpal
  committed
  e20043e
- Split safe move method across platforms
  agriyakhetarpal
  committed
  2796de5
- Remove outdated pyodide webloop warning ignore
  agriyakhetarpal
  committed
  3b31588
- Properly ignore benchmark tests when using posargs
  agriyakhetarpal
  committed
  716dbcb
- Experiment if we can remove the `slow_wasm` option now
  agriyakhetarpal
  committed
  3b64a34
- Update JSPI support message to align with 2026
  agriyakhetarpal
  committed
  3b86ee5
Commits on May 29, 2026
- Merge main
  agriyakhetarpal
  committed
  6fde927
- Use `--clear` when creating Pyodide venvs
  agriyakhetarpal
  committed
  33c7e5b
Commits on Jun 13, 2026
- Merge main
  agriyakhetarpal
  committed
  440995e
- Bump to stable Pyodide v314.0.0
  agriyakhetarpal
  committed
  4a6017b
- Temporarily pin pytest<9.1.0
  agriyakhetarpal
  committed
  1183f2a
Commits on Jun 18, 2026
- pytest v9.1.1 is out, update deps and lockfile
  agriyakhetarpal
  committed
  ae436c7
- Fix conflicts, undo pytest deps fix
  agriyakhetarpal
  committed
  d866c00
- Add emscripten platform markers back
  agriyakhetarpal
  committed
  beaf970
- Merge branch 'main' into emscripten
  agriyakhetarpal
  authored
  db06c2f
- Exercise CI to rerun uv failure
  agriyakhetarpal
  committed
  3260af8
- Add a space before uv dependency semicolon
  agriyakhetarpal
  committed
  debe5d3
Commits on Jun 26, 2026
- Merge main and resolve conflicts
  agriyakhetarpal
  committed
  3089427
Commits on Jul 12, 2026
- Merge main and fix conflicts
  agriyakhetarpal
  committed
  bd6f579
Commits on Jul 31, 2026
- Merge main and fix conflicts
  agriyakhetarpal
  committed
  7a8b206
- Bump to Pyodide 314.0.3
  agriyakhetarpal
  committed
  07e3d63
- Disable faulthandler watchdog on Pyodide tests
  agriyakhetarpal
  committed
  4644be6
- Skip new tests that can't run on WASM
  agriyakhetarpal
  committed
  913ccb6
- Make `FusedCodecPipeline` thread pool WASM safe
  agriyakhetarpal
  committed
  4841bb9
- Mark config-level `codec_pipeline.max_workers` as 1
  agriyakhetarpal
  committed
  eb5ab18
- Skip `test_sync_api_compute_off_event_loop` on WASM
  agriyakhetarpal
  committed
  4919ed6
- Fix `test_config_defaults_set`
  agriyakhetarpal
  committed
  f139c0e
- Enforce WASM `RuntimeError` at the pipeline level
  agriyakhetarpal
  committed
  e317504
- Add a few more tests cases
  agriyakhetarpal
  committed
  5fb09a9
- Block Zarr's thread pool executor on WASM
  agriyakhetarpal
  committed
  041af73
- Undo threading max workers at config level
  agriyakhetarpal
  committed
  f9f5f38
- Remove some unreachable WASM cases
  agriyakhetarpal
  committed
  aa064ca
- Make the changelog message more accurate
  agriyakhetarpal
  committed
  aa4f5a3
Commits on Aug 15, 2026
- Merge main and fix conflicts
  agriyakhetarpal
  committed
  e189fa8

Commits

Commits on May 23, 2024

Commits on May 27, 2024

Commits on May 28, 2024

Commits on May 26, 2025

Commits on May 27, 2025

Commits on May 28, 2025

Commits on May 29, 2025

Commits on May 30, 2025

Commits on Jun 3, 2025

Commits on May 16, 2026

Commits on May 17, 2026

Commits on May 29, 2026

Commits on Jun 13, 2026

Commits on Jun 18, 2026

Commits on Jun 26, 2026

Commits on Jul 12, 2026

Commits on Jul 31, 2026