Skip to content

feat: allow writing into open HDF5 file - #2637

Closed
flying-sheep wants to merge 7 commits into
ig/memory_store_for_testsfrom
mem4h5
Closed

feat: allow writing into open HDF5 file#2637
flying-sheep wants to merge 7 commits into
ig/memory_store_for_testsfrom
mem4h5

Conversation

@flying-sheep

@flying-sheep flying-sheep commented Sep 7, 2026

Copy link
Copy Markdown
Member

I decided against allowing to pass an open (Python) file, since

  1. what knobs would we use to wrap it in h5py.File? As seen in this PR, to open a BytesIO in memory, two parameters need to be passed to h5py.File
  2. that would make tests like the concat dask test silently pass, as BytesIO is pickleable, so it’d just get transferred over the wire, while h5py objects aren’t pickleable so they crash Dask as expected.

TODO:

  • test if this actually speeds up tests like @ilan-gold’s zarr variant of this

    /edit: no difference, so let’s forget about it.


  • Closes #
  • Tests added
  • Release note not necessary because:

@codecov

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.15686% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.85%. Comparing base (425e98e) to head (7f1090b).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/anndata/_core/anndata.py 77.77% 2 Missing ⚠️
src/anndata/_io/h5ad.py 86.66% 2 Missing ⚠️
Additional details and impacted files
@@                      Coverage Diff                      @@
##           ig/memory_store_for_tests    #2637      +/-   ##
=============================================================
+ Coverage                      85.83%   85.85%   +0.02%     
=============================================================
  Files                             49       49              
  Lines                           8133     8168      +35     
=============================================================
+ Hits                            6981     7013      +32     
- Misses                          1152     1155       +3     
Files with missing lines Coverage Δ
src/anndata/_core/merge.py 86.23% <100.00%> (ø)
src/anndata/_io/specs/lazy_methods.py 96.62% <100.00%> (+0.09%) ⬆️
src/anndata/experimental/backed/_lazy_arrays.py 92.37% <100.00%> (ø)
src/anndata/tests/helpers.py 84.21% <100.00%> (+0.43%) ⬆️
src/anndata/_core/anndata.py 86.21% <77.77%> (-0.19%) ⬇️
src/anndata/_io/h5ad.py 92.13% <86.66%> (-0.56%) ⬇️

... and 1 file with indirect coverage changes

@flying-sheep flying-sheep added this to the 0.14.0 milestone Sep 7, 2026
@flying-sheep
flying-sheep marked this pull request as ready for review September 7, 2026 10:55
@flying-sheep flying-sheep changed the title feat: allow writing into open h5ad file feat: allow writing into open HDF5 file Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant