WIP: Introduce plate class

[jo-mueller]

This is a first draft of an implementation of how writing plates could look like in a class-based API world, built around the NgffMultiscales class proposed in #515

[jo-mueller]

@will-moore if you're interested, this would be roughly my idea on how a class-based API could look like for writing HCS data (see classes/hcs.py)

In essence, one would have to create a dictionary of NgffMultiscales objects from #515 like this beforehand:

plate_dict = {
  ("A", 1): [ngff_rowA_col1_fov1, ngff_rowA_col1_fov2, ...], 
  ("A", 2): [ngff_rowA_col2_fov1, ngff_rowA_col2_fov2, ...],
  ("B", 1): [ngff_rowB_col1_fov1, ngff_rowB_col1_fov2, ...]
}

ngff_plate = NgffHCSPlate(plate_dict)
ngff_plate.to_ome_zarr(...)

this would then internally populate the omzp metadata, and dump everything to disk.

[will-moore]

So, you have to have the whole plate in memory / dask-arrays before you write?
That makes sense from an API point of view, but I'm not sure how practicable that is if you're writing a lot of data.

When we export Plates with omero-cli-zarr, we write each Image to disk at a time. https://github.com/ome/omero-cli-zarr/blob/9f844b877d8e19e42a69d7ba43f7dc086680fbec/src/omero_zarr/raw_pixels.py#L290
so it would be nice if that was an option.

In your plate_dict both row and column names should be strings. It might be nicer to simply use a key like "A/1" ?

[jo-mueller]

So, you have to have the whole plate in memory / dask-arrays before you write?
That makes sense from an API point of view, but I'm not sure how practicable that is if you're writing a lot of data.

Kind of. Under the hood, everything is coerced to a dask array, so the memory footprint should be low. Or at least not larger than whatever a user has in memory in the first place.

When we export Plates with omero-cli-zarr, we write each Image to disk at a time.

The to_ome_zarr function does effectively that. It iterates over all NgffMultiscales instances and if compute is True (which it defaults to) then every image is written at a time. You could set it to False and then write everything concurrently, but I'm not sure what one would gain from it 🤔

[imagesc-bot]

This pull request has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/how-to-convert-czi-zeiss-to-ome-zarr-in-the-context-of-high-content-screening/120809/6

[codecov]

Codecov Report

❌ Patch coverage is 21.11801% with 254 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.63%. Comparing base (94eaf20) to head (07cb0f6).

Files with missing lines	Patch %	Lines
ome_zarr/image.py	16.75%	154 Missing ⚠️
ome_zarr/scene.py	29.33%	53 Missing ⚠️
ome_zarr/classes/plate.py	21.66%	47 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (94eaf20) and HEAD (07cb0f6). Click for more details.

HEAD has 10 uploads less than BASE

Flag	BASE (94eaf20)	HEAD (07cb0f6)
	20	10

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #570      +/-   ##
==========================================
- Coverage   87.09%   79.63%   -7.46%     
==========================================
  Files          17       20       +3     
  Lines        2526     2848     +322     
==========================================
+ Hits         2200     2268      +68     
- Misses        326      580     +254     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jo-mueller]

@kevinyamauchi @will-moore I cleaned up here a bit, I think this is shaping a bit more into how I would envision things to work around reading/writing plates. I used it to create some sample data which is available here and checks out alright with the validator 🙂

Also: This PR probably needs a full rewrite in a separate PR because of the garbled up PR history, but since all changes are pretty much inside the classes/plate.py