Skip to content

dask indexing error when stacking series dask arrays #66

Description

@jmuhlich

Passing some dask arrays returned by to_dask to dask.array.stack and then calling compute on the result or any slice of it raises an Exception. See reproducer and stack trace below.

I ran into this when testing @tlambert03's bioio_bioformats use-bffile branch. The BioImage.get_dask_stack and get_xarray_dask_stack methods call da.stack internally which leads to the same problem.

import bffile
import dask.array as da

bf = bffile.BioFile('tests/data/s_3_t_1_c_3_z_5.ome.tiff')
bf.open()
istack = da.stack([s.to_dask() for s in bf])
print(istack.shape)
img = istack[0, 0, 0, 0].compute()
print(img.shape)
Traceback (most recent call last):
  File "/home/jmuhlich/development/bffile/.venv/lib/python3.11/site-packages/dask/array/chunk.py", line 430, in getitem
    result = obj[index]
             ~~~^^^^^^^
  File "/home/jmuhlich/development/bffile/src/bffile/_lazy_array.py", line 379, in __getitem__
    key = self._normalize_key(key)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmuhlich/development/bffile/src/bffile/_lazy_array.py", line 569, in _normalize_key
    raise IndexError(msg)
IndexError: too many indices for array: array is 5-dimensional, but 6 were indexed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jmuhlich/development/bffile/test1.py", line 8, in <module>
    img = istack[0, 0, 0, 0].compute()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmuhlich/development/bffile/.venv/lib/python3.11/site-packages/dask/base.py", line 377, in compute
    (result,) = compute(self, traverse=False, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmuhlich/development/bffile/.venv/lib/python3.11/site-packages/dask/base.py", line 685, in compute
    results = schedule(expr, keys, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmuhlich/development/bffile/.venv/lib/python3.11/site-packages/dask/array/chunk.py", line 432, in getitem
    raise ValueError(
ValueError: Array chunk size or shape is unknown. Possible solution with x.compute_chunk_sizes()

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions