Skip to content

Add datatype-aware numeric conversion reads #11

Description

@janosh

Problem

H5Dataset::read_raw::<T> intentionally exposes the stored representation and accepts a target based on element width. A consumer that wants semantic numeric values in one Rust type must instead inspect DatatypeMessage, dispatch across every supported integer or floating-point storage type, call read_raw with that exact type, and convert each value.

For example, reading arbitrary floating-point datasets as f64 requires separate f32 and f64 branches. Reading arbitrary integer datasets as one wide checked type requires branches for signedness and widths from 8 through 64 bits. Generic metadata and scientific-data readers duplicate this dispatch.

Proposed API

Please consider a separate conversion API that leaves read_raw unchanged, for example:

let floats = dataset.read_numeric_as::<f64>()?;
let integers = dataset.read_numeric_as::<i128>()?;

A corresponding hyperslab form would also be useful:

let values = dataset.read_numeric_slice_as::<f64>(&starts, &counts)?;

Requested semantics

  • Inspect the stored datatype class, signedness, byte order, and width rather than only comparing element sizes.
  • Support the standard primitive integer and floating-point datatypes.
  • Perform checked integer conversions and report overflow instead of wrapping.
  • Define float-to-integer and narrowing-float behavior explicitly, or reject those conversions initially.
  • Reuse the existing decoded data path so contiguous, chunked, and filtered datasets behave consistently.
  • Keep read_raw as the zero-conversion representation-level API.

Even an initial pair of focused helpers such as read_f64 and read_i128 would remove substantial downstream datatype dispatch while keeping conversion policy explicit.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions