v0.4.2

HuggingFaceDatasets v0.4.2

Diff since v0.4.1

Merged pull requests:

• Process-parallel DataLoader (num_workers) support for Dataset (#64) (@CarloLucibello)
• Drop DLPack dependency; reorganize examples as perf benchmarks (#67) (@CarloLucibello)

v0.4.1

HuggingFaceDatasets v0.4.1

Diff since v0.4.0

Merged pull requests:

• Add Features/ClassLabel schema views (#65) (@CarloLucibello)
• Use MLCore 1.1 extension for getobs(::Py), drop type piracy (#66) (@CarloLucibello)

v0.4.0

Breaking changes

This is a breaking release (0.3.x → 0.4.0). Datasets are now returned in the "julia" format by default, so observations come back as native Julia values instead of raw Python objects.

• "julia" is now the default format. load_dataset, DatasetDict, and the new Dataset(::AbstractDict/::NamedTuple) constructors return datasets in the "julia" format, so indexing an observation (ds[i]), a column (ds["label"]), and iteration all convert to native Julia types instead of raw Py objects. Wrapping a raw datasets.Dataset directly (Dataset(pyds)) stays format-neutral. Opt out with set_format!(ds, nothing) (or with_format(ds, nothing)).
• The "julia" format is now numpy-backed. Numeric array columns decode to real N-D Julia arrays and a range index stacks rows into a (dims…, N) tensor (observation axis last, the MLUtils convention). Consequently image columns now decode to raw numeric arrays rather than Gray/RGB colorviews — update downstream code expecting Colorant element types.
• Bumped the datasets Python dependency to >=4.0, <5 (previously <4). Environments pinned to datasets 3.x are no longer supported.
• reset_format! now restores the default "julia" format instead of stripping all formatting; use set_format!(ds, nothing) to strip to raw Python observations.
• py2jl on a datasets.Column (and hence string indexing of a julia-formatted Dataset) now returns a lazy Column view instead of eagerly materializing the whole column. It still behaves like a vector; call collect for a plain Vector.
• Invalid arguments now raise ArgumentError, and out-of-range indices raise BoundsError, instead of AssertionError — update any code catching AssertionError.

See the changelog for the full list of additions (Serialization support, streaming IterableDataset/IterableDatasetDict, Tables.jl construction, etc.).

v0.3.4

HuggingFaceDatasets v0.3.4

Diff since v0.3.3

Merged pull requests:

• relax version requirements for datasets python package (#30) (@CarloLucibello)

v0.3.3

HuggingFaceDatasets v0.3.3

Diff since v0.3.2

Merged pull requests:

• fix links (#22) (@CarloLucibello)
• misc updates (#23) (@CarloLucibello)

Closed issues:

• move to JuliaML org (#3)
• pin datasets to specific version (#19)
• check links in README and documentation (#21)

v0.3.2

HuggingFaceDatasets v0.3.2

Diff since v0.3.1

Closed issues:

• natively load datasets with Arrow.jl (#12)

Merged pull requests:

• CompatHelper: bump compat for ImageCore to 0.10, (keep existing compat) (#20) (@github-actions[bot])

v0.3.1

HuggingFaceDatasets v0.3.1

Diff since v0.3.0

Closed issues:

• Can't load Amazon Reviews in Julia (#16)
• Python: AttributeError: module 'PIL' has no attribute 'PngImagePlugin' (#17)

Merged pull requests:

• fix for PIL imports + fix DatasetDict julia format (#18) (@CarloLucibello)

v0.3.0

HuggingFaceDatasets v0.3.0

Diff since v0.2.1

Merged pull requests:

• remove numpy format when with_format("julia") (#15) (@CarloLucibello)

v0.2.1

HuggingFaceDatasets v0.2.1

Diff since v0.2.0

Merged pull requests:

• CompatHelper: bump compat for MLUtils to 0.4, (keep existing compat) (#13) (@github-actions[bot])

v0.2.0

HuggingFaceDatasets v0.2.0

Diff since v0.1.0