Summary
json_dataset() and csv_dataset() do not reliably resolve relative sample files on Windows or when the dataset path is a filesystem URI.
On Windows, the existing tests/dataset/test_dataset.py::test_dataset_image_paths test fails on current main because images/ballons.png remains relative instead of resolving beside images.jsonl.
Root cause
The readers store Dataset.location with os.path.abspath(), while resolve_sample_files() derives the parent by splitting that value on the fsspec filesystem separator.
- Windows
abspath() produces backslashes, while LocalFileSystem.sep is /, so the split does not find the parent directory.
os.path.abspath() treats file://, s3://, and other filesystem URIs as local path strings, corrupting the stored location before relative files are resolved.
Reproduction
uv run pytest tests/dataset/test_dataset.py::test_dataset_image_paths -vv
Current result on Windows:
AssertionError: assert WindowsPath('images/ballons.png').exists()
Expected behavior
Dataset locations should retain their filesystem scheme, and relative sample files should resolve against the dataset's parent directory on local and remote filesystems.
I can open a focused PR using the repository's filesystem-aware path helpers, with regression coverage for local paths and filesystem URIs.
Summary
json_dataset()andcsv_dataset()do not reliably resolve relative sample files on Windows or when the dataset path is a filesystem URI.On Windows, the existing
tests/dataset/test_dataset.py::test_dataset_image_pathstest fails on currentmainbecauseimages/ballons.pngremains relative instead of resolving besideimages.jsonl.Root cause
The readers store
Dataset.locationwithos.path.abspath(), whileresolve_sample_files()derives the parent by splitting that value on the fsspec filesystem separator.abspath()produces backslashes, whileLocalFileSystem.sepis/, so the split does not find the parent directory.os.path.abspath()treatsfile://,s3://, and other filesystem URIs as local path strings, corrupting the stored location before relative files are resolved.Reproduction
uv run pytest tests/dataset/test_dataset.py::test_dataset_image_paths -vvCurrent result on Windows:
Expected behavior
Dataset locations should retain their filesystem scheme, and relative sample files should resolve against the dataset's parent directory on local and remote filesystems.
I can open a focused PR using the repository's filesystem-aware path helpers, with regression coverage for local paths and filesystem URIs.