Skip to content

[BUG]: Can't Read File With All Null Column since PyArrow v25 #776

Description

@mukunku

Library Version

5.6.1

.NET Runtime

.NET 10

OS and Architecture

Windows (64-bit)

How to reproduce?

Test file: all_null_column_pyarrow_v25.zip

It can be opened with https://parquetreader.com/

Failing test

public async Task AllNullColumnPyArrowV25() {
	using Stream s = OpenTestFile("all_null_column_pyarrow_v25.parquet");
	await using ParquetReader r = await ParquetReader.CreateAsync(s);
	using ParquetRowGroupReader groupReader = r.OpenRowGroupReader(0);
	DataField[] fs = r.Schema.GetDataFields();
	double?[] data = await ReadNullableValuesAsync<double>(groupReader, fs[0]);
	Assert.Equal(46, data.Length);
	Assert.All(data, d => Assert.Null(d));

	data = await ReadNullableValuesAsync<double>(groupReader, fs[1]);
	Assert.Equal(46, data.Length);
	Assert.All(data, d => Assert.Equal(0, d));
}

Metadata

Metadata

Assignees

Labels

P1Priority 1 (highest). Always include in the next patch.🐞bug

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions