Skip to content

fix(dataframe): claim remaining restriction range on empty/header-only CSV reads - #39581

Open
ManvithPanyam wants to merge 1 commit into
apache:masterfrom
ManvithPanyam:fix-empty-csv-restriction-tracker
Open

fix(dataframe): claim remaining restriction range on empty/header-only CSV reads#39581
ManvithPanyam wants to merge 1 commit into
apache:masterfrom
ManvithPanyam:fix-empty-csv-restriction-tracker

Conversation

@ManvithPanyam

@ManvithPanyam ManvithPanyam commented Aug 2, 2026

Copy link
Copy Markdown

This is my first contribution to Apache Beam — happy to adjust the approach if there's a preferred pattern I'm missing.

What broke?

When reading an empty or header-only CSV file using beam.io.ReadFromCsv, the pipeline fails with:

ValueError: OffsetRestrictionTracker is not done since work in range [0, 16) has not been claimed.

Root Cause

In sdks/python/apache_beam/dataframe/io.py, _TruncatingFileHandle._read() returns early on an empty buffer without claiming the remaining restriction range, causing check_done() to fail.

Fix

if not self._buffer:
  self._tracker.try_claim(self._tracker.current_restriction().stop)
  self._done = True
  return self._empty

Verification

Added test_empty_csv_read to IOTest. Full suite (36 tests) passes with the fix; fails without it. Pre-existing Windows-only failures are unrelated (#20642).

…y CSV reads

Signed-off-by: ManvithPanyam <250704031+ManvithPanyam@users.noreply.github.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @tvalentyn for label python.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant