Skip to content

fix(rfc): refuse the ROWSKIPS ceiling instead of truncating, and bound the fallback retry - #139

Merged
jrosskopf merged 2 commits into
masterfrom
claude/fix-partition-address-limit
Sep 4, 2026
Merged

fix(rfc): refuse the ROWSKIPS ceiling instead of truncating, and bound the fallback retry#139
jrosskopf merged 2 commits into
masterfrom
claude/fix-partition-address-limit

Conversation

@jrosskopf

@jrosskopf jrosskopf commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Two defects in the partitioned scan path, both found by a six-persona review crew over the
day's diff and each verified against the code before being fixed.

1. The ROWSKIPS ceiling returned a truncated result instead of refusing

RfcRowWindowScheduler::Claim() signalled the ABAP INT4 limit by returning false — the
same answer it gives when a table is exhausted. RfcReadTableScanPartitioned treats that
as normal completion and retires the worker with an empty chunk, and DuckDB reads an
empty chunk as end-of-scan
. So a scan past 2,147,483,647 rows returned a truncated prefix
and called it success, while API_REFERENCE states that erpl refuses rather than wrapping.

The guard exists to stop next_offset wrapping and handing out already-read offsets —
duplicated rows. Collapsing it into "exhausted" traded that for missing rows, which is
no better and is harder to notice. This is the same silent-wrong-results class as the two
defects fixed earlier in this scan path, and I introduced it.

Claim now returns CLAIMED / EXHAUSTED / ADDRESS_LIMIT; the scan raises on the last,
naming the limit and what to do about it.

Off-by-one in the same guard. claimed > INT32_MAX - window_size refused the last
legal window
(start 2,147,450,880 at a 32768-row window). next_offset is 64-bit, so
subtracting the window bought no wrap protection, and every individual ROWSKIPS inside
that window is already range-checked loudly in CreateFunctionArguments. The bound is now
claimed > INT32_MAX.

An existing test asserted the old behaviour ("exactly one claim is possible") — it had
encoded the off-by-one, and now asserts the corrected contract: two starts are legal, the
third is refused.

2. The RFC_READ_TABLE fallback could retry without limit

In the TABLE_WITHOUT_DATA branch the retry used continue without incrementing attempt
and without sleeping. TrySelectFallbackReadTableFunction returns a cached success once
the switch has happened, so a second TABLE_WITHOUT_DATA from the fallback function was an
unbounded tight loop against the SAP system with no delay — multiplied by the partition
count. Switching is one-way, so a repeat is a real failure.

It now retries only when the selected function actually changed; otherwise the original
error propagates.

Verification

  • Offline: partition scheduler 13/13 (two new boundary cases), batching 6/6.
  • Live: RFC 32/32 on nwrfc, 32/32 on erpl-proto.

Not fixed here

The review's other high finding — partitioned scans invalidate the RFC connection at every
window and exhaust the 16-slot persistent cache, so the default configuration performs a
logon per call — is a real performance defect but wants its own change and its own
before/after logon-rate measurement, not a ride-along in a correctness PR.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…d the fallback retry

Two defects in the partitioned scan path, both found by a multi-agent review of the
day's diff and confirmed against the code.

1. Claim() reported the ABAP INT4 ceiling by returning false -- the same answer it
   gives when a table is exhausted. RfcReadTableScanPartitioned treats that as normal
   completion and retires the worker with an empty chunk, and DuckDB reads an empty
   chunk as end-of-scan. A scan past 2,147,483,647 rows therefore returned a truncated
   prefix and called it success, while API_REFERENCE says erpl refuses rather than
   wrapping.

   This guard was added to stop next_offset wrapping and handing out already-read
   offsets -- duplicated rows. Collapsing it into "exhausted" traded that for missing
   rows, which is no better and is harder to notice. Claim now returns
   CLAIMED / EXHAUSTED / ADDRESS_LIMIT and the scan raises on the last.

   The bound was also off by one: `claimed > INT32_MAX - window_size` refused the last
   legal window start (2,147,450,880 at a 32768 window). next_offset is 64-bit so
   subtracting the window bought no wrap protection, and every individual ROWSKIPS
   inside that window is already range-checked loudly in CreateFunctionArguments. An
   existing test asserted the old behaviour ("exactly one claim is possible") and had
   encoded the off-by-one; it now asserts the corrected contract.

2. The TABLE_WITHOUT_DATA fallback branch retried with `continue` without incrementing
   the attempt counter and without sleeping. TrySelectFallbackReadTableFunction returns
   a cached success once the switch has happened, so a second TABLE_WITHOUT_DATA from
   the fallback function looped forever with no delay, once per partition worker,
   against a shared SAP system. It now retries only if the selected function actually
   changed; otherwise the original error propagates.

Offline: partition scheduler 13/13, batching 6/6.
Live: RFC 32/32 on nwrfc and 32/32 on erpl-proto.
@jrosskopf
jrosskopf merged commit d96e3da into master Sep 4, 2026
53 checks passed
@jrosskopf
jrosskopf deleted the claude/fix-partition-address-limit branch September 4, 2026 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant