SCRUM-1161-explore(query): state the exact shortfall on a mid-batch refusal - #332
Open
emnarahmouni-exmergo wants to merge 1 commit into
Open
SCRUM-1161-explore(query): state the exact shortfall on a mid-batch refusal#332emnarahmouni-exmergo wants to merge 1 commit into
emnarahmouni-exmergo wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes : #321
("a multi-statement call strands its last statement, because the estimate reserves one 10 MB floor for the whole batch") and fixes the real, narrower gap found. What shipped is not the issue's literal proposal, because the literal bug doesn't reproduce.
What I found
The issue's headline claim doesn't reproduce on this codebase. A 7-statement batch, each statement reading almost nothing, already gets
estimated_bytes == 7 x 10,485,760(summed per-statement, since each already floors on its own distinct tables viaquery_estimate), and confirming at exactly that estimate already completes all 7 statements. Both acceptance criteria hold today for that literal scenario.Pushing further, the same execution-time billing variance #320 fixes within one statement also applies across a batch: if each statement's real bill sits just far enough above its own floor, and the dry run under-reports enough that the estimate still floors to exactly the per-query minimum (masking the gap) while the real bill stays above it, the shortfall compounds across the batch until the tail is stranded after several statements have already run and billed. I reproduced this exactly (6 of 7 statements succeed, the 7th fails just under the floor).
This is a real, narrower gap than the issue's literal framing, and it's why #320 and #321 are siblings: same underlying mechanism, one hitting a single multi-table statement's own cap, the other hitting a multi-statement batch's cumulative total.
The fix
_executealready keeps and reports every statement that completed before a cost-guard refusal, since it's already been paid for that part of the existing design is sound and unchanged. What was missing: the refusal for the failed statement said why it failed, but not what it would take to finish.A mid-batch
OverCeilingErrornow recomputes, from the samequery_estimatethe original handshake used, exactly how many more bytes complete every remaining (not-yet-run) statement, and states that in the same refusal:A caller now sets the right
--budgeton the first re-run instead of raising it by a guess and possibly hitting the same wall again. A no-op for a lone, non-batch statement, where the existing message already says everything there is to say.Before the fix :


After the fix :
the error now continues: "Completing the remaining 1 statement(s) in this batch needs at least 10,892,606 more bytes_scanned; the statement(s) already run above are saved, so a wider --budget on re-run does not re-pay for them."