Skip to content

feat(client): add queryStream for chunked query results - #55

Merged
AayushMainali-Github merged 1 commit into
mainfrom
feat/36-query-stream
Jul 25, 2026
Merged

feat(client): add queryStream for chunked query results#55
AayushMainali-Github merged 1 commit into
mainfrom
feat/36-query-stream

Conversation

@AayushMainali-Github

Copy link
Copy Markdown
Owner

Closes #36.

What the live server actually does

The open questions in the issue were answered against community-latest (2.4.3) before committing to a shape.

chunked=true with Accept: application/json is neither one JSON value nor NDJSON. It is successive JSON arrays written back to back with no separator, each about 500 rows:

[{"a":1},{"a":2}][{"a":3}]

An empty result is an empty body. Invalid SQL still returns HTTP 422 before any array is sent, so a mid-stream failure after HTTP 200 is a transport problem rather than an application error. The same NULL-omission and alphabetical key sorting as buffered JSON apply inside each array. Recorded in docs/compatibility.md.

What shipped

client.queryStream(statement, options?) returns an async generator of rows. It sets chunked=true, parses each complete array as bytes arrive, and yields its elements so memory stays proportional to one server batch.

query() is unchanged and still buffers with chunked=false.

Partial results and cancellation

If the stream fails after some rows have already been yielded, the iterator throws and those rows stay consumed — the caller can see it is a partial result. Breaking out of the loop, or aborting options.signal, cancels the underlying reader so the connection is not left half-read. Retries (when configured) apply only to failures before the response body starts; a mid-stream failure cannot be retried without re-running the statement.

Tests

Unit tests cover split chunks, brackets inside strings, empty bodies, truncated and malformed streams, early consumer exit, and client wiring (chunked=true only on the stream path). Integration tests write 1,200 points (enough to span multiple 500-row batches), assert every value arrives exactly once, and confirm empty results and bad SQL behave as above.

@github-actions github-actions Bot added area: testing Area: testing area: documentation Area: documentation area: release Area: release area: governance Area: governance labels Jul 25, 2026
@AayushMainali-Github
AayushMainali-Github merged commit f183f3b into main Jul 25, 2026
14 checks passed
@AayushMainali-Github
AayushMainali-Github deleted the feat/36-query-stream branch July 25, 2026 04:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: documentation Area: documentation area: governance Area: governance area: release Area: release area: testing Area: testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC]: Streaming and chunked query results

1 participant