feat: add chunk row accessors for typed and string reads#129
Conversation
Thanks for the draft. I think the overall shape looks good. Keeping it as an explicit low-level, opt-in path is the right call. Please keep it focused. We can add more typed accessors later. Maybe add a test for empty and non-inlined strings as well. |
Thanks, that makes sense. I'll keep this PR focused and add coverage for empty strings and non-inlined strings in the StringRefs tests. |
mlafeldt
left a comment
There was a problem hiding this comment.
Thanks for this. A few things to fix before merge:
StringRefscrashes on NULL VARCHAR rowsIsNulldoesn't validaterowIdxTimestampMicrosSlicedoc is inaccurate- CI fails:
golangci-lint runreports 8 issues
See my inline comments.
Summary
ChunkRowsplusConn.QueryChunksContextandStmt.QueryChunksContextfor chunk-by-chunk result accessDataChunkaccessors forINTEGER,BIGINT,DOUBLE,TIMESTAMP, and zero-copyVARCHARreadsMotivation
The existing
database/sqlrow path is still the right default, but some consumers need a lower-overhead read path when they already process results chunk-wise. This change keeps the existing API untouched and adds an opt-in chunk API on top of the existingDataChunkabstraction.The typed accessors are intentionally narrow:
IsNullInt32SliceInt64SliceFloat64SliceTimestampMicrosSliceStringRefsAll returned views are documented as valid only until the next
NextChunkcall orClose.Benchmarks
On an Apple M1 with:
BenchmarkQueryChunksContextGetValueNumericScan:37.57 ms/op,1.06 MB/op,131162 allocs/opBenchmarkQueryChunksContextTypedNumericScan:4.66 ms/op,13.9 KB/op,346 allocs/opBenchmarkQueryChunksContextTypedStringScan:13.51 ms/op,1.07 MB/op,444 allocs/opBenchmarkQueryChunksContextCount:4.71 ms/op,18.1 KB/op,411 allocs/opThe main improvement is removing per-cell boxing and
GetValueoverhead when the caller can already consume chunk-oriented data.Validation
go test ./...go test ./... -run '^$' -bench 'BenchmarkQueryChunksContext(TypedNumericScan|GetValueNumericScan|TypedStringScan|Count)$' -benchmem -benchtime=1x