Summary
When ExecuteStreamingSql returns a large nested ARRAY<STRUCT<...>> value split across multiple PartialResultSets with chunked_value=true, ResultSet::merge (spanner/src/reader.rs) fails with:
Internal: previous_last kind mismatch: only StringValue and ListValue can be chunked
Repro (no live Spanner needed)
let previous_last = Value { kind: Some(Kind::ListValue(ListValue {
values: vec![value("a"), Value { kind: Some(Kind::NullValue(0)) }],
}))};
let current_first = Value { kind: Some(Kind::ListValue(ListValue {
values: vec![value("b"), Value { kind: Some(Kind::BoolValue(true)) }],
}))};
ResultSet::merge(previous_last, current_first).unwrap();
// expected: ["a", null, "b", true]
// actual: Err(Internal: previous_last kind mismatch ...)
Hit in production on a Spanner change-stream query (SELECT ChangeRecord FROM READ_<stream>(...)), where the column is a deeply nested ARRAY<STRUCT<...>> mixing strings, lists, bools, nulls, and numerics. Large transactions force chunking.
Root cause
spanner/src/reader.rs:199-214 recurses unconditionally on the inner-last / inner-first pair when merging two ListValues:
let first_value_of_current = first.values.remove(0);
let merged = match last.values.pop() {
Some(last_value_of_previous) => {
ResultSet::merge(last_value_of_previous, first_value_of_current)? // unconditional
}
None => first_value_of_current,
};
If the popped tail is Null/Bool/Number, the recursive call hits the catch-all at reader.rs:220-223 and errors. But Spanner's chunking can split between complete inner elements at one nesting level while the outer list is still incomplete — those should be appended, not merged.
Reference behavior
Both SDKs gate the recursion on a mergeability check (StringValue or ListValue on both sides) and append otherwise. Both also short-circuit when either inner list is empty; this crate doesn't.
Impact
Any caller streaming nested ARRAY<STRUCT<...>> whose values trigger chunking can hit this. Spanner change-stream readers are particularly exposed. Retry resumes from last offset, so data isn't permanently lost, but you get duplicate emissions on retry and permanent split failure if the same row deterministically trips the boundary.
Proposed fix
Mirror the reference SDKs: gate recursion on (StringValue, StringValue) | (ListValue, ListValue), otherwise push both elements and concatenate; also handle empty inner lists. Happy to send a PR — I have a working patch with two new regression tests, all 24 reader tests passing.
Environment
gcloud-spanner 1.8.0, Rust stable
- Spanner change streams, reproducible at unit-test level
Summary
When
ExecuteStreamingSqlreturns a large nestedARRAY<STRUCT<...>>value split across multiplePartialResultSets withchunked_value=true,ResultSet::merge(spanner/src/reader.rs) fails with:Repro (no live Spanner needed)
Hit in production on a Spanner change-stream query (
SELECT ChangeRecord FROM READ_<stream>(...)), where the column is a deeply nestedARRAY<STRUCT<...>>mixing strings, lists, bools, nulls, and numerics. Large transactions force chunking.Root cause
spanner/src/reader.rs:199-214recurses unconditionally on the inner-last / inner-first pair when merging twoListValues:If the popped tail is
Null/Bool/Number, the recursive call hits the catch-all atreader.rs:220-223and errors. But Spanner's chunking can split between complete inner elements at one nesting level while the outer list is still incomplete — those should be appended, not merged.Reference behavior
Both SDKs gate the recursion on a mergeability check (
StringValueorListValueon both sides) and append otherwise. Both also short-circuit when either inner list is empty; this crate doesn't.isMergeable+merge(spanner/read.go:868-940)merge_values(src/spanner/src/result_set.rs:574-614)Impact
Any caller streaming nested
ARRAY<STRUCT<...>>whose values trigger chunking can hit this. Spanner change-stream readers are particularly exposed. Retry resumes from last offset, so data isn't permanently lost, but you get duplicate emissions on retry and permanent split failure if the same row deterministically trips the boundary.Proposed fix
Mirror the reference SDKs: gate recursion on
(StringValue, StringValue) | (ListValue, ListValue), otherwise push both elements and concatenate; also handle empty inner lists. Happy to send a PR — I have a working patch with two new regression tests, all 24 reader tests passing.Environment
gcloud-spanner1.8.0, Rust stable