Skip to content

bug: ResultSet::merge returns Internal: previous_last kind mismatch for chunked nested ListValues #438

Description

@tuantran0910

Summary

When ExecuteStreamingSql returns a large nested ARRAY<STRUCT<...>> value split across multiple PartialResultSets with chunked_value=true, ResultSet::merge (spanner/src/reader.rs) fails with:

Internal: previous_last kind mismatch: only StringValue and ListValue can be chunked

Repro (no live Spanner needed)

let previous_last = Value { kind: Some(Kind::ListValue(ListValue {
    values: vec![value("a"), Value { kind: Some(Kind::NullValue(0)) }],
}))};
let current_first = Value { kind: Some(Kind::ListValue(ListValue {
    values: vec![value("b"), Value { kind: Some(Kind::BoolValue(true)) }],
}))};
ResultSet::merge(previous_last, current_first).unwrap();
// expected: ["a", null, "b", true]
// actual:   Err(Internal: previous_last kind mismatch ...)

Hit in production on a Spanner change-stream query (SELECT ChangeRecord FROM READ_<stream>(...)), where the column is a deeply nested ARRAY<STRUCT<...>> mixing strings, lists, bools, nulls, and numerics. Large transactions force chunking.

Root cause

spanner/src/reader.rs:199-214 recurses unconditionally on the inner-last / inner-first pair when merging two ListValues:

let first_value_of_current = first.values.remove(0);
let merged = match last.values.pop() {
    Some(last_value_of_previous) => {
        ResultSet::merge(last_value_of_previous, first_value_of_current)?  // unconditional
    }
    None => first_value_of_current,
};

If the popped tail is Null/Bool/Number, the recursive call hits the catch-all at reader.rs:220-223 and errors. But Spanner's chunking can split between complete inner elements at one nesting level while the outer list is still incomplete — those should be appended, not merged.

Reference behavior

Both SDKs gate the recursion on a mergeability check (StringValue or ListValue on both sides) and append otherwise. Both also short-circuit when either inner list is empty; this crate doesn't.

Impact

Any caller streaming nested ARRAY<STRUCT<...>> whose values trigger chunking can hit this. Spanner change-stream readers are particularly exposed. Retry resumes from last offset, so data isn't permanently lost, but you get duplicate emissions on retry and permanent split failure if the same row deterministically trips the boundary.

Proposed fix

Mirror the reference SDKs: gate recursion on (StringValue, StringValue) | (ListValue, ListValue), otherwise push both elements and concatenate; also handle empty inner lists. Happy to send a PR — I have a working patch with two new regression tests, all 24 reader tests passing.

Environment

  • gcloud-spanner 1.8.0, Rust stable
  • Spanner change streams, reproducible at unit-test level

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions