Skip to content

perf(bigquery): [T1-8] avoid redundant per-row FieldValueList re-wrapping in TableResult - #7

Open
lqiu96 wants to merge 1 commit into
mainfrom
perf/bq-t1-08-tableresult-schema-wrapping
Open

lqiu96 wants to merge 1 commit into
mainfrom
perf/bq-t1-08-tableresult-schema-wrapping

Conversation

@lqiu96

@lqiu96 lqiu96 commented Oct 13, 2026

Copy link
Copy Markdown
Owner

Summary & Bottleneck

  • In BigQueryImpl.transformTableData and ArrowDeserializer, every FieldValueList in pageNoSchema is already constructed with schema.getFields() attached.
  • However, TableResult.getValues() and TableResult.iterateAll() unconditionally wrap the underlying page iterable in addSchema(...), which invokes Iterables.transform(iter, list -> list.withSchema(getSchema().getFields())).
  • On every row visited via getValues() or iterateAll(), TableResult calls getSchema().getFields() inside Function.apply, and FieldValueList.withSchema(FieldList) allocates a new FieldValueList instance and copies this.row via ImmutableList.copyOf even when this.schema is already the exact same FieldList.
  • In addition, wrapping pageNoSchema.getValues() in Iterables.transform returns a lazy FluentIterable view that does not implement java.util.Collection, stripping known-size Collection and RandomAccess fast-paths for callers collecting or streaming page results.

Changes

  • In java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/FieldValueList.java:
    • Short-circuit withSchema(FieldList) to return this when this.schema == schema || Objects.equals(this.schema, schema).
    • Expose package-private getSchema() so TableResult can inspect whether a page's rows already carry the target FieldList.
  • In java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/TableResult.java:
    • Hoist getSchema().getFields() outside the Iterables.transform Function in addSchema.
    • In getValues(), return pageNoSchema.getValues() directly when it is a Collection whose elements already carry the matching FieldList.
  • In java-bigquery/google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/FieldValueListTest.java and java-bigquery/google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/TableResultTest.java:
    • Add unit tests verifying that withSchema returns the same instance when the schema matches and that TableResult.getValues() and TableResult.iterateAll() preserve Collection and row identity when rows already carry the schema.

Expected Impact

  • Eliminates $O(\text{rows})$ redundant FieldValueList allocations (24 bytes per row) on every TableResult.getValues() and TableResult.iterateAll() traversal when rows already carry the schema.
  • Preserves the Collection and List interfaces on TableResult.getValues() when the underlying page holds a materialized collection.

Verification

  • Ran mvn test -pl java-bigquery/google-cloud-bigquery -Dtest=FieldValueListTest,TableResultTest,BigQueryImplTest -Dclirr.skip=true -Denforcer.skip=true -Dflatten.skip=true -Dcheckstyle.skip=true -Dfmt.skip=true (169 tests run, 0 failures).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant