[lqiu96]

Summary

Streams and unwraps BigQuery REST v2 TableCell {"v": ...} envelopes directly in HttpBigQueryRpc for row-returning RPCs (listTableData, getQueryResults, and queryRpc), avoiding reflective GenericJson / TableCell / ArrayMap / EnumSet allocations per cell and bringing google-cloud-java to parity with the other official BigQuery client libraries.

Cross-Language Parity

Among the official Google Cloud BigQuery client libraries, Java was the only SDK instantiating a reflective Discovery model class (TableCell extends GenericJson extends GenericData, which invokes Class.getDeclaredConstructor(), allocates an unknownFields ArrayMap, and allocates an EnumSet of flags) for every cell in every row. Because the BigQuery REST v2 row wire format ("rows": [{"f": [{"v": ...}]}]) is strictly fixed by the protocol, all other official SDKs avoid reflective model instantiation on the rows hot path and unwrap "f" and "v" directly:

• Python (google-cloud-bigquery):
  • Parses the HTTP response body with Python's built-in json.loads without Discovery model classes, then walks row["f"] and unwraps cell["v"] directly in _row_tuple_from_json and CellDataParser.record_to_py (google/cloud/bigquery/_helpers.py).
• Node.js (@google-cloud/bigquery):
  • Parses the HTTP response body with JSON.parse into plain objects without wrapper classes or reflection, then maps row.f and unwraps field.v (and val.v for REPEATED elements) in BigQuery.mergeSchemaWithRows_ (src/bigquery.js).
• Go (cloud.google.com/go/bigquery + google.golang.org/api/bigquery/v2):
  • Represents TableCell as a 1-field struct (type TableCell struct { V interface{} }) with no unknown-fields map or reflection (bigquery/v2/bigquery-gen.go), decodes nested RECORD and REPEATED values inside V as plain map[string]interface{} and []interface{} via encoding/json, and indexes ["v"] and ["f"] directly in convertValues (bigquery/value.go).
• Rust (google-cloud-bigquery / google-cloud-bigquery-v2):
  • Deserializes rows as Vec<wkt::Struct> (serde_json::Map) without reflective Discovery models and immediately removes "f" and "v" in convert_row, get_field_list, and get_field_value (src/query/row.rs).
• Swift (google-cloud-swift):
  • Streams the "rows" / "f" / "v" UTF-8 JSON tokens in a single pass and unwraps "v" on the fly in RowJSONScanner (Sources/GoogleCloudBigQuery/Row.swift).

Details

• HttpBigQueryRpc (java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/spi/v2/HttpBigQueryRpc.java):
  • Streams the top-level "rows" array via parseTableRows / parseRowCells / parseCellValue while delegating all top-level response metadata (schema, pageToken, totalRows, jobReference, errors, etc.) to google-http-client's ClassInfo / JsonParser.
  • Pre-sizes each row's cell ArrayList using the column count observed on the first row of the page.
  • Unwraps single-key {"v": ...} cell envelopes in place and represents single-key {"f": [...]} RECORD (STRUCT) values with Collections.singletonMap.
• FieldValue.fromPb (java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/FieldValue.java):
  • Accepts both unwrapped cell values and TableCell / Map wrappers.

Performance Benchmark (16 columns per row)

1. Deterministic Client-Side HTTP Stream Benchmark (300,000 rows / 4,800,000 cells)

Parsing an in-memory TableDataList JSON stream through BigQuery.listTableData -> HttpBigQueryRpc -> FieldValueList.fromPb (isolating client deserialization from BigQuery server and network variance):

Version	Rows (16 cols)	Parse + Iterate Time	Client Throughput	Total Heap Allocated	Allocation per Cell
Baseline (2.71.0)	300,000	2.704 s	110,950 rows/s	1,296.5 MB	283.2 bytes/cell
PR #4 (perf/bigquery-unwrap-cell-json)	300,000	0.699 s (-74.1%)	429,424 rows/s (3.87x)	701.4 MB (-45.9%)	153.2 bytes/cell (-130.0 B/cell)

2. Live BigQuery End-to-End Benchmark (tpch_10T.lineitem, 1k to 10M rows)

Measured against lawrence-test-project-2:tpch_10T.lineitem (16 columns per row) in fresh JVM processes. Live REST pagination is ~96% network-bound (~40 s of client CPU time out of ~1,000-1,150 s total wall time across 10M rows). Note that jobs.query below was run with setUseQueryCache(false) (forcing a cold server scan of tpch_10T.lineitem, e.g., 29.93 s initial server execution at 10M rows before returning page 1, followed by 989.64 s for row iteration).

Workload	Rows (N)	Non-Null Cells	Baseline (2.71.0) Wall Time	PR #4 Initial + Iter = Total Wall Time	Baseline Final / Peak RSS	PR #4 Final / Peak RSS	Peak RSS Delta	PR #4 CPU Time	PR #4 Heap Alloc (bytes/cell)
tabledata.list	1,000	16,000	7.18 s (139 rows/s)	7.10 s + 0.004 s = 7.11 s (141 rows/s)	413.9 / 413.9 MB	258.1 / 258.1 MB	-37.6%	0.654 s	155.2 MB
tabledata.list	100,000	1,600,000	13.44 s (7,438 rows/s)	6.85 s + 6.52 s = 13.36 s (7,483 rows/s)	642.8 / 642.8 MB	431.8 / 431.8 MB	-32.8%	1.027 s (-43.2%)	297.2 MB (194.8 B/cell, -46.9%)
tabledata.list	1,000,000	16,000,000	114.86 s (8,707 rows/s)	6.34 s + 107.80 s = 114.14 s (8,761 rows/s)	1,683.6 / 1,683.6 MB	1,117.8 / 1,117.8 MB	-33.6%	4.680 s	2,421.2 MB (158.7 B/cell)
tabledata.list	10,000,000	160,000,000	1,144.72 s (8,736 rows/s)	6.61 s + 1,146.55 s = 1,153.16 s (8,672 rows/s)	1,891.1 / 2,095.4 MB	1,479.4 / 1,565.0 MB	-25.3% (-21.8% Final)	39.401 s	23,892.9 MB (156.6 B/cell)
jobs.query	1,000	16,000	1.69 s (cached)	2.03 s (cached) / 3.93 s (cold, 0.004 s iter)	145.4 / 145.4 MB	139.7 / 139.7 MB	-3.9%	0.389 s	16.9 MB
jobs.query	100,000	1,600,000	12.69 s (cached) / 16.94 s (cold)	14.16 s (cached) / 18.03 s (cold)	587.3 / 587.3 MB	366.9 / 366.9 MB	-37.5%	0.953 s (-48.7%)	263.4 MB (172.6 B/cell, -46.1%)
jobs.query	1,000,000	16,000,000	147.83 s (cold)	13.30 s + 104.24 s = 117.55 s (8,507 rows/s)	787.1 / 787.1 MB	514.8 / 514.8 MB	-34.6%	4.638 s	2,440.7 MB (160.0 B/cell)
jobs.query	10,000,000	160,000,000	997.37 s (cached)	29.93 s (cold) + 989.64 s iter = 1,019.57 s	1,619.0 / 2,063.9 MB	1,178.4 / 1,548.7 MB	-25.0% (-27.2% Final)	40.671 s	24,099.6 MB (157.9 B/cell)