[lqiu96]

Summary & Bottleneck

Streaming ingestion (InsertAllRequest, BigQueryImpl.insertAll, and InsertAllResponse) performs several redundant collection allocations and un-sized transformations that scale with row count $O(\text{rows})$:

• InsertAllRequest.Builder.setRows(Iterable<RowToInsert>) copies incoming rows into a java.util.LinkedList (Lists.newLinkedList), allocating a 24-byte LinkedList$Node wrapper per row, right before new InsertAllRequest(Builder) copies the list again into ImmutableList.copyOf(builder.rows).
• BigQueryImpl.insertAll transforms request.getRows() using FluentIterable.from(request.getRows()).transform(...).toList() and a 1-element boolean[] heap array. Because FluentIterable.transform wraps the list in a non-Collection Iterable, .toList() loses the known row count and repeatedly resizes its internal Object[] buffer (~14 array copies for a 10,000-row batch).
• InsertAllResponse.fromPb builds an intermediate HashMap populated with lazy Lists.transform views over ErrorProto wire objects and then copies the map into ImmutableMap.copyOf(insertErrors). Similarly, BigQueryImpl.queryRpc, queryRpcArrow, and queryArrow pass lazy Lists.transform error views to BigQueryException. These lazy views re-execute BigQueryError.fromPb on every traversal and retain the underlying ErrorProto (GenericJson) wire objects in memory.

Changes

• In java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/InsertAllRequest.java, replace Lists.newLinkedList with Lists.newArrayList in Builder.setRows.
• In java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java, replace the FluentIterable.from(request.getRows()).transform(...).toList() pipeline and boolean[1] array in insertAll with a pre-sized new ArrayList<>(rows.size()) loop and a primitive boolean allInsertIdsSet.
• In java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/InsertAllResponse.java and java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java, build ImmutableMap.builderWithExpectedSize directly in InsertAllResponse.fromPb and eagerly materialize ImmutableList<BigQueryError> instances using ImmutableList.builderWithExpectedSize.
• In java-bigquery/google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/InsertAllResponseTest.java, assert that InsertAllResponse.fromPb returns materialized BigQueryError instances across repeated accesses.

Expected Impact

• Eliminates $N$ LinkedList$Node allocations (24 bytes per row) when staging batch rows in InsertAllRequest.Builder.setRows.
• Eliminates FluentIterable, Function, boolean[1], and ~14 intermediate Object[] resize copies per 10,000-row insertAll batch.
• Eliminates the intermediate HashMap allocation and re-hashing in InsertAllResponse.fromPb, prevents repeated BigQueryError allocations on inspection, and allows ErrorProto wire DTOs to be garbage-collected immediately.

Verification

• Ran unit tests in InsertAllRequestTest, InsertAllResponseTest, and BigQueryImplTest:

  mvn test -pl java-bigquery/google-cloud-bigquery -Dtest=InsertAllRequestTest,InsertAllResponseTest,BigQueryImplTest -Dclirr.skip=true -Denforcer.skip=true -Dflatten.skip=true -Dcheckstyle.skip=true -Dfmt.skip=true