Skip to content

perf(bigquery): [T1-5] eliminate BigDecimal/String.split allocations and precision loss in BigQueryResultImpl - #22

Open
lqiu96 wants to merge 1 commit into
mainfrom
perf/bq-t1-05-resultset-numeric-and-time
Open

lqiu96 wants to merge 1 commit into
mainfrom
perf/bq-t1-05-resultset-numeric-and-time

Conversation

@lqiu96

@lqiu96 lqiu96 commented Oct 13, 2026

Copy link
Copy Markdown
Owner

Summary & Bottleneck

  • Target file: java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryResultImpl.java (L122-L133, L387-L567, L751-L769)
  • Scaling cardinality: $O(\text{rows} \times \text{cols})$ allocations and CPU overhead across JDBC ResultSet numeric, decimal, time, and array getters.
  • Bottleneck details:
    1. getInt, getLong, and getDouble on FieldValueList rows called fieldValue.getNumericValue() (new BigDecimal(getStringValue())) on every cell instead of parsing primitive int, long, and double values directly. In addition, getInt(int columnIndex) omitted wasNull = false on non-null FieldValueList cells.
    2. On the Storage Read API (Arrow) branch, getDouble converted an already-numeric Double or BigDecimal object into a String and parsed a new BigDecimal (new BigDecimal(curVal.toString()).doubleValue()).
    3. getBigDecimal called BigDecimal.valueOf(fieldValue.getNumericValue().doubleValue()) on FieldValueList rows and BigDecimal.valueOf(getDouble(fieldName)) on Read API rows, allocating two BigDecimal objects per cell and truncating 38-digit NUMERIC and 76-digit BIGNUMERIC values to 53-bit IEEE-754 double precision.
    4. getTimeFromFieldVal invoked strTime.split(":") and timeSplt[2].split("\\.") (compiling a regex Pattern on every fractional TIME cell because "\\." is two characters).
    5. BigQueryArrayResult copied Object[] into new ArrayList<>(Arrays.asList(...)) and copied FieldValueList into new ArrayList<>((FieldValueList) array) on every getArray call.

Changes

  • Primitive numeric parsing and wasNull fix:
    • Added parseFieldValueInt and parseFieldValueLong to parse integer strings directly via Integer.parseInt and Long.parseLong (falling back to getNumericValue() only when decimal or exponent characters are present), and updated getInt(int columnIndex) to set wasNull = false.
    • Updated getDouble to call fieldValue.getDoubleValue() on FieldValueList rows and ((Number) curVal).doubleValue() on Read API Number values.
  • Direct BigDecimal retrieval without double truncation:
    • Updated getBigDecimal to return fieldValue.getNumericValue() on FieldValueList rows and return (BigDecimal) curVal directly when the Read API value is already a BigDecimal.
  • Regex-free TIME parsing and zero-copy BigQueryArrayResult:
    • Replaced strTime.split(":") and split("\\.") in getTimeFromFieldVal with indexOf(':') and indexOf('.').
    • Updated BigQueryArrayResult to wrap Object[] via Arrays.asList and retain List<?> (including FieldValueList) without copying elements into a new ArrayList.

Expected Impact

  • Eliminates $R \times C$ intermediate BigDecimal and String allocations across getInt, getLong, getDouble, and getBigDecimal.
  • Fixes silent 53-bit double precision loss on NUMERIC (38 digits) and BIGNUMERIC (76 digits) columns in ResultSet.getBigDecimal.
  • Eliminates regex Pattern compilation and String[] allocations on every ResultSet.getTime call and eliminates redundant ArrayList copies on ResultSet.getArray.

Verification

  • Ran mvn test -pl java-bigquery/google-cloud-bigquery -Dtest=BigQueryResultImplTest,ConnectionImplTest (26 tests run, 0 failures).
  • Added BigQueryResultImplTest.testResultSetNumericPrecisionWasNullAndFractionalTime verifying 38-digit BigDecimal precision preservation across both FieldValueList and Read API rows, wasNull reset on getInt(int), and fractional TIME parsing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant