Skip to content

perf(bigquery): [T1-18] short-circuit duplicate retry evaluation and pre-compile regexes in BigQueryRetryAlgorithm - #23

Open
lqiu96 wants to merge 1 commit into
mainfrom
perf/bq-t1-18-retry-algorithm-regex
Open

lqiu96 wants to merge 1 commit into
mainfrom
perf/bq-t1-18-retry-algorithm-regex

Conversation

@lqiu96

@lqiu96 lqiu96 commented Oct 13, 2026

Copy link
Copy Markdown
Owner

Summary & Bottleneck

Every RPC invoked through BigQueryRetryHelper.runWithRetries calls BigQueryRetryAlgorithm.shouldRetry(Throwable, ResponseT, TimedAttemptSettings) (java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryRetryAlgorithm.java:L82-L90). On every retry check (including the initial createFirstAttempt() call where nextAttemptSettings == null on every single RPC), shouldRetry evaluated shouldRetryBasedOnResult(previousThrowable, previousResponse) and shouldRetryBasedOnBigQueryRetryConfig(previousThrowable, previousResponse, bigQueryRetryConfig) before checking nextAttemptSettings != null, AND then evaluated both predicates a second time inside the return statement. Furthermore, matchRegEx (BigQueryRetryAlgorithm.java:L164-L176) called Pattern.compile(regEx, Pattern.CASE_INSENSITIVE) on every retry evaluation, matchErrorMessages (BigQueryRetryAlgorithm.java:L149-L162) called .toLowerCase() on every configured error message string on every retry check, and BigQueryImpl.create(JobInfo, JobOption...) (java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java:L458) compiled Pattern.compile("Already Exists: Job (.+):(.+)\\.(.+)", Pattern.CASE_INSENSITIVE) inline.

Changes

  • Short-circuit BigQueryRetryAlgorithm.shouldRetry (java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryRetryAlgorithm.java) to return false immediately when nextAttemptSettings == null and evaluate the retry predicates once instead of twice.
  • Pre-compute ImmutableSet<String> lowerCaseRetriableErrorMessages and ImmutableList<Pattern> compiledRetriableRegExes inside BigQueryRetryConfig (java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryRetryConfig.java) at config construction time, and use them in BigQueryRetryAlgorithm.matchErrorMessages and BigQueryRetryAlgorithm.matchRegEx.
  • Hoist ALREADY_EXISTS_JOB_PATTERN as a private static final Pattern constant in BigQueryImpl (java-bigquery/google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryImpl.java).

Expected Impact

  • Complexity / Scaling: $O(\text{RPCs})$ and $O(\text{retry attempts})$.
  • Eliminates redundant predicate evaluation on the initial attempt of every RPC and halves retry evaluation overhead on failed attempts.
  • Eliminates per-attempt Pattern.compile(...) and per-config-entry String.toLowerCase(...) allocations.

Verification

  • Verified with mvn test -pl java-bigquery/google-cloud-bigquery -Dtest=BigQueryImplTest -Dclirr.skip=true -Denforcer.skip=true -Dcheckstyle.skip=true (151 tests passing).

…pre-compile regexes in BigQueryRetryAlgorithm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant