feat(pubsub): implement publish hedging to reduce tail latency by tonyyyycui · Pull Request #13735 · googleapis/google-cloud-java

tonyyyycui · 2026-07-13T17:31:20Z

This PR implements publish hedging in the Java Cloud Pub/Sub Publisher. Specifically, it adds support for scheduling "hedged" publish attempts when a publish call is slow to respond. A token-based method is utilized to rate-limit hedged requests and a coordinator is used to manage/cancel concurrent requests to prevent duplicate publishes.

gemini-code-assist

Code Review

This pull request introduces a hedging mechanism for the Pub/Sub Publisher, adding HedgeSettings for configuration and a thread-safe HedgeTokenBucket to limit hedged requests. The feedback suggests replacing the synchronized methods in HedgeTokenBucket with explicit ReentrantLock to reduce lock contention and improve performance on the critical path.

…ngs and tests

michaelpri10

At a higher level, I think the full implementation should be one PR (instead of the , given any submitted changes become public. It would be unexpected for customers to be able to set HedgingSettings without anything actually happening, so the final PR should have everything needed for the implementation included.

michaelpri10 · 2026-07-17T20:01:39Z

+   *
+   * @return the hedging delay.
+   */
+  public Duration getHedgeDelay() {


nit: Does this method need to be public? I think leaving it without an access modifier (i.e., making it package-private) should be sufficient.

I've removed it for now, but I think it would also make sense if the user could see what they see the hedge delay is set to since it's a configurable field. Leaving this conversation unresolved.

…lso add Unit tests

michaelpri10 · 2026-07-24T17:00:52Z

+    final int batchSize = outstandingBatch.outstandingPublishes.size();
+    for (final OutstandingPublish outstanding : outstandingBatch.outstandingPublishes) {
+      outstanding.publishResult.addListener(
+          new Runnable() {


I worried that this approach could have cause additional, unneeded overhead. The cancellation check be done in processQueue instead to avoid needing to create these Runnables, but then we run into an issue of the cancellation not propagating to in-flight hedged requests (i.e., publish at t=0, hedge at t=1000, cancellation at t=1050). This may be okay behavior given that cancellations are best effort. I'll raise this issue with the team.

…and maxtokens, updated tests.

…r for hedgeTokenBucket. Also added HEDGE_TOKEN_SCALE.

…CancellationSharer.java

…delay

…thod

…rpc call

abbrowne126 · 2026-07-30T16:25:09Z

+    }
+    this.clock = builder.clock != null ? builder.clock : CurrentMillisClock.getDefaultClock();
    this.publishContext = GrpcCallContext.createDefault();
+    this.hedgingMetadata = ImmutableMap.of("x-goog-pubsub-hedged", ImmutableList.of("true"));


I think we wanted this to be the attempt count, not just true/false. Is it simple to adjust this to something like x-google-pubsub-hedged-count?

…failure codes

…le-cloud-java into publish-hedging-settings # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

michaelpri10 · 2026-07-30T20:16:17Z

    shutdown = new AtomicBoolean(false);
    messagesWaiter = new Waiter();
+    this.hedgeSettings = builder.hedgeSettings;
+    if (this.hedgeSettings != null) {


Could this be checked in the Publisher.Builder.build() method instead?

michaelpri10 · 2026-07-30T20:23:42Z

+  }
+
+  void removeFromHedgingQueue(CancellationSharer coordinator) {
+    queueLock.lock();


Is a lock actually needed here (and in processQueue)? hedgingQueue is a concurrent data structure and there are other atomic variables used as well.

michaelpri10 · 2026-07-31T17:43:04Z

+    return publishCall(outstandingBatch, 0);
+  }
+
+  private ApiFuture<PublishResponse> publishCall(


Something I missed from the design:

The timeout of the hedged RPCs should be “total deadline - now”, but capped at 10s overall which is the total time that the server allows for a publish attempt.

I think this will need to be enforced in this method by adding a timeout on the context object we are already modify. We'll need to propagate the absolute deadline here somehow.

michaelpri10 · 2026-07-31T17:50:52Z


+  private final HedgeSettings hedgeSettings;
+  private final HedgeTokenBucket hedgeTokenBucket;
+  private final ApiClock clock;


Sounds good, I think it is fine to use this then.

michaelpri10 · 2026-07-31T17:54:37Z

+   * Creates a publish start event that is tied to the publish RPC span time, marking hedged
+   * attempts explicitly.
+   */
+  void addPublishStartEvent(int attemptNumber) {


We should have a publish end (hedged) event as well.

michaelpri10 · 2026-07-31T19:50:20Z

+  private void processQueue() {
+    queueLock.lock();
+    try {
+      isQueueProcessingScheduled.set(false);


Going along with this comment, I think we will also need to move the isQueueProcessingScheduled.set(false) call to after the while loop (but before the scheduleQueueProcessing() call). This will prevent a scheduleQueueProcessing call from startHedgedCall from scheduling another processQueue() call without needing the lock in processQueue().

michaelpri10 · 2026-07-31T20:24:37Z

+   * @param attemptNumber the 1-based index of the attempt (1 is original, 2+ are hedged)
+   * @param future the future representing the gRPC call for this attempt
+   */
+  void addAttempt(final int attemptNumber, ApiFuture<PublishResponse> future) {


We may need a better synchronization method across done and runningAttempts here using a lock that controls both of them. For example, if checkCompletionOnQueueExit races with handleAttemptFailure, the following can happen:

handleAttemptFailure -> fails with PERMISSION_DENIED Calls runningAttempts.remove(attemptNumber) // may become empty // switch threads In checkCompletionOnQueueExit, if (!done.get() && runningAttempts.isEmpty() && !isInQueue.get()) -> true because runningAttempts just became empty setException(RuntimeException("Hedging failed with no active attempts")) // switch threads setException(PERMISSION_DENIED) // not honored because setException already called

We'd return the wrong error in this case. I believe there are similar issues across all of the methods here, so we should use a lock here to guard the done and runningAttempts members. They likely don't need to be concurrent/atomic data structures anymore.

…n to publisher

…ket operations

Tony Cui added 4 commits July 13, 2026 15:16

Add hedgeSettings with configurable hedgeDelay

61d0b60

Add HedgeTokenBucket class to allow for token operations

19afd68

Add publisher integration with HedgeSettings

49426a9

Restore original @ignore annotations and code in PublisherImplTest

22417d1

tonyyyycui requested review from a team as code owners July 13, 2026 17:31

gemini-code-assist Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread java-pubsub/google-cloud-pubsub/src/main/java/com/google/cloud/pubsub/v1/HedgeTokenBucket.java Outdated

tonyyyycui changed the title ~~Add HedgeSettings and integrate with publisher for token operations~~ feat(pubsub): add HedgeSettings and integrate with Publisher Jul 13, 2026

tonyyyycui changed the title ~~feat(pubsub): add HedgeSettings and integrate with Publisher~~ feat(pubsub): implement HedgeSettings and integrate with Publisher Jul 13, 2026

Tony Cui and others added 3 commits July 13, 2026 17:39

style(pubsub): fix checkstyle violations in HedgeSettings

91706a2

style(pubsub): fix formatting and checkstyle violations in HedgeSetti…

bd76ede

…ngs and tests

Merge branch 'main' into publish-hedging-settings

61aef08

michaelpri10 reviewed Jul 17, 2026

View reviewed changes

Tony Cui added 4 commits July 20, 2026 15:17

Resolve git comments related to access modifiers and constraints

227f808

Add CancellationSharer and HedgedRequest helper classes

e29eab8

Integrate CancellationSharer and HedgedRequest into Publisher.java. A…

ed05c60

…lso add Unit tests

fix lint

23b8432

tonyyyycui changed the title ~~feat(pubsub): implement HedgeSettings and integrate with Publisher~~ feat(pubsub): implement publish hedging to reduce tail latency Jul 20, 2026

Modifying comment line

3a23d60

michaelpri10 requested changes Jul 24, 2026

View reviewed changes

Tony Cui added 9 commits July 27, 2026 14:40

Modified default values, added configurable settings to refill ratio …

da4307e

…and maxtokens, updated tests.

Removed HedgeTokenBucket.java class in favor of using an atomicintege…

336c809

…r for hedgeTokenBucket. Also added HEDGE_TOKEN_SCALE.

Add documentation for scaling and remove unused getAttemptCount() in …

59a7a09

…CancellationSharer.java

Added method overloading to handle OTel span for hedged publish events

79411a8

Verify that the per-RPC timeout is strictly greater than the hedging …

e545cd6

…delay

Moved static methods in Publisher.java above the builder

8543d23

Configure empty token bucket and fix naming

3c48a1a

Change from synchronization to actual lock object for processQueue me…

0929a1b

…thod

Add header injection for hedged publish requests

0295144

Tony Cui added 2 commits July 28, 2026 20:19

Cleanup code and fix bug with handling failure attempts in the first …

c8296f5

…rpc call

Disable retries for failing rpcs when publish hedging

c8fd6e9

abbrowne126 reviewed Jul 30, 2026

View reviewed changes

Change how cancellationSharer propagates cancellations for different …

0b88e33

…failure codes

tonyyyycui force-pushed the publish-hedging-settings branch from d9a8655 to 0b88e33 Compare July 30, 2026 19:11

Tony Cui and others added 5 commits July 30, 2026 19:29

Fixing race condition in unit test setup

b6af015

Merge branch 'main' into publish-hedging-settings

be69b98

Modify header to use an integer count instead of a bool

1a4644f

Fix lint

dd288c7

michaelpri10 reviewed Aug 3, 2026

View reviewed changes

Tony Cui added 6 commits August 4, 2026 01:15

Modify scale factor and minimum refillRatio value. Add test annotatio…

8885dc2

…n to publisher

Update scale factor, refill ratio constraints, and simplify token buc…

2eb687f

…ket operations

Rename HedgeSettings to HedgingSettings

100db18

Inline the hedging attempt count headers map creation

ba05985

Added new subsystem to loggerUtil and adjusted logging usage

97820e7

Add debugging for token bucket rate limiting

58a4f70

Uh oh!

Conversation

tonyyyycui commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

michaelpri10 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tonyyyycui commented Jul 13, 2026 •

edited

Loading