Skip to content

client/v3: emit a single resolver update in EtcdManualResolver.Build - #22133

Open
Sunnatillo wants to merge 1 commit into
etcd-io:mainfrom
Nordix:Sunnatillo/rm-unnecessary-conn-to-grpc
Open

client/v3: emit a single resolver update in EtcdManualResolver.Build#22133
Sunnatillo wants to merge 1 commit into
etcd-io:mainfrom
Nordix:Sunnatillo/rm-unnecessary-conn-to-grpc

Conversation

@Sunnatillo

@Sunnatillo Sunnatillo commented Jul 21, 2026

Copy link
Copy Markdown

Build pushed the endpoints and round_robin ServiceConfig via a follow-up updateState call after building the embedded manual.Resolver. gRPC saw an initial state without the ServiceConfig, then a second update with it, forcing a mid-connection balancer switch that canceled an in-flight dial and logged spurious "operation was canceled" warnings on every client.

Seed the state via manual.Resolver.InitialState before Build so gRPC gets one update with both endpoints and the ServiceConfig. Add resolver tests and allowlist the experimental resolver fields they read.

Coauthored with: BootstrapperSBL
Fixes: #21660

@kubernetes-prow

Copy link
Copy Markdown

Hi @Sunnatillo. Thanks for your PR.

I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@Sunnatillo

Copy link
Copy Markdown
Author

/cc @ahrtr

@kubernetes-prow
kubernetes-prow Bot requested a review from ahrtr July 21, 2026 12:23
Comment on lines +19 to +20
resolver.ServerName
resolver.ServiceConfig

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Reworked it to add no experimental API - reverted the allowlist and rewrote the tests to compare whole resolver.State values

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ahrtr do the changes after experimental api removal looks got to you?

@Sunnatillo
Sunnatillo force-pushed the Sunnatillo/rm-unnecessary-conn-to-grpc branch from 236e128 to 199cf78 Compare July 21, 2026 21:19
@Sunnatillo

Copy link
Copy Markdown
Author

/cc @fuweid @serathius

@kubernetes-prow
kubernetes-prow Bot requested review from fuweid and serathius July 31, 2026 07:19
@ahrtr

ahrtr commented Aug 15, 2026

Copy link
Copy Markdown
Member

/ok-to-test

@codecov

codecov Bot commented Aug 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.74%. Comparing base (81e09b0) to head (37e8925).
⚠️ Report is 152 commits behind head on main.

Additional details and impacted files

see 46 files with indirect coverage changes

@@            Coverage Diff             @@
##             main   #22133      +/-   ##
==========================================
+ Coverage   69.66%   69.74%   +0.07%     
==========================================
  Files         449      449              
  Lines       38177    38191      +14     
==========================================
+ Hits        26597    26636      +39     
+ Misses      10148    10133      -15     
+ Partials     1432     1422      -10     

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 81e09b0...37e8925. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ahrtr

ahrtr commented Aug 15, 2026

Copy link
Copy Markdown
Member

/retest

@ahrtr

ahrtr commented Aug 15, 2026

Copy link
Copy Markdown
Member

@ahrtr ahrtr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your first contribution.

cc @fuweid @ivanvc @serathius

@kubernetes-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahrtr, Sunnatillo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Comment thread client/v3/internal/resolver/resolver.go Outdated
"google.golang.org/grpc/serviceconfig"
)

// fakeClientConn records every state passed to UpdateState so tests can assert

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we merge those tests first? Would like to understand what's the current state before we start changes.

@Sunnatillo Sunnatillo Aug 19, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests are written for changed behavior. It expects Build() emits one update. In old behaviour it emits two update.
Do you want me to add tests for old behavior and change the test in this PR?

If we add the test in current format to main branch first it will fail.

@serathius

Copy link
Copy Markdown
Member

Overall LGTM, left comments to remove AI slop comments and make change safer by merging tests first.

Build pushed the endpoints and round_robin ServiceConfig via a follow-up
updateState call after building the embedded manual.Resolver. gRPC saw an
initial state without the ServiceConfig, then a second update with it,
forcing a mid-connection balancer switch that canceled an in-flight dial
and logged spurious "operation was canceled" warnings on every client.

Seed the state via manual.Resolver.InitialState before Build so gRPC gets
one update with both endpoints and the ServiceConfig. Extract a shared
state() helper reused by updateState (SetEndpoints), and add resolver tests
that assert Build emits exactly one update carrying both endpoints and the
ServiceConfig. Assertions compare whole resolver.State values so no
experimental gRPC API is introduced.

Fixes etcd-io#21660

Signed-off-by: Sunnatillo <sunnat.samadov@est.tech>
@Sunnatillo
Sunnatillo force-pushed the Sunnatillo/rm-unnecessary-conn-to-grpc branch from 199cf78 to 37e8925 Compare August 19, 2026 07:02
@Sunnatillo

Copy link
Copy Markdown
Author

Test failures are unrelated to the changes

@ahrtr

ahrtr commented Aug 19, 2026

Copy link
Copy Markdown
Member

/retest

@ahrtr

ahrtr commented Aug 19, 2026

Copy link
Copy Markdown
Member

make change safer by merging tests first.

@Sunnatillo can you run the test cases without product code change locally and show us that how it fails; afterwards apply the patch in resolver.go and prove that it resolves the test failure?

Normally we add the test cases in the first commit, and add the patch in the second commit. (Personally I don't treat this as a hard requirement)

@kubernetes-prow

kubernetes-prow Bot commented Aug 19, 2026

Copy link
Copy Markdown

@Sunnatillo: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-etcd-e2e-amd64 37e8925 link true /test pull-etcd-e2e-amd64

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

gyuho added a commit to gyuho/etcd-infra that referenced this pull request Aug 24, 2026
The document is the measured evaluation of the leader-aware client balancer
(upstream proposal etcd-io/etcd#22268, base branch etcd-io/etcd#22133); the
name says what it is.
@kubernetes-prow

Copy link
Copy Markdown

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

EtcdManualResolver.Build() sends two resolver updates causing unnecessary gRPC balancer switch and wasted connections

3 participants