Commit 284ce13
authored
feat(bigtable): BigtableDataClientFactory session support (#13829)
This PR extends the session-based API path to support
`BigtableDataClientFactory`.
Now the factory creates a single shared `ChannelPool` and
`ClientConfigurationManager` once and hands lightweight child clients a
reference to both.
The bulk of the implementation work is a rewrite of `ChannelPoolDpImpl`
to correctly handle multiple tenants (different instances/app profiles)
sharing a single channel pool.
### `ChannelPoolDpImpl`: New Placement Model
#### Old design
The old pool organized channels into `AfeChannelGroup` objects — one
deque of channels per observed AFE. When a channel's first session
revealed its AFE (via `onBeforeSessionStart`), the channel was rehomed
from the temporary `startingGroup` into the matching `AfeChannelGroup`,
creating one if it didn't exist. New streams were then placed by picking
the group with the fewest sessions.
This design had a problem for the multi-tenant factory case:
AFE affinity was tracked per channel, but for factory children different
tenants on the same channel can be routed to different AFEs by RLS. A
channel in group "AFE-7" might route tenant A to AFE-7 but tenant B
somewhere else entirely.
#### New design
The new pool replaces the nested group structure with a flat `channels`
list plus a `routeObservations` map keyed on `(channelId, tenantKey) →
lastObservedAfeId`.
##### Data structures:
- `channels: List<ChannelWrapper>` — all channels in ACTIVE or DRAINING
state
- `routeObservations: Map<RouteKey, AfeId>` — the last AFE seen for a
given (channel, tenant) pair, populated on every `onBeforeSessionStart`
and self-healing as RLS routes drift over time
- `sessionsPerAfeId: Multiset<AfeId>` — global count of live sessions
per AFE, used to find underloaded targets
- `TenantKey` (new value type) — stamped onto `CallOptions` by
`TableBase` so the pool can distinguish tenants within the same shared
pool
##### Placement (`newStream`):
1. Capacity mode — find the AFE with the smallest session count that is
still below `softMaxPerGroup`:
- 3a — Preferred: pick the ACTIVE channel whose last observed route for
this tenant was that AFE, choosing the least-loaded among candidates.
This is the steady-state path: RLS is stable per (channel, tenant), so
repeat sessions for the same tenant land on the same AFE.
- 3b — Unobserved-tenant fallback: if no route observation exists for
this tenant on any channel, pick any ACTIVE channel with remaining
capacity (least-loaded). The actual AFE is unknown until
`onBeforeSessionStart` fires; `routeObservations` is updated then for
future placements.
2. Diversity mode — entered when all known AFEs are at cap, or the pool
is empty: prefer an ACTIVE channel with fewer than `softMaxPerGroup / 2`
outstanding streams. If none exists, open a new channel and absorb
whichever AFE it lands on.
##### Scale-in (DRAINING state):
The old "thinning" logic would call `channel.shutdown()` during
serviceChannels(). The new approach introduces a two-state channel
lifecycle (`ACTIVE` / `DRAINING`). When `serviceChannels()` decides to
shrink the pool it marks excess channels DRAINING — they stop accepting
new streams but continue serving existing ones. A DRAINING channel is
removed only when its `numOutstanding` reaches zero (in
`releaseChannel`). Drain candidates are chosen by `pickDrainCandidates`,
which prefers idle channels first, then least-recently-used, then
oldest.
##### Route observation cleanup:
`removeChannel` purges all `routeObservations` entries for the removed
channel's ID, preventing the map from growing unboundedly as channels
cycle.
### Other changes
#### `BigtableDataClientFactory` / `BigtableClientContext`:
- `BigtableDataClientFactory.create()` now calls
`BigtableClientContext.createForFactory()` (a new entry point) instead
of manually disabling sessions. The factory context initializes a shared
`ShimImpl` (channel pool + config manager) that is reused by all
children.
- `BigtableClientContext.createChild()` now creates a lightweight
`ShimImpl` child via `ShimImpl.createForFactoryChild()` rather than
sharing a `DisabledShim`.
#### `ShimImpl` / `Client`:
- `Client.channelPool` changed from a bare `ChannelPool` reference to
`Resource<ChannelPool>` so the factory-child constructor can hold a
shared (non-closing) reference and the standard constructor holds an
owned one.
- `ShimImpl.configManager` similarly wrapped in `Resource<>` so
`close()` is a no-op for factory children (the parent owns the manager's
lifecycle).1 parent ccd13eb commit 284ce13
9 files changed
Lines changed: 463 additions & 238 deletions
File tree
- java-bigtable/google-cloud-bigtable/src
- main/java/com/google/cloud/bigtable/data/v2
- internal
- api
- channels
- compat
- stub
- test/java/com/google/cloud/bigtable/data/v2/internal/channels
Lines changed: 0 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | 77 | | |
82 | 78 | | |
83 | 79 | | |
| |||
Lines changed: 54 additions & 10 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
100 | 100 | | |
101 | 101 | | |
102 | 102 | | |
103 | | - | |
| 103 | + | |
104 | 104 | | |
105 | 105 | | |
106 | 106 | | |
| |||
185 | 185 | | |
186 | 186 | | |
187 | 187 | | |
188 | | - | |
189 | 188 | | |
190 | 189 | | |
191 | 190 | | |
192 | 191 | | |
193 | | - | |
| 192 | + | |
| 193 | + | |
194 | 194 | | |
195 | 195 | | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
196 | 200 | | |
197 | 201 | | |
198 | 202 | | |
199 | | - | |
200 | 203 | | |
201 | 204 | | |
202 | 205 | | |
203 | | - | |
| 206 | + | |
| 207 | + | |
204 | 208 | | |
205 | 209 | | |
206 | 210 | | |
| |||
224 | 228 | | |
225 | 229 | | |
226 | 230 | | |
227 | | - | |
| 231 | + | |
228 | 232 | | |
229 | 233 | | |
230 | 234 | | |
231 | 235 | | |
232 | 236 | | |
233 | | - | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
234 | 263 | | |
235 | 264 | | |
236 | 265 | | |
| |||
329 | 358 | | |
330 | 359 | | |
331 | 360 | | |
332 | | - | |
| 361 | + | |
333 | 362 | | |
334 | 363 | | |
335 | 364 | | |
| |||
353 | 382 | | |
354 | 383 | | |
355 | 384 | | |
356 | | - | |
| 385 | + | |
357 | 386 | | |
358 | 387 | | |
359 | 388 | | |
| |||
378 | 407 | | |
379 | 408 | | |
380 | 409 | | |
381 | | - | |
| 410 | + | |
382 | 411 | | |
383 | 412 | | |
384 | 413 | | |
| |||
391 | 420 | | |
392 | 421 | | |
393 | 422 | | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
394 | 438 | | |
395 | 439 | | |
396 | 440 | | |
| |||
Lines changed: 9 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
| 26 | + | |
25 | 27 | | |
26 | 28 | | |
27 | 29 | | |
| |||
66 | 68 | | |
67 | 69 | | |
68 | 70 | | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
69 | 77 | | |
70 | 78 | | |
71 | 79 | | |
72 | 80 | | |
73 | 81 | | |
74 | 82 | | |
75 | 83 | | |
76 | | - | |
| 84 | + | |
77 | 85 | | |
78 | 86 | | |
79 | 87 | | |
| |||
0 commit comments