Endpoint fast path: propagate endpoint updates without waiting for full translation (EDS-only pushes)

[zhaohuabing]

Description

In large clusters, endpoint updates are delayed by however long a full translation takes, because endpoints only reach Envoy by riding along in a complete rebuild of everything. When translations are slow, Envoy routes to stale endpoints for the entire build duration, which can cause availability loss.

Current behavior

Every EndpointSlice event takes the full pipeline:

1. Provider: full reconcile rebuilds the entire resource tree for the GatewayClass.
2. Gateway API runner: full translation of all resources to IR.
3. xDS runner: full translation of the IR to all xDS resource types, then a new snapshot.

While a build is in flight, watchable coalescing merges incoming updates into the next build — so endpoints continuously lag by roughly one full build duration.

At large scale (tens of thousands of routes, many EnvoyPatchPolicy resources with expensive JSON patches), a single full translation can take multiple seconds to tens of seconds. Under heavy churn, builds run back-to-back, so endpoint staleness is sustained rather than episodic.

Why this needs decoupling, not just optimization

The main reason is update frequency: in a large deployment with constant pod churn (rollouts, autoscaling, node drains), endpoint updates are multiple orders of magnitude more frequent than any other input — endpoints change at pod timescale while routes and policies change at human timescale. Today the highest-frequency, cheapest-to-compute signal flows through the lowest-frequency, most expensive machinery: every endpoint change pays the cost of a full translation, and under sustained churn that cost is paid continuously.

Decoupling also isolates faults: the main loop can be degraded by expensive or invalid configuration — very large route counts, costly EnvoyPatchPolicy JSON patches, broken references re-resolved every cycle. In a multi-tenant setup, one tenant's config delays endpoint delivery for every tenant sharing the control plane. Optimizing translation shrinks the staleness window, but only decoupling removes the failure class.

Proposal

An opt-in endpoint fast path: EndpointSlice changes are published on a dedicated channel and translated into updated ClusterLoadAssignments using a cached per-cluster endpoint context from the last successful full build (destination shape, health check config, zone/locality settings), then patched into the current snapshot with only the EDS version bumped. Semantics are best-effort: fresh endpoints against last-known-good config; full builds remain authoritative and re-apply the latest endpoint state on publish.

Cases where an endpoint change affects config-owned behavior escalate to the full path (e.g., zero↔non-zero endpoint transitions, clusters targeted by EnvoyPatchPolicy CLA patches or extension-server endpoint hooks).

Prior art: Istio's incremental (EDS-only) push — endpoint updates bypass PushContext recomputation, regenerate only affected CLAs, and escalate to a full push when cluster-level config is affected (DiscoveryServer.EDSUpdate, EndpointIndex.UpdateServiceEndpoints).

Relationship to update debouncing #9803: complementary — debouncing bounds push frequency; this bounds endpoint staleness. The fast-path channel would use the same debounce machinery with shorter intervals.

Alternatives considered

• Optimizing translation only — worthwhile independently, but leaves endpoint freshness coupled to worst-case config cost.

[zhaohuabing]

Design doc: https://docs.google.com/document/d/1r_-k6pu0eptbYucMgbilol91-0DmoekMf3MNums1luM/edit?usp=sharing

[arkodg]

hey thanks for raising this issue
for v1.9 can we try improving propagation time by splitting by base translation and policy translation

gateway/internal/gatewayapi/translator.go

Line 423 in 3b80726

, so we can push twice from the gateway api translator without flapping the policy

for future releases and also for ^, to make a better decision, some data points on how much time each layer is taking (via the tracing feature) would help decide where to invest the cycles

[zhaohuabing]

for v1.9 can we try improving propagation time by splitting by base translation and policy translation

Want to make sure I'm reading this right: the idea is to publish the xdsIR twice per cycle from the gateway api runner — once after base translation (listeners/routes/clusters/endpoints, everything above L423), then again once the policies are processed? Let me know if I've got that wrong.

Assuming that's the shape, a few things I ran into while looking at it:

• The first push isn't safe to serve. A base-only IR has no SecurityPolicy on it, so for the window between the two pushes a route is exposed without authn/authz — and the same for rate limits and other policy-enforced limits. Carrying the previous cycle's policy state avoids the flapping case, but not a newly added route: there's no prior state to carry, and the first push would expose it unauthenticated until the second lands.
• The seam isn't clean. BuildBTPIndexes and BuildCTPClusterSettingsIndex run above L423 and route processing consumes them (routing type, cluster settings, consistent-hash LB), so the "base" half already walks the policies and depends on them for cluster shape.
• Two publishes means two full xDS translations per cycle, including processJSONPatches for EnvoyPatchPolicy — one of the expensive things this issue is about.

for future releases and also for ^, to make a better decision, some data points on how much time each layer is taking (via the tracing feature) would help decide where to invest the cycles

#9672 landed the per-phase spans. It should show how the time splits between base vs policy translation, and how much sits in xDS translation.