Current Behavior
When an HTTPRoute's parentRefs are repointed from a Gateway managed by controller A to a
Gateway managed by controller B (a different GatewayClass.spec.controllerName), controller B
picks the route up correctly — but controller A never removes the configuration it had already
pushed. Its data plane keeps serving the route indefinitely.
The chain, on master (be4aaef) and on the released 2.1.0:
-
httproute_controller.go:75 — the reconciler is registered with
For(&gatewayv1.HTTPRoute{}) and no controller-name predicate, so every controller
instance watches every HTTPRoute. Repointing parentRefs bumps the generation, so
GenerationChangedPredicate passes and controller A does reconcile the change.
-
utils.go:357-359 — ParseRouteParentRefs skips parentRefs whose GatewayClass belongs to
another controller:
if string(gatewayClass.Spec.ControllerName) != config.ControllerConfig.ControllerName {
continue
}
For controller A the route's only remaining parentRef is now controller B's Gateway, so the
returned slice is empty.
-
httproute_controller.go:172-174 — the empty slice returns early with no cleanup:
if len(gateways) == 0 {
return ctrl.Result{}, nil
}
Provider.Delete is only reachable from the IgnoreNotFound branch at :147, i.e. when the
object is deleted from the cluster. There is no path for "this object used to be mine and no
longer is".
-
adc/client/client.go:290,303 — the periodic sync replays the in-memory store
(ConfigManager.List() → GetResources(name)); it never re-derives state from Kubernetes.
The orphaned service therefore gets re-pushed to the data plane every sync period, forever.
Restarting the controller pod clears it, because startup rebuilds the store from the informer's
initial list and the route no longer produces an entry for controller A.
The same early return exists in all five Gateway API route reconcilers:
httproute_controller.go:172
grpcroute_controller.go:190
tcproute_controller.go:283
udproute_controller.go:283
tlsroute_controller.go:283
Why this is hard to notice
Every Kubernetes-level signal is correct and green while it happens:
- the HTTPRoute's
status.parents lists only controller B, with Accepted=True
- controller B's Gateway shows the route in
status.listeners[].attachedRoutes
- controller A's Gateway shows the correct (reduced)
attachedRoutes count
- controller A logs no errors — its sync loop reports success every period
- GitOps tooling reports the manifests as fully synced
The stale configuration is visible only in the old data plane's own control API
(:9090/v1/services). In our case the orphaned routes carried public hostnames, so this failed
open: hosts that were supposed to have been taken off the public gateway stayed publicly
reachable for over 21 hours, with the label metadata and modifiedIndex still showing values
from weeks before the change.
Prior art
PR #2543 ("fix: residual data issue when updating ingressClassName", merged 2025-09-08) fixed
exactly this failure mode for Ingress, and its description states the problem in general terms:
the controller simply ignores the Ingress resource. However, this behavior is problematic: the
corresponding configuration is not removed and remains in the data plane, which may result in
incorrect routing or stale configurations.
The resulting code in ingress_controller.go:159-166 calls Provider.Delete when the
IngressClass no longer matches. The Gateway API route reconcilers were never given the
equivalent.
Expected Behavior
When a route no longer references any Gateway managed by this controller, the controller removes
the configuration it previously pushed for that route, so the data plane stops serving it —
matching the Ingress/IngressClass behaviour from #2543.
Error Logs
None. That is a significant part of the problem: the controller logs no error at any log level,
and the ADC sync reports success on every tick while re-pushing the orphaned configuration.
Steps to Reproduce
-
Install two APISIX ingress controller instances with distinct controller names, e.g.
apisix.apache.org/apisix-ingress-controller and
apisix.apache.org/apisix-ingress-controller-internal, each with its own GatewayClass and
Gateway (gateway-a in namespace apisix, gateway-b in namespace apisix-internal).
-
Create an HTTPRoute with parentRefs: [gateway-a] and a hostname, and confirm it serves:
$ curl -H 'Host: demo.example.com' http://<gateway-a>/
200
-
Change only the parentRefs to gateway-b (keep the same name/namespace) and apply.
-
Observe that controller B accepts the route and gateway-b serves it — and that gateway-a
still serves it too:
$ curl -H 'Host: demo.example.com' http://<gateway-a>/
200 # expected: 404
-
Confirm the orphan is in controller A's data plane, and that it is never removed:
$ kubectl port-forward -n apisix pod/<apisix-pod> 9090:9090
$ curl -s localhost:9090/v1/services | jq -r '.[].value | "\(.labels["k8s/name"]) \(.hosts)"'
demo ["demo.example.com"] # still present, indefinitely
-
kubectl rollout restart deploy/apisix-ingress-controller -n apisix — the entry disappears,
confirming the store is the only thing holding it.
The same reproduction applies to GRPCRoute, TCPRoute, UDPRoute and TLSRoute.
Environment
- APISIX Ingress controller version:
2.1.0 (also reproduces on master, be4aaef)
- ADC version:
0.26.0
- Mode: standalone / ADC (
config_provider: yaml)
- Kubernetes cluster version:
v1.34.9
- Gateway API version:
v1.3.0
Current Behavior
When an HTTPRoute's
parentRefsare repointed from a Gateway managed by controller A to aGateway managed by controller B (a different
GatewayClass.spec.controllerName), controller Bpicks the route up correctly — but controller A never removes the configuration it had already
pushed. Its data plane keeps serving the route indefinitely.
The chain, on
master(be4aaef) and on the released2.1.0:httproute_controller.go:75— the reconciler is registered withFor(&gatewayv1.HTTPRoute{})and no controller-name predicate, so every controllerinstance watches every HTTPRoute. Repointing
parentRefsbumps the generation, soGenerationChangedPredicatepasses and controller A does reconcile the change.utils.go:357-359—ParseRouteParentRefsskips parentRefs whose GatewayClass belongs toanother controller:
For controller A the route's only remaining parentRef is now controller B's Gateway, so the
returned slice is empty.
httproute_controller.go:172-174— the empty slice returns early with no cleanup:Provider.Deleteis only reachable from theIgnoreNotFoundbranch at :147, i.e. when theobject is deleted from the cluster. There is no path for "this object used to be mine and no
longer is".
adc/client/client.go:290,303— the periodic sync replays the in-memory store(
ConfigManager.List()→GetResources(name)); it never re-derives state from Kubernetes.The orphaned service therefore gets re-pushed to the data plane every sync period, forever.
Restarting the controller pod clears it, because startup rebuilds the store from the informer's
initial list and the route no longer produces an entry for controller A.
The same early return exists in all five Gateway API route reconcilers:
Why this is hard to notice
Every Kubernetes-level signal is correct and green while it happens:
status.parentslists only controller B, withAccepted=Truestatus.listeners[].attachedRoutesattachedRoutescountThe stale configuration is visible only in the old data plane's own control API
(
:9090/v1/services). In our case the orphaned routes carried public hostnames, so this failedopen: hosts that were supposed to have been taken off the public gateway stayed publicly
reachable for over 21 hours, with the label metadata and
modifiedIndexstill showing valuesfrom weeks before the change.
Prior art
PR #2543 ("fix: residual data issue when updating ingressClassName", merged 2025-09-08) fixed
exactly this failure mode for Ingress, and its description states the problem in general terms:
The resulting code in
ingress_controller.go:159-166callsProvider.Deletewhen theIngressClass no longer matches. The Gateway API route reconcilers were never given the
equivalent.
Expected Behavior
When a route no longer references any Gateway managed by this controller, the controller removes
the configuration it previously pushed for that route, so the data plane stops serving it —
matching the Ingress/IngressClass behaviour from #2543.
Error Logs
None. That is a significant part of the problem: the controller logs no error at any log level,
and the ADC sync reports success on every tick while re-pushing the orphaned configuration.
Steps to Reproduce
Install two APISIX ingress controller instances with distinct controller names, e.g.
apisix.apache.org/apisix-ingress-controllerandapisix.apache.org/apisix-ingress-controller-internal, each with its own GatewayClass andGateway (
gateway-ain namespaceapisix,gateway-bin namespaceapisix-internal).Create an HTTPRoute with
parentRefs: [gateway-a]and a hostname, and confirm it serves:Change only the
parentRefstogateway-b(keep the same name/namespace) and apply.Observe that controller B accepts the route and
gateway-bserves it — and thatgateway-astill serves it too:
Confirm the orphan is in controller A's data plane, and that it is never removed:
kubectl rollout restart deploy/apisix-ingress-controller -n apisix— the entry disappears,confirming the store is the only thing holding it.
The same reproduction applies to GRPCRoute, TCPRoute, UDPRoute and TLSRoute.
Environment
2.1.0(also reproduces onmaster,be4aaef)0.26.0config_provider: yaml)v1.34.9v1.3.0