Support backend dependencies for EnvoyExtensionPolicy Dynamic Modules

[michalskalski]

Description:

Envoy HTTP Dynamic Modules can make HTTP callouts through named Envoy clusters. In Envoy Gateway, module specific configuration supplied through EnvoyExtensionPolicy is opaque to Envoy Gateway.

EnvoyExtensionPolicy has no supported way for a Dynamic Module to declare the backend it calls. The policy author and infrastructure operator must coordinate a cluster name in the opaque configuration and provision the corresponding cluster separately.

This works today, but Envoy Gateway cannot associate the backend with the policy. It therefore cannot resolve or validate the dependency, generate the required cluster, or report a missing or invalid dependency through policy status.

Use case

I encountered this while developing envoy-web-bot-auth. The module makes an HTTP callout to a resolver, and its opaque configuration contains the Envoy cluster name:

dynamicModule:
  - name: envoy-web-bot-auth
    filterName: web-bot-auth
    config:
      resolver:
        cluster: web-bot-auth-key-resolver

The module passes that value to Envoy's HTTP callout API:

https://github.com/michalskalski/envoy-web-bot-auth/blob/f446a83663af32a18c50d29b4bac14dae0a23057/crates/module/src/filter.rs#L218-L223

The resolver is deployed behind a Kubernetes Service, while the corresponding cluster is added separately through EnvoyProxy.spec.bootstrap:

https://github.com/michalskalski/envoy-web-bot-auth/blob/f446a83663af32a18c50d29b4bac14dae0a23057/examples/kind/overlays/external-resolver/kustomization.yaml#L7-L27

This is the only concrete use case I can currently provide, so I do not yet have evidence about how frequently other Dynamic Modules need similar backend dependencies.

However, the same issue may affect other HTTP Dynamic Modules. Any module that uses Envoy's callout API to contact a cluster not otherwise created by Envoy Gateway requires the operator to provision that cluster separately. Because Dynamic Module support in Envoy Gateway is relatively new, there may not yet be many public examples. This issue asks whether backend dependencies should become part of the supported integration model or remain operator managed.

Existing mechanisms

EnvoyProxy.spec.bootstrap can create the required cluster, but this requires writing low-level Envoy configuration and manually keeping its name consistent with opaque module configuration. Envoy Gateway documents that backward compatibility of this field not guaranteed across minor versions.

EnvoyPatchPolicy can add the same cluster to generated xDS, as demonstrated by #9711. It still relies on manual name coordination and is documented as unstable, with behavior that may change across versions

Envoy Gateway already uses backend references for features such as ExtProc. The BackendEndpoint API supports FQDN, IP, and Unix-domain-socket endpoints. However, Dynamic Module configuration has no equivalent way to reference one of these backends.

API considerations

Because DynamicModule.config is opaque to Envoy Gateway, a backend reference alone may not define the complete integration. Any API would also need to define how the module refers to the generated Envoy cluster without requiring Envoy Gateway to interpret module specific configuration or requiring users to depend on Envoy Gateway's internal cluster naming.

Questions for discussion may include:

• where the dependency should be declared: EnvoyExtensionPolicy, EnvoyProxy, or another resource
• how the module should identify the resulting cluster
• which backend types, reference authorization rules, and backend policies should apply.

Desired behavior

Provide a supported way to associate an EnvoyExtensionPolicy Dynamic Module instance with one or more backend dependencies.

Envoy Gateway should be able to resolve and validate those dependencies, generate the required Envoy clusters, and make them usable by the module through a documented contract. Module specific configuration should remain opaque, and generated xDS cluster names should remain an implementation detail.

This issue can be considered complete when Envoy Gateway provides a supported and documented contract through which Dynamic Modules can declare and use backend dependencies, including appropriate validation, status reporting, and test coverage.

Relevant links

• Developer Slack discussion
• Dynamic Modules documentation
• DynamicModule API reference
• Backend API documentation
• Previous BackendRef discussion for ExtAuth
• HTTP callout support for Dynamic Modules (Envoy PR #39151)
• EnvoyPatchPolicy cluster example (PR #9711)

[michalskalski]

I gave this a bit more thought. One possible approach would involve changes in both Envoy Gateway and Envoy Proxy. For example, a dynamic module could declare a backend dependency like this:

dynamicModule:
  - name: my-dynamic-module
    filterName: my-filter
    backends:
      - name: my-service
        backendRef:
          name: my-backend
          port: 8080
    config:
      external_service:
        backend: my-service

my-service is an alias module uses to refer to this backend. my-backend is the actual Kubernetes Service name. The module would look up this alias only in its own backend bindings. In this example, the module reads the alias from config.external_service.backend. Gateway would leave the configuration inside config opaque.

The flow could be:

1. Add support for named backend references to the DynamicModule API. (Gateway)

2. Validate the references and generate the required Envoy clusters configuration to reach the backends. (Gateway)

3. Add a backend_bindings field to Envoy's Dynamic Module filter configuration. Gateway would fill it with the mapping from each alias to its generated cluster, alongside the opaque module configuration. (Envoy + Gateway)

   filter_name: my-filter
   backend_bindings: # Proposed Envoy field
     my-service:
       cluster: generated-cluster-name
   filter_config:
     "@type": type.googleapis.com/google.protobuf.Struct
     external_service:
       backend: my-service

4. Add an SDK lookup, something like get_backend_cluster("my-service") which have to be backed by a new ABI callback. It would look up the alias in this filter's configuration and return the cluster name or indicate that it is missing. (Envoy)

5. Look up the alias during initialization and pass the returned cluster name to the existing send_http_callout API when making requests. (module)

[webdevsamran]

Hi team, I would like to investigate and work on this enhancement.

Currently, EnvoyExtensionPolicy does not allow Dynamic Modules to declare HTTP callout backend dependencies, requiring manual cluster coordination out of band. I can look into extending EnvoyExtensionPolicy to support declaring backend cluster references or backendRefs so Envoy Gateway can automatically provision and manage target clusters for dynamic modules, with tests and schema validation.

Could you please assign this to me? Thanks!