contrib: add per-websocket-connection frame rate limit filter

[Amila-Rukshan]

Related: #28724

Adds envoy.filters.http.ws_local_ratelimit, a contrib HTTP filter that rate limits WebSocket data frames (text/binary/continuation) per WebSocket session using a local (non-shared) token bucket, with no descriptor matching.

Changes

• One token bucket per WebSocket session, tracked via StreamInfo::FilterState at LifeSpan::Request. This matters specifically for HTTP/2 and HTTP/3 downstream connections: multiple independent WebSocket sessions can be multiplexed as separate streams over one physical connection (RFC 8441 / RFC 9220 extended CONNECT), and each must get its own bucket rather than sharing one across the whole connection.
• Optional rejection_message: sent back to the client as a WebSocket text frame when a frame is rejected; if unset, the frame is silently dropped.
• Uses the unified HTTP filter factory pattern (Common::UnifiedFactoryBase), matching every other contrib HTTP filter.

Testing

• Unit tests: contrib/ws_local_ratelimit/filters/http/test:ws_local_ratelimit_filter_test, :config_test.
• Integration tests: :ws_local_ratelimit_integration_test — HTTP/1.1, HTTP/2, and HTTP/3 downstream, including multi-stream multiplexing over one connection for H2/H3.
• Manual verification against a real envoy-static binary with Go clients (golang.org/x/net/http2, quic-go/http3) confirming single-connection multiplexing and correct per-session rate limiting over the real wire protocol.

ws rate limit config example

envoy config

admin:
  address:
    socket_address:
      protocol: TCP
      address: 127.0.0.1
      port_value: 9902
  allow_paths:
  - exact: /ready
  - prefix: /stats
  - prefix: /

static_resources:
  listeners:

  # TCP listener for HTTPS / HTTP2. WebSocket sessions are bootstrapped over HTTP/2 via RFC 8441
  # Extended CONNECT (allow_connect), so several independent WebSocket sessions can be
  # multiplexed as separate streams over ONE downstream connection here.
  - name: http2_listener

    address:
      socket_address:
        address: 0.0.0.0
        port_value: 10000
        protocol: TCP

    filter_chains:
    - transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
          common_tls_context:
            alpn_protocols:
            - h2
            - http/1.1
            tls_certificates:
            - certificate_chain:
                filename: certs/dev-diary-p-256/archive/developerdiary.me/fullchain1.pem
              private_key:
                filename: certs/dev-diary-p-256/archive/developerdiary.me/privkey1.pem

      filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager

          codec_type: AUTO
          stat_prefix: ingress_ws_h2

          http2_protocol_options:
            allow_connect: true

          upgrade_configs:
          - upgrade_type: websocket

          route_config:
            name: local_route
            virtual_hosts:
            - name: app
              domains:
              - "*"

              # Tell the browser that HTTP/3 is also available on this same port.
              response_headers_to_add:
              - header:
                  key: alt-svc
                  value: 'h3=":10001"; ma=86400'

              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: service_ws

          http_filters:
          - name: envoy.filters.http.ws_local_ratelimit
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.ws_local_ratelimit.v3alpha.WsLocalRateLimit
              stat_prefix: ws_local_rate_limiter
              rejection_message: "{ error: 'rate limit exceeded' }"
              token_bucket:
                max_tokens: 5
                tokens_per_fill: 5
                fill_interval: 10s
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

  # UDP listener for QUIC / HTTP3. Same idea as the HTTP/2 listener above, but WebSocket
  # multiplexing here uses the (still work-in-progress in Envoy) HTTP/3 extended CONNECT
  # mechanism, gated by allow_extended_connect. Browser support for WebSocket-over-HTTP/3 is
  # inconsistent, so treat this as a best-effort/experimental check, not the primary one.
  - name: http3_listener

    address:
      socket_address:
        address: 0.0.0.0
        port_value: 10000
        protocol: UDP

    udp_listener_config:
      quic_options: {}

    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager

          codec_type: HTTP3
          stat_prefix: ingress_ws_h3

          http3_protocol_options:
            allow_extended_connect: true

          upgrade_configs:
          - upgrade_type: websocket

          route_config:
            name: local_route
            virtual_hosts:
            - name: app
              domains:
              - "*"

              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: service_ws

          http_filters:
          - name: envoy.filters.http.ws_local_ratelimit
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.ws_local_ratelimit.v3alpha.WsLocalRateLimit
              stat_prefix: ws_local_rate_limiter
              rejection_message: "{ error: 'rate limit exceeded' }"
              token_bucket:
                max_tokens: 2
                tokens_per_fill: 2
                fill_interval: 5s
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

      transport_socket:
        name: envoy.transport_sockets.quic
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.quic.v3.QuicDownstreamTransport
          downstream_tls_context:
            common_tls_context:
              tls_certificates:
              - certificate_chain:
                  filename: certs/dev-diary-p-256/archive/developerdiary.me/fullchain1.pem
                private_key:
                  filename: certs/dev-diary-p-256/archive/developerdiary.me/privkey1.pem

  clusters:
  - name: service_ws
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: service_ws
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.1
                port_value: 18088

[mathetake]

what's the motivation here? can't you just maintain it as a dynamic module with Rust SDK, no?

[Amila-Rukshan]

@mathetake, wouldn’t the dynamic module still require the Envoy user to implement the WebSocket codec themselves? This filter is being added as a contrib extension for the general use case, as suggested #28724 (comment). The WebSocket codec was originally added to support frame-level limiting, as discussed #13877 (comment). The related link in the issue description will connect all discussions that have happened previously.

[mathetake]

I am sorry but it seems like you are the only person who is likely to use/maintain. One criteria we have for contrib is that it at least needs to be useful fo/interesting to other end users. Given the lack of interests (the original issue is a few years ago and no one commented on it) and lack of end users requesting for this exact feature, i would recommend you maintain the code by yourself rather than putting a burden on maintainer

[Amila-Rukshan]

I understand your call on this as a maintainer. Thanks for the clarification!