Skip to content

feat: expose gRPC access log buffering on the ALS and OpenTelemetry sinks #9725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
xiongzubiao wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: expose gRPC access log buffering on the ALS and OpenTelemetry sinks #9725

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 42 additions & 1 deletion api/v1alpha1/envoyproxy_accesslogging_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@

package v1alpha1

import gwapiv1 "sigs.k8s.io/gateway-api/apis/v1"
import (
"k8s.io/apimachinery/pkg/api/resource"
gwapiv1 "sigs.k8s.io/gateway-api/apis/v1"
)

type ProxyAccessLog struct {
// Disable disables access logging for managed proxies if set to true.
Expand Down Expand Up @@ -134,6 +137,36 @@ type ProxyAccessLogSink struct {
OpenTelemetry *OpenTelemetryEnvoyProxyAccessLog `json:"openTelemetry,omitempty"`
}

// GRPCAccessLogBufferSettings configures how Envoy buffers access log entries before
// flushing them to a gRPC access log sink.
//
// Entries accumulate until FlushInterval elapses or SizeBytes worth have been buffered,
// whichever comes first, at which point Envoy flushes. Entries are discarded, and counted
// by the sink's `logs_dropped` stat, only when a flush cannot drain the buffer. How much
// of a sink outage the buffer absorbs therefore depends on both settings, since only
// entries still buffered when the sink returns are delivered.
type GRPCAccessLogBufferSettings struct {
// FlushInterval defines how often buffered access log entries are flushed to the sink.
// Entries are flushed when this interval elapses or when SizeBytes worth of entries have
// been buffered, whichever comes first.
// Must be greater than 0. Defaults to 1s.
//
// +kubebuilder:validation:XValidation:rule="duration(self) > duration('0s')",message="flushInterval must be greater than 0"
// +optional
FlushInterval *gwapiv1.Duration `json:"flushInterval,omitempty"`
// SizeBytes defines the soft size limit of the access log entry buffer.
// For example, 20Mi, 1Gi, 256Ki etc.
// Note that when the suffix is not provided, the value is interpreted as bytes.
// Envoy carries this as a 32-bit unsigned integer, so values of 4Gi and above are
// rejected when the EnvoyProxy is translated rather than at admission.
// Defaults to 16384 bytes.
//
// +kubebuilder:validation:XIntOrString
// +kubebuilder:validation:Pattern="^[1-9]+[0-9]*([EPTGMK]i|[EPTGMk])?$"
// +optional
SizeBytes *resource.Quantity `json:"sizeBytes,omitempty"`
}

type ALSEnvoyProxyAccessLogType string

const (
Expand Down Expand Up @@ -169,6 +202,10 @@ type ALSEnvoyProxyAccessLog struct {
// HTTP defines additional configuration specific to HTTP access logs.
// +optional
HTTP *ALSEnvoyProxyHTTPAccessLogConfig `json:"http,omitempty"`
// Buffer defines how access log entries are buffered before being flushed to the
// access log service.
// +optional
Buffer *GRPCAccessLogBufferSettings `json:"buffer,omitempty"`

@zirain zirain Aug 28, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this be more generic to support more options in the future?

}

type ALSEnvoyProxyHTTPAccessLogConfig struct {
Expand Down Expand Up @@ -228,6 +265,10 @@ type OpenTelemetryEnvoyProxyAccessLog struct {
// +kubebuilder:validation:MinItems=1
// +kubebuilder:validation:MaxItems=32
Headers []gwapiv1.HTTPHeader `json:"headers,omitempty"`
// Buffer defines how access log entries are buffered before being flushed to the
// OpenTelemetry collector.
// +optional
Buffer *GRPCAccessLogBufferSettings `json:"buffer,omitempty"`

// TODO: support more OpenTelemetry accesslog options(e.g. TLS, auth etc.) in the future.
}
Expand Down
35 changes: 35 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

66 changes: 66 additions & 0 deletions charts/gateway-crds-helm/templates/generated/gateway.envoyproxy.io_envoyproxies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13153,6 +13153,39 @@ spec:
&& !(has(self.loadBalancer) && has(self.loadBalancer.type)
&& self.loadBalancer.type in [''Random'',
''RoundRobin'']))'
buffer:
description: |-
Buffer defines how access log entries are buffered before being flushed to the
access log service.
properties:
flushInterval:
description: |-
FlushInterval defines how often buffered access log entries are flushed to the sink.
Entries are flushed when this interval elapses or when SizeBytes worth of entries have
been buffered, whichever comes first.
Must be greater than 0. Defaults to 1s.
pattern: ^([0-9]{1,5}(h|m|s|ms)){1,4}$
type: string
x-kubernetes-validations:
- message: flushInterval must be greater
than 0
rule: duration(self) > duration('0s')
sizeBytes:
allOf:
- pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
- pattern: ^[1-9]+[0-9]*([EPTGMK]i|[EPTGMk])?$
anyOf:
- type: integer
- type: string
description: |-
SizeBytes defines the soft size limit of the access log entry buffer.
For example, 20Mi, 1Gi, 256Ki etc.
Note that when the suffix is not provided, the value is interpreted as bytes.
Envoy carries this as a 32-bit unsigned integer, so values of 4Gi and above are
rejected when the EnvoyProxy is translated rather than at admission.
Defaults to 16384 bytes.
x-kubernetes-int-or-string: true
type: object
http:
description: HTTP defines additional configuration
specific to HTTP access logs.
Expand Down Expand Up @@ -14847,6 +14880,39 @@ spec:
&& !(has(self.loadBalancer) && has(self.loadBalancer.type)
&& self.loadBalancer.type in [''Random'',
''RoundRobin'']))'
buffer:
description: |-
Buffer defines how access log entries are buffered before being flushed to the
OpenTelemetry collector.
properties:
flushInterval:
description: |-
FlushInterval defines how often buffered access log entries are flushed to the sink.
Entries are flushed when this interval elapses or when SizeBytes worth of entries have
been buffered, whichever comes first.
Must be greater than 0. Defaults to 1s.
pattern: ^([0-9]{1,5}(h|m|s|ms)){1,4}$
type: string
x-kubernetes-validations:
- message: flushInterval must be greater
than 0
rule: duration(self) > duration('0s')
sizeBytes:
allOf:
- pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
- pattern: ^[1-9]+[0-9]*([EPTGMK]i|[EPTGMk])?$
anyOf:
- type: integer
- type: string
description: |-
SizeBytes defines the soft size limit of the access log entry buffer.
For example, 20Mi, 1Gi, 256Ki etc.
Note that when the suffix is not provided, the value is interpreted as bytes.
Envoy carries this as a 32-bit unsigned integer, so values of 4Gi and above are
rejected when the EnvoyProxy is translated rather than at admission.
Defaults to 16384 bytes.
x-kubernetes-int-or-string: true
type: object
headers:
description: |-
Headers is a list of additional headers to send with OTLP export requests.
Expand Down
66 changes: 66 additions & 0 deletions charts/gateway-helm/charts/crds/crds/generated/gateway.envoyproxy.io_envoyproxies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13152,6 +13152,39 @@ spec:
&& !(has(self.loadBalancer) && has(self.loadBalancer.type)
&& self.loadBalancer.type in [''Random'',
''RoundRobin'']))'
buffer:
description: |-
Buffer defines how access log entries are buffered before being flushed to the
access log service.
properties:
flushInterval:
description: |-
FlushInterval defines how often buffered access log entries are flushed to the sink.
Entries are flushed when this interval elapses or when SizeBytes worth of entries have
been buffered, whichever comes first.
Must be greater than 0. Defaults to 1s.
pattern: ^([0-9]{1,5}(h|m|s|ms)){1,4}$
type: string
x-kubernetes-validations:
- message: flushInterval must be greater
than 0
rule: duration(self) > duration('0s')
sizeBytes:
allOf:
- pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
- pattern: ^[1-9]+[0-9]*([EPTGMK]i|[EPTGMk])?$
anyOf:
- type: integer
- type: string
description: |-
SizeBytes defines the soft size limit of the access log entry buffer.
For example, 20Mi, 1Gi, 256Ki etc.
Note that when the suffix is not provided, the value is interpreted as bytes.
Envoy carries this as a 32-bit unsigned integer, so values of 4Gi and above are
rejected when the EnvoyProxy is translated rather than at admission.
Defaults to 16384 bytes.
x-kubernetes-int-or-string: true
type: object
http:
description: HTTP defines additional configuration
specific to HTTP access logs.
Expand Down Expand Up @@ -14846,6 +14879,39 @@ spec:
&& !(has(self.loadBalancer) && has(self.loadBalancer.type)
&& self.loadBalancer.type in [''Random'',
''RoundRobin'']))'
buffer:
description: |-
Buffer defines how access log entries are buffered before being flushed to the
OpenTelemetry collector.
properties:
flushInterval:
description: |-
FlushInterval defines how often buffered access log entries are flushed to the sink.
Entries are flushed when this interval elapses or when SizeBytes worth of entries have
been buffered, whichever comes first.
Must be greater than 0. Defaults to 1s.
pattern: ^([0-9]{1,5}(h|m|s|ms)){1,4}$
type: string
x-kubernetes-validations:
- message: flushInterval must be greater
than 0
rule: duration(self) > duration('0s')
sizeBytes:
allOf:
- pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
- pattern: ^[1-9]+[0-9]*([EPTGMK]i|[EPTGMk])?$
anyOf:
- type: integer
- type: string
description: |-
SizeBytes defines the soft size limit of the access log entry buffer.
For example, 20Mi, 1Gi, 256Ki etc.
Note that when the suffix is not provided, the value is interpreted as bytes.
Envoy carries this as a 32-bit unsigned integer, so values of 4Gi and above are
rejected when the EnvoyProxy is translated rather than at admission.
Defaults to 16384 bytes.
x-kubernetes-int-or-string: true
type: object
headers:
description: |-
Headers is a list of additional headers to send with OTLP export requests.
Expand Down
Loading
Loading