From 88b66c15b578020b806d3fe4c92bde8a60c3fac5 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Tue, 4 Aug 2026 10:09:51 +0200 Subject: [PATCH 1/2] docs: pipeline: filters: expect: document per-record batch behavior The filter now validates every record in a batch instead of stopping at the first mismatch, which changes what users observe for two actions: warn logs one warning per failing record, and result_key marks each record with its own result instead of applying a single batch-wide result. Group marker records are also passed through untouched. - Add a Batch behavior section describing what warn, exit, and result_key each do across a batch. - Clarify in the action row that warn logs per failing record, and link to the new section. Signed-off-by: Eric D. Schabell --- pipeline/filters/expect.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/pipeline/filters/expect.md b/pipeline/filters/expect.md index 8f3abbd31..da693ea93 100644 --- a/pipeline/filters/expect.md +++ b/pipeline/filters/expect.md @@ -18,10 +18,20 @@ The plugin supports the following configuration parameters: | Key | Description | Default | | :--- | :--- | :--- | -| `action` | Action to take when a rule doesn't match. Available options are `warn`, `exit` or `result_key`. On `warn`, a warning message is sent to the logging layer when a mismatch of the `key*` rules is found. Using `exit` makes Fluent Bit exit with status code `255`. `result_key` adds a matching result to each record. | `warn` | +| `action` | Action to take when a rule doesn't match. Available options are `warn`, `exit` or `result_key`. On `warn`, a warning message is sent to the logging layer for each record that fails a `key*` rule. Using `exit` makes Fluent Bit exit with status code `255`. `result_key` adds a matching result to each record. For how each action treats a batch of records, see [Batch behavior](#batch-behavior). | `warn` | | `key_exists` | Check if a key with a given name exists in the record. | _none_ | | `key_not_exists` | Check if a key doesn't exist in the record. | _none_ | | `key_val_eq` | Check that the value of the key equals the given value in the configuration. | _none_ | | `key_val_is_not_null` | Check that the value of the key is `NOT NULL`. | _none_ | | `key_val_is_null` | Check that the value of the key is `NULL`. | _none_ | | `result_key` | Specify a key name for the matching result added when `action` is set to `result_key`. | `matched` | + +## Batch behavior + +Fluent Bit delivers records to filters in batches. The expect filter evaluates every record in a batch against your rules, and the configured `action` determines what happens next: + +- `warn`: each record that fails a rule logs its own `expect check failed` warning. A batch that contains several failing records produces several warnings. +- `exit`: Fluent Bit stops at the first record that fails a rule and exits with status code `255`. The remaining records in the batch aren't evaluated. +- `result_key`: each record gets its own result. A record that passes every rule is marked `true`, and a record that fails any rule is marked `false`, so one failing record doesn't change the result recorded for the other records in the same batch. + +Some event types, such as OpenTelemetry logs, include internal marker records that delimit a group of records. The expect filter passes these markers through unchanged and doesn't add the result key to them. From f24419723c1d48b593eb5d647fd4c39521771501 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Tue, 4 Aug 2026 16:07:42 +0200 Subject: [PATCH 2/2] docs: docs: pipeline: filters: expect: clarify batch behavior overview - Reword the batch overview to say "data records" instead of "every record", since marker records in the batch are never evaluated - State in the overview that `exit` stops evaluation at the first failing record instead of continuing through the batch - State in the overview that `result_key` never adds a result to marker records Signed-off-by: Eric D. Schabell --- pipeline/filters/expect.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/filters/expect.md b/pipeline/filters/expect.md index da693ea93..a6c056efd 100644 --- a/pipeline/filters/expect.md +++ b/pipeline/filters/expect.md @@ -28,7 +28,7 @@ The plugin supports the following configuration parameters: ## Batch behavior -Fluent Bit delivers records to filters in batches. The expect filter evaluates every record in a batch against your rules, and the configured `action` determines what happens next: +Fluent Bit delivers records to filters in batches. The expect filter evaluates each data record in a batch against your rules, and the configured `action` determines what happens next. Two exceptions apply across the batch: `exit` stops evaluation at the first record that fails a rule, and `result_key` never adds a result to marker records. - `warn`: each record that fails a rule logs its own `expect check failed` warning. A batch that contains several failing records produces several warnings. - `exit`: Fluent Bit stops at the first record that fails a rule and exits with status code `255`. The remaining records in the batch aren't evaluated.