From 19482bcde9a4e4823403abf92ece52134972a8e1 Mon Sep 17 00:00:00 2001 From: Lynette Miles Date: Wed, 2 Jul 2025 14:04:36 -0700 Subject: [PATCH 1/2] Pipeline: input: win event WEM: style Signed-off-by: Lynette Miles --- pipeline/inputs/windows-exporter-metrics.md | 148 ++++++++++---------- vale-styles/FluentBit/Acronyms.yml | 2 + 2 files changed, 76 insertions(+), 74 deletions(-) diff --git a/pipeline/inputs/windows-exporter-metrics.md b/pipeline/inputs/windows-exporter-metrics.md index f1b5ed126..28e6937e7 100644 --- a/pipeline/inputs/windows-exporter-metrics.md +++ b/pipeline/inputs/windows-exporter-metrics.md @@ -1,79 +1,77 @@ --- -description: >- - A plugin based on Prometheus Windows Exporter to collect system / host level +description: A plugin based on Prometheus Windows Exporter to collect system and host level metrics --- -# Windows Exporter Metrics +# Windows Exporter metrics -[Prometheus Windows Exporter](https://github.com/prometheus-community/windows_exporter) is a popular way to collect system level metrics from microsoft windows, such as CPU / Disk / Network / Process statistics. Fluent Bit 1.9.0 includes windows exporter metrics plugin that builds off the Prometheus design to collect system level metrics without having to manage two separate processes or agents. +[Prometheus Windows Exporter](https://github.com/prometheus-community/windows_exporter) is a popular way to collect system level metrics from Microsoft Windows, such as CPU, Disk, Network, and Process statistics. Fluent Bit 1.9.0 and later includes the Windows Exporter metrics plugin that builds off the Prometheus design to collect system level metrics without having to manage two separate processes or agents. -The initial release of Windows Exporter Metrics contains a single collector available from Prometheus Windows Exporter and we plan to expand it over time. - -**Important note:** Metrics collected with Windows Exporter Metrics flow through a separate pipeline from logs and current filters do not operate on top of metrics. +The initial release of Windows Exporter metrics contains a single collector available from Prometheus Windows Exporter. +Metrics collected with Windows Exporter metrics flow through a separate pipeline from logs and current filters don't operate on top of metrics. ## Configuration | Key | Description | Default | | --------------- | ---------------------------------------------------------------------- | --------- | -| scrape\_interval | The rate at which metrics are collected from the host operating system | 5 seconds | -| we.logical\_disk.allow\_disk\_regex | Specify the regex for logical disk metrics to allow collection of. Collect all by default. | "/.+/" | -| we.logical\_disk.deny\_disk\_regex | Specify the regex for logical disk metrics to prevent collection of/ignore. Allow all by default. | `NULL` | -|we.net.allow\_nic\_regex | Specify the regex for network metrics captured by the name of the NIC, by default captures all NICs but to exclude adjust the regex. | "/.+/" | -| we.service.where | Specify the WHERE clause for retrieving service metrics. | `NULL` | -| we.service.include | Specify the key value pairs for the include condition for the WHERE clause of service metrics. | `NULL` | -| we.service.exclude | Specify the key value pairs for the exclude condition for the WHERE clause of service metrics. | `NULL` | -| we.process.allow\_process\_regex | Specify the regex covering the process metrics to collect. Collect all by default. | "/.+/" | -| we.process.deny\_process\_regex | Specify the regex for process metrics to prevent collection of/ignore. Allow all by default. | `NULL` | -| collector.cpu.scrape\_interval | The rate in seconds at which cpu metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.net.scrape\_interval | The rate in seconds at which net metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.logical_disk.scrape\_interval | The rate in seconds at which logical\_disk metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.cs.scrape\_interval | The rate in seconds at which cs metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.os.scrape\_interval | The rate in seconds at which os metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.thermalzone.scrape\_interval | The rate in seconds at which thermalzone metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used.| 0 seconds | -| collector.cpu\_info.scrape\_interval | The rate in seconds at which cpu\_info metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.logon.scrape\_interval | The rate in seconds at which logon metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.system.scrape\_interval | The rate in seconds at which system metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.service.scrape\_interval | The rate in seconds at which service metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.memory.scrape\_interval | The rate in seconds at which memory metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.paging_file.scrape\_interval | The rate in seconds at which paging_file metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| collector.process.scrape\_interval | The rate in seconds at which process metrics are collected from the host operating system. If a value greater than 0 is used then it overrides the global default otherwise the global default is used. | 0 seconds | -| metrics | To specify which metrics are collected from the host operating system. | `"cpu,cpu_info,os,net,logical_disk,cs,thermalzone,logon,system,service"` | +| `scrape_interval` | The rate at which metrics are collected. | `5 seconds` | +| `we.logical_disk.allow_disk_regex` | Specify the regular expression for logical disk metrics to allow collection of. | `"/.+/"` (all) | +| `we.logical_disk.deny_disk_regex` | Specify the regular expression for logical disk metrics to prevent collection of or ignore. | `NULL` (all) | +| `we.net.allow_nic_regex` | Specify the regular expression for network metrics captured by the name of the NIC | `"/.+/"` (all) | +| `we.service.where` | Specify the `WHERE` clause for retrieving service metrics. | `NULL` | +| `we.service.include` | Specify the key value pairs for the include condition for the `WHERE` clause of service metrics. | `NULL` | +| `we.service.exclude` | Specify the key value pairs for the exclude condition for the `WHERE` clause of service metrics. | `NULL` | +| `we.process.allow_process_regex` | Specify the regular expression covering the process metrics to collect. | `"/.+/"` (all) | +| `we.process.deny_process_regex` | Specify the regular expression for process metrics to prevent collection of or ignore. | `NULL` (all) | +| `collector.cpu.scrape_interval` | The rate in seconds at which `cpu` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.net.scrape_interval` | The rate in seconds at which `net` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.logical_disk.scrape_interval` | The rate in seconds at which `logical_disk` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.cs.scrape_interval` | The rate in seconds at which `cs` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.os.scrape_interval`| The rate in seconds at which `os` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.thermalzone.scrape_interval`| The rate in seconds at which `thermalzone` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.cpu_info.scrape_interval`| The rate in seconds at which `cpu_info` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.logon.scrape_interval` | The rate in seconds at which `logon` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.system.scrape_interval` | The rate in seconds at which `system` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.service.scrape_interval` | The rate in seconds at which `service` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.memory.scrape_interval` | The rate in seconds at which `memory` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.paging_file.scrape_interval` | The rate in seconds at which `paging_file` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `collector.process.scrape_interval` | The rate in seconds at which `process` metrics are collected. Values greater than `0` override the global default. Otherwise, the global default is used. | `0 seconds` | +| `metrics` | Specify which metrics are collected. | `"cpu,cpu_info,os,net,logical_disk,cs,thermalzone,logon,system,service"` | ## Collectors available -The following table describes the available collectors as part of this plugin. All of them are enabled by default and respects the original metrics name, descriptions, and types from Prometheus Windows Exporter, so you can use your current dashboards without any compatibility problem. +The following table describes the available collectors as part of this plugin. All collectors are enabled by default and respect the original metrics name, descriptions, and types from Prometheus Windows Exporter, so you can use your current dashboards without any compatibility problem. -> note: the Version column specifies the Fluent Bit version where the collector is available. +The Version column specifies the Fluent Bit version where the collector is available. | Name | Description | OS | Version | | ------------- | ------------------------------------------------------------------------------------------------ | ------- | ------- | -| cpu | Exposes CPU statistics. | Windows | v1.9 | -| net | Exposes Network statistics. | Windows | v2.0.8 | -| logical\_disk | Exposes logical\_disk statistics. | Windows | v2.0.8 | -| cs | Exposes cs statistics. | Windows | v2.0.8 | -| os | Exposes OS statistics. | Windows | v2.0.8 | -| thermalzone | Exposes thermalzone statistics. | Windows | v2.0.8 | -| cpu\_info | Exposes cpu\_info statistics. | Windows | v2.0.8 | -| logon | Exposes logon statistics. | Windows | v2.0.8 | -| system | Exposes system statistics. | Windows | v2.0.8 | -| service | Exposes service statistics. | Windows | v2.1.6 | -| memory | Exposes memory statistics. | Windows | v2.1.9 | -| paging\_file | Exposes paging\_file statistics. | Windows | v2.1.9 | -| process | Exposes process statistics. | Windows | v2.1.9 | +| `cpu` | Exposes CPU statistics. | Windows | v1.9 | +| `net` | Exposes Network statistics. | Windows | v2.0.8 | +| `logical_disk` | Exposes `logical_disk` statistics. | Windows | v2.0.8 | +| `cs` | Exposes `cs` statistics. | Windows | v2.0.8 | +| `os` | Exposes OS statistics. | Windows | v2.0.8 | +| `thermalzone` | Exposes `thermalzone` statistics. | Windows | v2.0.8 | +| `cpu_info` | Exposes `cpu_info` statistics. | Windows | v2.0.8 | +| `logon` | Exposes `logon` statistics. | Windows | v2.0.8 | +| `system` | Exposes `system` statistics. | Windows | v2.0.8 | +| `service` | Exposes `service` statistics. | Windows | v2.1.6 | +| `memory` | Exposes `memory` statistics. | Windows | v2.1.9 | +| `paging_file` | Exposes `paging_file` statistics. | Windows | v2.1.9 | +| `process` | Exposes `process` statistics. | Windows | v2.1.9 | ## Threading This input always runs in its own [thread](../../administration/multithreading.md#inputs). -## Getting Started +## Get started -### Simple Configuration File +### Configuration file -In the following configuration file, the input plugin _windows_exporter_metrics collects _metrics every 2 seconds and exposes them through our [Prometheus Exporter](../outputs/prometheus-exporter.md) output plugin on HTTP/TCP port 2021. +In the following configuration file, the input plugin `windows_exporter_metrics` collects `metrics` every 2 seconds and exposes them through the [Prometheus Exporter](../outputs/prometheus-exporter.md) output plugin on HTTP/TCP port `2021`. -``` +```text # Node Exporter Metrics + Prometheus Exporter # ------------------------------------------- # The following example collect host metrics on Linux and expose @@ -101,7 +99,7 @@ In the following configuration file, the input plugin _windows_exporter_metrics ``` -You can test the expose of the metrics by using _curl:_ +You can test the expose of the metrics by using `curl`: ```bash curl http://127.0.0.1:2021/metrics @@ -112,30 +110,31 @@ curl http://127.0.0.1:2021/metrics Windows service collector will retrieve all of the service information for the local node or container. `we.service.where`, `we.service.include`, and `we.service.exclude` can be used to filter the service metrics. -To filter these metrics, users should specify a WHERE clause. -This syntax is defined in [the WMI Query Language(WQL)](https://learn.microsoft.com/en-us/windows/win32/wmisdk/wql-sql-for-wmi). +To filter these metrics, users should specify a `WHERE` clause. +This syntax is defined in [the WMI Query Language (WQL)](https://learn.microsoft.com/en-us/windows/win32/wmisdk/wql-sql-for-wmi). Here is how these parameters should work: -#### we.service.where +#### `we.service.where` -`we.service.where` is handled as a raw WHERE clause. +`we.service.where` is handled as a raw `WHERE` clause. For example, when a user specifies the parameter as follows: -``` +```text we.service.where Status!='OK' ``` -This creates a WMI query like so: -``` +This creates a WMI query like: + +```text SELECT * FROM Win32_Service WHERE Status!='OK' ``` -The WMI mechanism will then handle it and return the information which has a "not OK" status in this example. +The WMI mechanism will then handle it and return the information which has a `not OK` status in this example. -### we.service.include +### `we.service.include` -When defined, the `we.service.include` is interpreted into a WHERE clause. +When defined, the `we.service.include` is interpreted into a `WHERE` clause. If multiple key-value pairs are specified, the values will be concatenated with `OR`. Also, if the values contain `%` character then a `LIKE` operator will be used in the clause instead of the `=` operator. When a user specifies the parameter as follows: @@ -145,19 +144,20 @@ we.service.include {"Name":"docker","Name":"%Svc%", "Name":"%Service"} ``` The parameter will be interpreted as: -``` + +```text (Name='docker' OR Name LIKE '%Svc%' OR Name LIKE '%Service') ``` The WMI query will be called with the translated parameter as: -``` +```text SELECT * FROM Win32_Service WHERE (Name='docker' OR Name LIKE '%Svc%' OR Name LIKE '%Service') ``` -### we.service.exclude +### `we.service.exclude` -When defined, the `we.service.exclude` is interpreted into a WHERE clause. +When defined, the `we.service.exclude` is interpreted into a `WHERE` clause. If multiple key-value pairs are specified, the values will be concatenated with `AND`. Also, if the values contain `%` character then a `LIKE` operator will be used in the translated clause instead of the `!=` operator. @@ -168,13 +168,14 @@ we.service.exclude {"Name":"UdkUserSvc%","Name":"webthreatdefusersvc%","Name":"X ``` The parameter will be interpreted as: -``` + +```text (NOT Name LIKE 'UdkUserSvc%' AND NOT Name LIKE 'webthreatdefusersvc%' AND Name!='XboxNetApiSvc') ``` The WMI query will be called with the translated parameter as: -``` +```text SELECT * FROM Win32_Service WHERE (NOT Name LIKE 'UdkUserSvc%' AND NOT Name LIKE 'webthreatdefusersvc%' AND Name!='XboxNetApiSvc') ``` @@ -185,12 +186,12 @@ SELECT * FROM Win32_Service WHERE (NOT Name LIKE 'UdkUserSvc%' AND NOT Name LIKE 1. `we.service.include` translated and applied into the where clause in the service collector 1. `we.service.exclude` translated and applied into the where clause in the service collector 1. If the `we.service.include` is applied, translated `we.service.include` and `we.service.exclude` conditions are concatenated with `AND`. -1. `we.service.where` is just handled as-is into the where clause in the service collector . - 1. If either of the above parameters is applied, the clause will be applied with `AND (` _the value of `we.service.where`_ `)`. +1. `we.service.where` is handled as-is into the where clause in the service collector . + 1. If either of the previous parameters is applied, the clause will be applied with `AND (` _the value of `we.service.where`_ `)`. For example, when a user specifies the parameter as follows: -``` +```text we.service.include {"Name":"docker","Name":"%Svc%", "Name":"%Service"} we.service.exclude {"Name":"UdkUserSvc%","Name":"XboxNetApiSvc"} we.service.where NOT Name LIKE 'webthreatdefusersvc%' @@ -198,13 +199,12 @@ we.service.where NOT Name LIKE 'webthreatdefusersvc%' The WMI query will be called with the translated parameter as: -``` +```text SELECT * FROM Win32_Service WHERE (Name='docker' OR Name LIKE '%Svc%' OR Name LIKE '%Service') AND (NOT Name LIKE 'UdkUserSvc%' AND Name!='XboxNetApiSvc') AND (NOT Name LIKE 'webthreatdefusersvc%') ``` +## Enhancement requests -## Enhancement Requests +The plugin implements a subset of the available collectors in the original Prometheus Windows Exporter. If you would like a specific collector prioritized, open a Github issue by using the following template: -Our current plugin implements a sub-set of the available collectors in the original Prometheus Windows Exporter, if you would like that we prioritize a specific collector please open a Github issue by using the following template:\ -\ -\- [in_windows_exporter_metrics](https://github.com/fluent/fluent-bit/issues/new?assignees=\&labels=\&template=feature_request.md\&title=in_windows_exporter_metrics:%20add%20ABC%20collector) +- [`in_windows_exporter_metrics`](https://github.com/fluent/fluent-bit/issues/new?assignees=\&labels=\&template=feature_request.md\&title=in_windows_exporter_metrics:%20add%20ABC%20collector) diff --git a/vale-styles/FluentBit/Acronyms.yml b/vale-styles/FluentBit/Acronyms.yml index d6eb84dcc..1f4544c5c 100644 --- a/vale-styles/FluentBit/Acronyms.yml +++ b/vale-styles/FluentBit/Acronyms.yml @@ -56,6 +56,7 @@ exceptions: - MQTT - NET - NGINX + - NIC - NOTE - NVDA - OSS @@ -96,6 +97,7 @@ exceptions: - UTF - UUID - WASI + - WMI - XML - XSS - YAML From d7cd5c02c80d6752e15395637d34b7191cdfd4d3 Mon Sep 17 00:00:00 2001 From: Lynette Miles <6818907+esmerel@users.noreply.github.com> Date: Thu, 3 Jul 2025 10:40:57 -0700 Subject: [PATCH 2/2] Apply suggestions from code review Co-authored-by: Craig Norris <112565517+cnorris-cs@users.noreply.github.com> Signed-off-by: Lynette Miles <6818907+esmerel@users.noreply.github.com> --- pipeline/inputs/windows-exporter-metrics.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/pipeline/inputs/windows-exporter-metrics.md b/pipeline/inputs/windows-exporter-metrics.md index 28e6937e7..a85262311 100644 --- a/pipeline/inputs/windows-exporter-metrics.md +++ b/pipeline/inputs/windows-exporter-metrics.md @@ -1,6 +1,5 @@ --- -description: A plugin based on Prometheus Windows Exporter to collect system and host level - metrics +description: A plugin based on Prometheus Windows Exporter to collect system and host level metrics --- # Windows Exporter metrics @@ -18,7 +17,7 @@ Metrics collected with Windows Exporter metrics flow through a separate pipeline | `scrape_interval` | The rate at which metrics are collected. | `5 seconds` | | `we.logical_disk.allow_disk_regex` | Specify the regular expression for logical disk metrics to allow collection of. | `"/.+/"` (all) | | `we.logical_disk.deny_disk_regex` | Specify the regular expression for logical disk metrics to prevent collection of or ignore. | `NULL` (all) | -| `we.net.allow_nic_regex` | Specify the regular expression for network metrics captured by the name of the NIC | `"/.+/"` (all) | +| `we.net.allow_nic_regex` | Specify the regular expression for network metrics captured by the name of the NIC. | `"/.+/"` (all) | | `we.service.where` | Specify the `WHERE` clause for retrieving service metrics. | `NULL` | | `we.service.include` | Specify the key value pairs for the include condition for the `WHERE` clause of service metrics. | `NULL` | | `we.service.exclude` | Specify the key value pairs for the exclude condition for the `WHERE` clause of service metrics. | `NULL` | @@ -69,7 +68,7 @@ This input always runs in its own [thread](../../administration/multithreading.m ### Configuration file -In the following configuration file, the input plugin `windows_exporter_metrics` collects `metrics` every 2 seconds and exposes them through the [Prometheus Exporter](../outputs/prometheus-exporter.md) output plugin on HTTP/TCP port `2021`. +In the following configuration file, the input plugin `windows_exporter_metrics` collects `metrics` every two seconds and exposes them through the [Prometheus Exporter](../outputs/prometheus-exporter.md) output plugin on HTTP/TCP port `2021`. ```text # Node Exporter Metrics + Prometheus Exporter