Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/linkcheck.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
id: lychee
uses: lycheeverse/lychee-action@v2
with:
args: --accept '100..=103,200..=299,401,403,429' --verbose --no-progress .
args: --accept '100..=103,200..=299,401,403,429,999' --verbose --no-progress .
fail: false
jobSummary: true

Expand Down
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

First of all, thanks for taking the time to read this guide. The fact that you're here means you're interested in contributing to Fluent Bit, and we greatly appreciate your time.

This repository contains the files for the [Fluent Bit documentation library](https://docs.fluentbit.io/manual/). Keeping these docs separate from the [main Fluent Bit repository](https://github.com/fluent/fluent-bit) helps reduce the number of commits to the Fluent Bit source code and makes it easier to maintain both projects.
This repository contains the files for the [Fluent Bit documentation library](https://docs.fluentbit.io/manual). Keeping these docs separate from the [main Fluent Bit repository](https://github.com/fluent/fluent-bit) helps reduce the number of commits to the Fluent Bit source code and makes it easier to maintain both projects.

Fluent Bit has a group of dedicated maintainers who oversee this repository, including several technical writers. These writers will review any pull requests you open, so don't be afraid to contribute, even if you're not a writer by trade. Your suggestions are valuable, and we'll help you wrangle any stray commas.

Expand Down Expand Up @@ -102,7 +102,7 @@ awkwardly.
### Quotes

By default, Google Docs and Microsoft Word turn standard straight quotes into "smart"
curly quotes. If you copy-paste from one of these tools, you must correct the quotes back to straight quotes. You can also turn off smart quotes in [Google Docs](https://support.google.com/docs/thread/217182974/can-i-turn-smart-quotes-off-in-a-google-doc?hl=en) or [Microsoft Word](https://support.microsoft.com/en-us/office/smart-quotes-in-word-and-powerpoint-702fc92e-b723-4e3d-b2cc-71dedaf2f343) to prevent this problem.
curly quotes. If you copy-paste from one of these tools, you must correct the quotes back to straight quotes. You can also turn off smart quotes in [Google Docs](https://support.google.com/docs/thread/217182974/can-i-turn-smart-quotes-off-in-a-google-doc?hl=en) or [Microsoft Word](https://support.microsoft.com/en-US/Word/smart-quotes-in-word-and-powerpoint) to prevent this problem.

### Table of contents

Expand Down Expand Up @@ -174,7 +174,7 @@ This repository runs linters as GitHub Actions for each pull request. If a linte

### Vale

[Vale](https://vale.sh/docs) lints prose for style and clarity. In addition to reviewing the results of each Vale test in GitHub, you can use the [Vale plugin for VSCode](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode) to view errors and suggestions locally.
[Vale](https://docs.vale.sh/) lints prose for style and clarity. In addition to reviewing the results of each Vale test in GitHub, you can use the [Vale plugin for VSCode](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode) to view errors and suggestions locally.

Vale tests for the Fluent Bit docs are stored in the [`/vale-styles`](https://github.com/fluent/fluent-bit-docs/tree/master/vale-styles) folder. Most Vale tests are at the `suggestion` or `warning` level and won't block pull requests from merging. However, tests at the `error` level will block merging until the associated issue is fixed.

Expand Down
2 changes: 1 addition & 1 deletion MAINTAINERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ Fluent Bit is developed and supported by many individuals and companies. The fol
| [Jose Lecaros](https://github.com/lecaros) | All | [Palo Alto Networks](https://www.paloaltonetworks.com) |
| [Paige Cruz](https://github.com/paigerduty) | All | [Palo Alto Networks](https://www.paloaltonetworks.com) |
| [Eric D. Schabell](https://github.com/eschabell) | All | [SUSE](https://www.suse.com/) |
| [Patrick Stephens](https://github.com/patrick-stephens)| All | [Telemetry Forge](https://telemetryforge.io) |
| [Patrick Stephens](https://github.com/patrick-stephens)| All | [Telemetry Forge](https://www.telemetryforge.io) |
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ If you are upgrading from the Fluent Bit `4.2` series, start with [What's new in

Fluent Bit is a [CNCF](https://www.cncf.io/) graduated sub-project under the umbrella of [Fluentd](https://www.fluentd.org).

Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/) and is now sponsored by [Chronosphere](https://chronosphere.io/). As a CNCF-hosted project, it's a fully vendor-neutral and community-driven project.
Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper) and is now sponsored by [Chronosphere](https://chronosphere.io/). As a CNCF-hosted project, it's a fully vendor-neutral and community-driven project.

## License

Expand Down
2 changes: 1 addition & 1 deletion development/external-libraries.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ These libraries are developed and maintained by the Fluent Bit team:
| Library | Purpose | License |
|---------|---------|---------|
| [`LuaJIT`](https://luajit.org/) | `Just-in-time (JIT)` compiler for Lua. Powers the [Lua filter](../pipeline/filters/lua.md) plugin for custom record processing. | Massachusetts Institute of Technology (MIT) |
| [`WAMR`](https://github.com/bytecodealliance/wasm-micro-runtime) | WebAssembly Micro Runtime from the Bytecode Alliance. Enables [WASM filter plugins](wasm-filter-plugins.md) and [WASM input plugins](wasm-input-plugins.md). Supports `interpreter`, `AOT`, and `JIT` execution modes. | Apache 2.0 |
| [`WAMR`](https://github.com/wasm-micro-runtime/wasm-micro-runtime) | WebAssembly Micro Runtime from the Bytecode Alliance. Enables [WASM filter plugins](wasm-filter-plugins.md) and [WASM input plugins](wasm-input-plugins.md). Supports `interpreter`, `AOT`, and `JIT` execution modes. | Apache 2.0 |

## Integration libraries

Expand Down
2 changes: 1 addition & 1 deletion development/wasm-filter-plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ There are no additional requirements to execute Wasm plugins.

#### Build `flb-wamrc` (optional)

`flb-wamrc` is a `flb`-prefixed Ahead of Time (AOT) compiler that's provided from [`wasm-micro-runtime`](https://github.com/bytecodealliance/wasm-micro-runtime).
`flb-wamrc` is a `flb`-prefixed Ahead of Time (AOT) compiler that's provided from [`wasm-micro-runtime`](https://github.com/wasm-micro-runtime/wasm-micro-runtime).

For `flb-wamrc` support, you must install the LLVM infrastructure and some additional libraries (`libmlir`, `libPolly`, `libedit`, and `libpfm`). For example:

Expand Down
4 changes: 2 additions & 2 deletions pipeline/filters/tensorflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
**Supported event types:** `logs`
{% endhint %}

The _Tensorflow_ filter plugin allows running machine learning inference tasks on the records of data coming from input plugins or stream processors. This filter uses [Tensorflow Lite](https://ai.google.dev/edge/litert) as the inference engine, and requires Tensorflow Lite shared library to be present during build and at runtime.
The _Tensorflow_ filter plugin allows running machine learning inference tasks on the records of data coming from input plugins or stream processors. This filter uses [Tensorflow Lite](https://developers.google.com/edge/litert) as the inference engine, and requires Tensorflow Lite shared library to be present during build and at runtime.

Tensorflow Lite is a lightweight open source deep learning framework used for mobile and IoT applications. Tensorflow Lite only handles inference, not training. It loads pre-trained models (`.tflite` files) that are converted into Tensorflow Lite format (`FlatBuffer`). You can read more on converting [Tensorflow models](https://ai.google.dev/edge/litert/conversion/tensorflow/overview).
Tensorflow Lite is a lightweight open source deep learning framework used for mobile and IoT applications. Tensorflow Lite only handles inference, not training. It loads pre-trained models (`.tflite` files) that are converted into Tensorflow Lite format (`FlatBuffer`). You can read more on converting [Tensorflow models](https://developers.google.com/edge/litert/conversion/tensorflow/overview).

The Tensorflow plugin for Fluent Bit has the following limitations:

Expand Down
4 changes: 2 additions & 2 deletions pipeline/outputs/azure_kusto.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Send logs to Azure Data Explorer (Kusto)

# Azure Data Explorer

The _Kusto_ output plugin lets you ingest your logs into an [Azure Data Explorer](https://azure.microsoft.com/en-us/products/data-explorer/) cluster, using the [Queued Ingestion](https://learn.microsoft.com/en-us/kusto/api/netfx/about-kusto-ingest?view=azure-data-explorer&preserve-view=true&tabs=csharp#queued-ingestion) mechanism. This output plugin can also be used to ingest logs into an [Eventhouse](https://blog.fabric.microsoft.com/en-us/blog/eventhouse-overview-handling-real-time-data-with-microsoft-fabric/) cluster in Microsoft Fabric Real Time Analytics.
The _Kusto_ output plugin lets you ingest your logs into an [Azure Data Explorer](https://azure.microsoft.com/en-us/products/data-explorer/) cluster, using the [Queued Ingestion](https://learn.microsoft.com/en-us/kusto/api/netfx/about-kusto-ingest?view=azure-data-explorer&preserve-view=true&tabs=csharp#queued-ingestion) mechanism. This output plugin can also be used to ingest logs into an [Eventhouse](https://learn.microsoft.com/en-us/fabric/real-time-intelligence/eventhouse) cluster in Microsoft Fabric Real Time Analytics.

## Authentication methods

Expand All @@ -16,7 +16,7 @@ For service principal authentication, you'll need to create an Azure AD applicat

- [Register an Application](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app#register-an-application)
- [Add a client secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app#add-a-client-secret)
- [Authorize the app in your database](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/management/access-control/principals-and-identity-providers#azure-ad-tenants)
- [Authorize the app in your database](https://learn.microsoft.com/en-us/kusto/management/reference-security-principals?view=azure-data-explorer#referencing-microsoft-entra-principals-and-groups)

Configure Fluent Bit with your application's `tenant_id`, `client_id`, and `client_secret`.

Expand Down
6 changes: 3 additions & 3 deletions pipeline/outputs/bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ The _Google Cloud BigQuery_ output plugin is an experimental plugin that lets yo
The implementation doesn't support the following, which would be expected in a full production version:

- [Application Default Credentials](https://docs.cloud.google.com/docs/authentication).
- [Data deduplication](https://docs.cloud.google.com/bigquery/docs/streaming-data-into-bigquery) using `insertId`.
- [Template tables](https://docs.cloud.google.com/bigquery/docs/streaming-data-into-bigquery) using `templateSuffix`.
- [Data deduplication](https://docs.cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll) using `insertId`.
- [Template tables](https://docs.cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll) using `templateSuffix`.

## Google Cloud configuration

Expand All @@ -25,7 +25,7 @@ Fluent Bit streams data into an existing BigQuery table using a service account
You must configure workload identity federation in GCP before using it with Fluent Bit.

- [Configuring workload identity federation](https://cloud.google.com/iam/docs/configuring-workload-identity-federation#aws)
- [Obtaining short-lived credentials with identity federation](https://cloud.google.com/iam/docs/workload-identity-federation-with-other-clouds)
- [Obtaining short-lived credentials with identity federation](https://docs.cloud.google.com/iam/docs/workload-identity-federation-with-other-clouds)

## Configuration parameters

Expand Down
2 changes: 1 addition & 1 deletion pipeline/outputs/chronicle.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Fluent Bit streams data into an existing Google Chronicle tenant using a service
| `service_account_secret` | Private key content associated with the service account. Only available if no credentials file has been provided. | Value of the environment variable `$SERVICE_ACCOUNT_SECRET` |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |

See Google's [official documentation](https://cloud.google.com/chronicle/docs/reference/ingestion-api) for further details.
See Google's [official documentation](https://docs.cloud.google.com/chronicle/docs/reference/ingestion-api) for further details.

## Configuration file

Expand Down
6 changes: 3 additions & 3 deletions pipeline/outputs/forward.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
**Supported event types:** `logs` `metrics` `traces`
{% endhint %}

_Forward_ is the protocol used by [Fluentd](https://www.fluentd.org) to route messages between peers. The `forward` output plugin provides interoperability between [Fluent Bit](https://fluentbit.io) and [Fluentd](https://fluentd.org).
_Forward_ is the protocol used by [Fluentd](https://www.fluentd.org) to route messages between peers. The `forward` output plugin provides interoperability between [Fluent Bit](https://fluentbit.io) and [Fluentd](https://www.fluentd.org).

There are no configuration steps required besides specifying where [Fluentd](https://fluentd.org) is located, which can be a local or a remote destination.
There are no configuration steps required besides specifying where [Fluentd](https://www.fluentd.org) is located, which can be a local or a remote destination.

This plugin offers the following transports and modes:

Expand Down Expand Up @@ -56,7 +56,7 @@ When using Secure Forward mode, the [TLS](../../administration/transport-securit

## Forward setup

Before proceeding, ensure that [Fluentd](https://fluentd.org) is installed. If it's not, refer to the [Fluentd Installation](https://docs.fluentd.org/installation) document.
Before proceeding, ensure that [Fluentd](https://www.fluentd.org) is installed. If it's not, refer to the [Fluentd Installation](https://docs.fluentd.org/installation) document.

After installing Fluentd, create the following configuration file example which lets you to stream data into it:

Expand Down
2 changes: 1 addition & 1 deletion pipeline/outputs/influxdb.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
**Supported event types:** `logs` `metrics`
{% endhint %}

The _InfluxDB_ output plugin lets you flush your records into a [InfluxDB](https://www.influxdata.com/products/influxdb-overview) time series database. The following instructions assume that you have an operational InfluxDB service running in your system.
The _InfluxDB_ output plugin lets you flush your records into a [InfluxDB](https://www.influxdata.com/products/influxdb-overview/) time series database. The following instructions assume that you have an operational InfluxDB service running in your system.

## Configuration parameters

Expand Down
2 changes: 1 addition & 1 deletion pipeline/outputs/s3.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ When Fluent Bit receives logs, it stores them in chunks, either in memory or the

The S3 output plugin conforms to the Fluent Bit output plugin specification. Because S3's use case is to upload large files (over 2 MB), its behavior is different. S3's `flush` callback function buffers the incoming chunk to the filesystem, and returns an `FLB_OK`. This means Prometheus metrics available from the Fluent Bit HTTP server are meaningless for S3. In addition, the `storage.total_limit_size` parameter isn't meaningful for S3 since it has its own buffering system in the `store_dir`. Instead, use `store_dir_limit_size`. S3 requires a writeable filesystem. Running Fluent Bit on a read-only filesystem won't work with the S3 output.

S3 uploads primarily initiate using the S3 [`timer`](https://docs.aws.amazon.com/iotevents/latest/apireference/API_iotevents-data_Timer.html) callback function, which runs separately from its `flush`.
S3 uploads primarily initiate using the S3 `timer` callback function, which runs separately from its `flush`.

S3 has its own buffering system and its own callback to upload data, so the normal sequential data ordering of chunks provided by the Fluent Bit engine can be compromised. S3 has the `preserve_data_ordering` option which ensures data is uploaded in the original order it was collected by Fluent Bit.

Expand Down