Skip to content

Commit aba7dd0

Browse files
antoniomrfrancoeschabell
authored and
patrick-stephens
committed
docs: administration: http-proxy: document tls.proxy.* options (#2651)
* docs: administration: http-proxy: document tls.proxy.* options Fluent Bit 5.1 adds tls.proxy.ca_file, tls.proxy.ca_path, tls.proxy.verify and tls.proxy.verify_hostname, which configure certificate verification for the proxy leg when an output connects through an HTTPS proxy (HTTP_PROXY using the https:// scheme), independent from the destination's own tls.* settings. - administration/http-proxy.md: explain the https:// proxy scheme and add a new "TLS to the proxy" section with the properties table and a config example. - administration/transport-security.md: add the same properties to the shared tls.* reference table. - pipeline/outputs/s3.md, opentelemetry.md, azure_kusto.md: add the properties to each page's own inline tls.* table. - administration/networking.md and the three plugin pages above: fix net.proxy_env_ignore's description, which referenced a non-existent HTTPS_PROXY environment variable instead of the https:// scheme inside HTTP_PROXY/http_proxy. Signed-off-by: Antônio Franco <13881523+antoniomrfranco@users.noreply.github.com> * docs: docs: administration: http-proxy: transport-security: fix formatting and lint - http-proxy: add missing lead-in sentence for the HTTP_PROXY example that followed the config tabs with no introduction. - http-proxy: use Title_Case Tls in the classic config example to match repo convention. Dotted keys such as tls.proxy.ca_file stay lowercase. - http-proxy: drop italics from the "Supported in v5.1 or later." version note so it matches transport-security, s3, opentelemetry, and azure_kusto. - transport-security: convert the tls.* properties table to compact style. The long tls.proxy.* rows broke the table's aligned style, adding 12 markdownlint MD060 errors. Aligning instead would require padding every row to over 330 characters. Also clears 12 pre-existing MD060 errors. Signed-off-by: Eric D. Schabell <eric@schabell.org> --------- Signed-off-by: Antônio Franco <13881523+antoniomrfranco@users.noreply.github.com> Signed-off-by: Eric D. Schabell <eric@schabell.org> Co-authored-by: Eric D. Schabell <eric@schabell.org> Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
1 parent 3b83d66 commit aba7dd0

6 files changed

Lines changed: 101 additions & 21 deletions

File tree

‎administration/http-proxy.md‎

Lines changed: 65 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,9 @@ description: Enable traffic through a proxy server using the HTTP_PROXY environm
66

77
Fluent Bit supports configuring an HTTP proxy for all egress HTTP/HTTPS traffic using the `HTTP_PROXY` or `http_proxy` environment variable.
88

9-
The format for the HTTP proxy environment variable is `http://USER:PASS@HOST:PORT`, where:
9+
The format for the HTTP proxy environment variable is `SCHEME://USER:PASS@HOST:PORT`, where:
1010

11+
- _`SCHEME`_ is either `http` or `https`. Use `https` when the connection to the proxy itself must be TLS-encrypted, for example when the proxy sits behind a corporate TLS-terminating gateway. There's no separate `HTTPS_PROXY` environment variable: the scheme lives inside the same `HTTP_PROXY`/`http_proxy` value. See [TLS to the proxy](#tls-to-the-proxy) to configure certificate verification for this connection.
1112
- _`USER`_ is the username when using basic authentication.
1213
- _`PASS`_ is the password when using basic authentication.
1314
- _`HOST`_ is the HTTP proxy hostname or IP address.
@@ -33,6 +34,69 @@ The [HTTP output plugin](../pipeline/outputs/http.md) also supports configuring
3334

3435
{% endhint %}
3536

37+
## TLS to the proxy
38+
39+
When `HTTP_PROXY`/`http_proxy` uses the `https` scheme, Fluent Bit establishes a TLS connection to the proxy itself before issuing the `HTTP CONNECT` request described previously. This proxy-side TLS connection is configured independently from the destination's own `tls.*` settings (see [TLS/SSL](transport-security.md)), using a dedicated set of properties:
40+
41+
| Key | Description | Default |
42+
| --- | ----------- | ------- |
43+
| `tls.proxy.ca_file` | Absolute path to the CA certificate file used to verify the HTTPS proxy's certificate. Independent from `tls.ca_file`, which verifies the destination's certificate. Only applies to output plugins. Supported in v5.1 or later. | _none_ |
44+
| `tls.proxy.ca_path` | Absolute path to scan for CA certificate files used to verify the HTTPS proxy's certificate. Only applies to output plugins. Supported in v5.1 or later. | _none_ |
45+
| `tls.proxy.verify` | Force certificate validation for the HTTPS proxy connection. Only applies to output plugins. Supported in v5.1 or later. | `on` |
46+
| `tls.proxy.verify_hostname` | Force hostname verification for the HTTPS proxy connection. Only applies to output plugins. Supported in v5.1 or later. | `on` |
47+
48+
{% hint style="info" %}
49+
50+
`tls.proxy.*` properties are set on the output plugin, the same way as `tls.*` properties. When `tls.proxy.ca_file` and `tls.proxy.ca_path` are both left unset, Fluent Bit falls back to the system's default trust store to verify the HTTPS proxy's certificate.
51+
52+
{% endhint %}
53+
54+
For example, to reach an HTTPS proxy signed by a private or corporate CA:
55+
56+
{% tabs %}
57+
{% tab title="fluent-bit.yaml" %}
58+
59+
```yaml
60+
pipeline:
61+
inputs:
62+
- name: cpu
63+
tag: cpu
64+
65+
outputs:
66+
- name: http
67+
match: '*'
68+
host: backend.example.com
69+
port: 443
70+
tls: on
71+
tls.proxy.ca_file: /etc/certs/proxy-ca.crt
72+
```
73+
74+
{% endtab %}
75+
{% tab title="fluent-bit.conf" %}
76+
77+
```text
78+
[INPUT]
79+
Name cpu
80+
Tag cpu
81+
82+
[OUTPUT]
83+
Name http
84+
Match *
85+
Host backend.example.com
86+
Port 443
87+
Tls on
88+
tls.proxy.ca_file /etc/certs/proxy-ca.crt
89+
```
90+
91+
{% endtab %}
92+
{% endtabs %}
93+
94+
Then set the proxy environment variable to the HTTPS proxy:
95+
96+
```text
97+
HTTP_PROXY='https://proxy.example.com:3129'
98+
```
99+
36100
## `NO_PROXY`
37101

38102
Use the `NO_PROXY` environment variable when traffic shouldn't flow through the HTTP proxy. The `no_proxy` environment variable is also supported. When both `NO_PROXY` and `no_proxy` environment variables are provided, `NO_PROXY` takes precedence.

‎administration/networking.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ The following table describes the network configuration properties available and
6969
| `net.dns.resolver` | Select the primary DNS resolver type (`LEGACY` or `ASYNC`). | _none_ |
7070
| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it's retired. | `2000` |
7171
| `net.max_worker_connections` | Set maximum number of TCP connections that can be established per worker. | `0` |
72-
| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` |
72+
| `net.proxy_env_ignore` | Ignore the `HTTP_PROXY`/`http_proxy` and `NO_PROXY`/`no_proxy` environment variables when set. | `false` |
7373
| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` |
7474
| `net.tcp_keepalive_time` | Interval between the last data packet sent and the first TCP keepalive probe. | `-1` |
7575
| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a `keepidle` probe. | `-1` |

‎administration/transport-security.md‎

Lines changed: 20 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -12,22 +12,26 @@ Both NPN and ALPN are used when client and server are establishing SSL/TLS conne
1212

1313
Both input and output plugins that perform Network I/O can optionally enable TLS and configure the behavior. The following table describes the properties available:
1414

15-
| Property | Description | Default |
16-
|:----------------------|:----------------------------------------------------------------------------------------------------------------------------------------|:--------|
17-
| `tls` | Enable or disable TLS support. | `off` |
18-
| `tls.debug` | Set TLS debug verbosity level. Accepted values: `0` (No debug), `1` (Error), `2` (State change), `3` (Informational) and `4` (Verbose). | `1` |
19-
| `tls.ca_file` | Absolute path to CA certificate file. | _none_ |
20-
| `tls.ca_path` | Absolute path to scan for certificate files. | _none_ |
21-
| `tls.ciphers` | Specify TLS ciphers up to TLSv1.2. | _none_ |
22-
| `tls.crt_file` | Absolute path to Certificate file. | _none_ |
23-
| `tls.key_file` | Absolute path to private Key file. | _none_ |
24-
| `tls.key_passwd` | Optional password for `tls.key_file` file. | _none_ |
25-
| `tls.max_version` | Specify the maximum version of TLS. | _none_ |
26-
| `tls.min_version` | Specify the minimum version of TLS. | _none_ |
27-
| `tls.verify` | Force certificate validation. | `on` |
28-
| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ |
29-
| `tls.verify_hostname` | Force TLS verification of host names. | `off` |
30-
| `tls.verify_client_cert` | Require and verify the TLS certificate presented by a connecting client. Enables mutual TLS (mTLS) for input plugins. Only applies to input plugins. | `off` |
15+
| Property | Description | Default |
16+
| :--- | :--- | :--- |
17+
| `tls` | Enable or disable TLS support. | `off` |
18+
| `tls.debug` | Set TLS debug verbosity level. Accepted values: `0` (No debug), `1` (Error), `2` (State change), `3` (Informational) and `4` (Verbose). | `1` |
19+
| `tls.ca_file` | Absolute path to CA certificate file. | _none_ |
20+
| `tls.ca_path` | Absolute path to scan for certificate files. | _none_ |
21+
| `tls.ciphers` | Specify TLS ciphers up to TLSv1.2. | _none_ |
22+
| `tls.crt_file` | Absolute path to Certificate file. | _none_ |
23+
| `tls.key_file` | Absolute path to private Key file. | _none_ |
24+
| `tls.key_passwd` | Optional password for `tls.key_file` file. | _none_ |
25+
| `tls.max_version` | Specify the maximum version of TLS. | _none_ |
26+
| `tls.min_version` | Specify the minimum version of TLS. | _none_ |
27+
| `tls.proxy.ca_file` | Absolute path to the CA certificate file used to verify the HTTPS proxy's certificate. Independent from `tls.ca_file`, which verifies the destination's certificate. Only applies to output plugins connecting through an HTTPS proxy. See [HTTP proxy](http-proxy.md). Supported in v5.1 or later. | _none_ |
28+
| `tls.proxy.ca_path` | Absolute path to scan for CA certificate files used to verify the HTTPS proxy's certificate. Only applies to output plugins connecting through an HTTPS proxy. Supported in v5.1 or later. | _none_ |
29+
| `tls.proxy.verify` | Force certificate validation for the HTTPS proxy connection. Only applies to output plugins connecting through an HTTPS proxy. Supported in v5.1 or later. | `on` |
30+
| `tls.proxy.verify_hostname` | Force hostname verification for the HTTPS proxy connection. Only applies to output plugins connecting through an HTTPS proxy. Supported in v5.1 or later. | `on` |
31+
| `tls.verify` | Force certificate validation. | `on` |
32+
| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ |
33+
| `tls.verify_hostname` | Force TLS verification of host names. | `off` |
34+
| `tls.verify_client_cert` | Require and verify the TLS certificate presented by a connecting client. Enables mutual TLS (mTLS) for input plugins. Only applies to input plugins. | `off` |
3135

3236
{% hint style="info" %}
3337

‎pipeline/outputs/azure_kusto.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ By default, Kusto will insert incoming ingestion data into a table by inferring
120120
| `net.keepalive_idle_timeout` | Set maximum time allowed for an idle `Keepalive` connection.. | `false` |
121121
| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it's retried. | `2000` |
122122
| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` |
123-
| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` |
123+
| `net.proxy_env_ignore` | Ignore the `HTTP_PROXY`/`http_proxy` and `NO_PROXY`/`no_proxy` environment variables when set. | `false` |
124124
| `net.source_address` | Specify network address to bind for data traffic. | _none_ |
125125
| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` |
126126
| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a `keepidle` probe. | `-1` |
@@ -144,6 +144,10 @@ By default, Kusto will insert incoming ingestion data into a table by inferring
144144
| `tls.key_passwd` | Optional password for tls.key_file file. | _none_ |
145145
| `tls.max_version` | Specify the maximum version of TLS. | _none_ |
146146
| `tls.min_version` | Specify the minimum version of TLS. | _none_ |
147+
| `tls.proxy.ca_file` | Absolute path to the CA certificate file used to verify the HTTPS proxy's certificate. Independent from `tls.ca_file`. See [HTTP proxy](../../administration/http-proxy.md). Supported in v5.1 or later. | _none_ |
148+
| `tls.proxy.ca_path` | Absolute path to scan for CA certificate files used to verify the HTTPS proxy's certificate. Supported in v5.1 or later. | _none_ |
149+
| `tls.proxy.verify` | Force certificate validation for the HTTPS proxy connection. Supported in v5.1 or later. | `on` |
150+
| `tls.proxy.verify_hostname` | Force hostname verification for the HTTPS proxy connection. Supported in v5.1 or later. | `on` |
147151
| `tls.verify` | Force certificate validation. | `on` |
148152
| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` |
149153
| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ |

‎pipeline/outputs/opentelemetry.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ pipeline:
114114
| `net.io_timeout` | Set maximum time a connection can stay idle while assigned. | `0s` |
115115
| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it retries. | `2000` |
116116
| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` |
117-
| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` |
117+
| `net.proxy_env_ignore` | Ignore the `HTTP_PROXY`/`http_proxy` and `NO_PROXY`/`no_proxy` environment variables when set. | `false` |
118118
| `net.source_address` | Specify network address to bind for data traffic. | _none_ |
119119
| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` |
120120
| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a `keepidle` probe. | `-1` |
@@ -151,6 +151,10 @@ pipeline:
151151
| `tls.key_passwd` | Optional password for tls.key_file. | _none_ |
152152
| `tls.max_version` | Specify the maximum version of TLS. | _none_ |
153153
| `tls.min_version` | Specify the minimum version of TLS. | _none_ |
154+
| `tls.proxy.ca_file` | Absolute path to the CA certificate file used to verify the HTTPS proxy's certificate. Independent from `tls.ca_file`. See [HTTP proxy](../../administration/http-proxy.md). Supported in v5.1 or later. | _none_ |
155+
| `tls.proxy.ca_path` | Absolute path to scan for CA certificate files used to verify the HTTPS proxy's certificate. Supported in v5.1 or later. | _none_ |
156+
| `tls.proxy.verify` | Force certificate validation for the HTTPS proxy connection. Supported in v5.1 or later. | `on` |
157+
| `tls.proxy.verify_hostname` | Force hostname verification for the HTTPS proxy connection. Supported in v5.1 or later. | `on` |
154158
| `tls.verify` | Force certificate validation. | `on` |
155159
| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` |
156160
| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ |

‎pipeline/outputs/s3.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ The [Prometheus success/retry/error metrics values](../../administration/monitor
6868
| `net.io_timeout` | Set maximum time a connection can stay idle while assigned. | `0s` |
6969
| `net.keepalive_max_recycle` | Set maximum number of times a keepalive connection can be used before it retries. | `2000` |
7070
| `net.max_worker_connections` | Set the maximum number of active TCP connections that can be used per worker thread. | `0` |
71-
| `net.proxy_env_ignore` | Ignore the environment variables `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` when set. | `false` |
71+
| `net.proxy_env_ignore` | Ignore the `HTTP_PROXY`/`http_proxy` and `NO_PROXY`/`no_proxy` environment variables when set. | `false` |
7272
| `net.source_address` | Specify network address to bind for data traffic. | _none_ |
7373
| `net.tcp_keepalive` | Enable or disable Keepalive support. | `off` |
7474
| `net.tcp_keepalive_interval` | Interval between TCP keepalive probes when no response is received on a `keepidle` probe. | `-1` |
@@ -102,6 +102,10 @@ The [Prometheus success/retry/error metrics values](../../administration/monitor
102102
| `tls.key_passwd` | Optional password for tls.key_file file. | _none_ |
103103
| `tls.max_version` | Specify the maximum version of TLS. | _none_ |
104104
| `tls.min_version` | Specify the minimum version of TLS. | _none_ |
105+
| `tls.proxy.ca_file` | Absolute path to the CA certificate file used to verify the HTTPS proxy's certificate. Independent from `tls.ca_file`. See [HTTP proxy](../../administration/http-proxy.md). Supported in v5.1 or later. | _none_ |
106+
| `tls.proxy.ca_path` | Absolute path to scan for CA certificate files used to verify the HTTPS proxy's certificate. Supported in v5.1 or later. | _none_ |
107+
| `tls.proxy.verify` | Force certificate validation for the HTTPS proxy connection. Supported in v5.1 or later. | `on` |
108+
| `tls.proxy.verify_hostname` | Force hostname verification for the HTTPS proxy connection. Supported in v5.1 or later. | `on` |
105109
| `tls.verify` | Force certificate validation. | `on` |
106110
| `tls.verify_hostname` | Enable or disable to verify hostname. | `off` |
107111
| `tls.vhost` | Hostname to be used for TLS SNI extension. | _none_ |

0 commit comments

Comments
 (0)