Skip to content

docs: administration: document input rate metrics and rate gate - #2637

Open
eschabell wants to merge 2 commits into
fluent:masterfrom
eschabell:erics_in_gate_rate_rate_metrics_updates
Open

docs: administration: document input rate metrics and rate gate#2637
eschabell wants to merge 2 commits into
fluent:masterfrom
eschabell:erics_in_gate_rate_rate_metrics_updates

Conversation

@eschabell

@eschabell eschabell commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Add the rate_window and rate_gate settings shared by all input plugins to the pipeline section, covering the pause and resume thresholds and how backpressure scales the effective limit. Add the five new fluentbit_input_rate_* metrics to the monitoring reference.

Note: this update needed to cover code changes without doc PR

Summary by CodeRabbit

  • Documentation
    • Added guidance for monitoring input ingestion rates and rate-limiting behavior.
    • Documented new Prometheus metrics for byte rate, record rate, busy chunks, limiting status, and retry attempts.
    • Included configuration options, defaults, threshold behavior, backpressure handling, and an HTTP input example.

@eschabell eschabell self-assigned this Jul 31, 2026
@eschabell
eschabell requested a review from a team as a code owner July 31, 2026 16:03
@eschabell eschabell added waiting-on-review Waiting on a review from mainteners 5.0.9 labels Jul 31, 2026
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 09e4c7b8-ae1c-4fe5-8e4b-0c8635102958

📥 Commits

Reviewing files that changed from the base of the PR and between 6d81f90 and 347afbb.

📒 Files selected for processing (2)
  • administration/configuring-fluent-bit/yaml/pipeline-section.md
  • administration/monitoring.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • administration/monitoring.md

📝 Walkthrough

Walkthrough

The documentation adds input ingestion-rate metrics, optional rate-gate configuration, rate-gate behavior, and five V2 Prometheus metrics.

Changes

Input rate documentation

Layer / File(s) Summary
Rate metrics and rate-gate documentation
administration/configuring-fluent-bit/yaml/pipeline-section.md, administration/monitoring.md
Documents ingestion-rate measurement, byte and record limits, backpressure adjustment, pause and resume thresholds, rate-gate states, retry attempts, and an HTTP configuration example.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested reviewers: patrick-stephens

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the documentation of input rate metrics and the rate gate.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@eschabell

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@administration/configuring-fluent-bit/yaml/pipeline-section.md`:
- Line 99: Update the gate behavior description to explain the combined-limit
semantics: pausing occurs when either the byte-rate or record-rate limit is
exceeded, and resumption requires both measured rates to fall below their
respective effective limits multiplied by rate_gate.resume_ratio.
- Line 97: Update the rate gate activation wording near the rate_gate
configuration description to state that rate_gate must be enabled and at least
one of rate_gate.max_bytes or rate_gate.max_records must be set to a non-zero
value; preserve that zero means unlimited.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: be8a90a6-52e4-4804-8bd1-619b83cdca10

📥 Commits

Reviewing files that changed from the base of the PR and between 0eb06a6 and 6d81f90.

📒 Files selected for processing (2)
  • administration/configuring-fluent-bit/yaml/pipeline-section.md
  • administration/monitoring.md

Comment thread administration/configuring-fluent-bit/yaml/pipeline-section.md Outdated
Comment thread administration/configuring-fluent-bit/yaml/pipeline-section.md Outdated
@eschabell
eschabell force-pushed the erics_in_gate_rate_rate_metrics_updates branch 2 times, most recently from 62885b2 to 047e6d3 Compare August 1, 2026 08:41
@eschabell

Copy link
Copy Markdown
Collaborator Author

@patrick-stephens ready for review!

Comment thread administration/configuring-fluent-bit/yaml/pipeline-section.md Outdated
Comment thread administration/configuring-fluent-bit/yaml/pipeline-section.md
@eschabell eschabell added 5.1 and removed 5.0.9 labels Aug 6, 2026
Add the rate_window and rate_gate settings shared by all input plugins
to the pipeline section, covering the pause and resume thresholds and
how backpressure scales the effective limit. Add the five new
fluentbit_input_rate_* metrics to the monitoring reference. Also
clarify rate gate pause and resume semantics and rate gate activation
requirements.

Note: this update needed to cover code changes without doc PR

Signed-off-by: Eric D. Schabell <eric@schabell.org>
…example

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@eschabell
eschabell force-pushed the erics_in_gate_rate_rate_metrics_updates branch from 047e6d3 to 347afbb Compare August 9, 2026 18:29
@eschabell

Copy link
Copy Markdown
Collaborator Author

@patrick-stephens fixed conflicts, back to you for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5.1 waiting-on-review Waiting on a review from mainteners

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants