Skip to content

out_file: support timestamp placeholder - #12245

Open
cosmo0920 wants to merge 5 commits into
masterfrom
cosmo0920-support-timestamp-placeholder
Open

out_file: support timestamp placeholder#12245
cosmo0920 wants to merge 5 commits into
masterfrom
cosmo0920-support-timestamp-placeholder

Conversation

@cosmo0920

@cosmo0920 cosmo0920 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

This should be a parity feature of Fluentd's placeholder support on Fluent Bit's out_file plugin.

Implemented and verified on Windows.

  • Added UTC event-timestamp expansion for strftime placeholders in file.c.
  • Added %Y%m%d coverage, including path/$TAG/$proxy_name/file.%Y%m%d, in out_file.c.
  • Added shared Windows-compatible directory helpers in flb_tests_runtime.h.in.
  • Enabled both runtime suites on Windows in CMakeLists.txt.
  • Ported rotation tests to Fluent Bit’s pthread compatibility layer and fixed cross-platform compilation in out_file_rotation.c.

The reported failures occurred because duplicate instances of the same test binary ran concurrently and shared their fixed fixture directories. Isolated sequential execution passes consistently:

ctest --test-dir build -R "^flb-rt-out_file(_rotation)?$" --output-on-failure

flb-rt-out_file           Passed — 22/22 cases
flb-rt-out_file_rotation  Passed — 25/25 cases
Total: 2/2 targets passed in 99.73 seconds

Targeted MSVC rebuild passed, git diff --check passed, and the repository commit-prefix checker passed. The branch is clean at 6174e0eee.

No focused tests/integration scenario exists for out_file. Platform memory checking was not run because this is Windows; Valgrind and macOS Leaks are unavailable.

Closes #12233.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features

    • Added UTC strftime placeholders for dynamic output paths and filenames when enabled.
    • Timestamp-based destinations are formatted for each event, including non-metric records.
    • Dynamic paths support combining timestamps with record values, tags, and literal percent characters.
    • Configuration documentation now explains timestamp formatting behavior.
  • Bug Fixes

    • Improved validation and cleanup for dynamic destinations.
    • Enhanced cross-platform compatibility for file output and rotation, including Windows.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 04441fb6-481b-4194-aa8b-fb7965ad22a1

📥 Commits

Reviewing files that changed from the base of the PR and between f21930c and 613d0da.

📒 Files selected for processing (1)
  • tests/runtime/out_file.c
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/runtime/out_file.c

📝 Walkthrough

Walkthrough

The file output plugin now supports UTC strftime placeholders in dynamic paths and filenames. Event timestamps flow into destination resolution. Runtime tests cover dated files, percent handling, and portable directory operations on all platforms.

Changes

Timestamp-based file destinations

Layer / File(s) Summary
Destination timestamp resolution
plugins/out_file/file.c
The plugin detects timestamp placeholders, formats UTC event timestamps in path and file, resolves record accessors, validates destinations, and documents the configuration.
Timestamp destination runtime coverage
tests/runtime/flb_tests_runtime.h.in, tests/runtime/out_file.c
Runtime tests verify literal percent handling, dated filenames, dynamic paths, record-derived percent values, fallback behavior, and portable directory cleanup.
Portable rotation test execution
tests/runtime/CMakeLists.txt, tests/runtime/out_file_rotation.c
File runtime tests run on all platforms. Rotation tests use compatibility APIs and portable timestamp and directory handling.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant EventDecoder
  participant OutFilePlugin
  participant OutputFile
  EventDecoder->>OutFilePlugin: decoded record with timestamp
  OutFilePlugin->>OutFilePlugin: format UTC strftime placeholders
  OutFilePlugin->>OutputFile: write to resolved path and filename
Loading

Possibly related PRs

Suggested reviewers: edsiper

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies timestamp placeholder support in the out_file plugin, which is the primary change.
Linked Issues check ✅ Passed The changes implement tag, record-field, and UTC event-date placeholders requested by issue #12233, with runtime test coverage.
Out of Scope Changes check ✅ Passed The portability helpers and Windows runtime-test updates support validation of the requested out_file behavior and are within scope.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cosmo0920-support-timestamp-placeholder

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/out_file/file.c`:
- Around line 421-428: Update the timestamp expansion logic in
plugins/out_file/file.c at lines 421-428 and 431-463 so strftime processes only
configured path/file placeholders, preserving record-accessor values literally,
including values containing %Y. Add or update the mixed accessor/timestamp
coverage in tests/runtime/out_file.c lines 618-662 to verify only configured
placeholders expand and accessor output remains unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7277a0d0-45a7-4062-80c7-0ab20c32757e

📥 Commits

Reviewing files that changed from the base of the PR and between 08609b6 and 6174e0e.

📒 Files selected for processing (5)
  • plugins/out_file/file.c
  • tests/runtime/CMakeLists.txt
  • tests/runtime/flb_tests_runtime.h.in
  • tests/runtime/out_file.c
  • tests/runtime/out_file_rotation.c

Comment thread plugins/out_file/file.c Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6174e0eee9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/out_file/file.c Outdated
Comment thread plugins/out_file/file.c Outdated
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

wwip
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tests/runtime/out_file.c (1)

643-645: 🎯 Functional Correctness | 🔵 Trivial | 🏗️ Heavy lift

Make the UTC assertions timezone-sensitive.

A local-time implementation can pass these tests when the runner timezone is UTC. Run the scenarios with a controlled non-UTC local timezone and use a timestamp that crosses a local-date boundary. Restore the process timezone after each test.

  • tests/runtime/out_file.c#L643-L645: verify the date-expanded filename uses the UTC event date.
  • tests/runtime/out_file.c#L673-L719: make the standalone timestamp test distinguish UTC from local time.
  • tests/runtime/out_file.c#L721-L778: retain the percent-value coverage while verifying the UTC year and date components.

As per coding guidelines, runtime tests must cover plugin-level semantics.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/runtime/out_file.c` around lines 643 - 645, Make the UTC assertions in
tests/runtime/out_file.c:643-645, tests/runtime/out_file.c:673-719, and
tests/runtime/out_file.c:721-778 timezone-sensitive by setting a controlled
non-UTC process timezone for each scenario, using timestamps that cross a
local-date boundary, and restoring the original timezone afterward; verify UTC
date expansion, distinguish standalone UTC timestamps from local time, and
preserve percent-value coverage while asserting UTC year and date components.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/runtime/out_file.c`:
- Around line 569-603: Update flb_test_file_literal_percent to construct its
output and fallback filenames under TEST_LOGPATH instead of the working
directory, including configuring both paths accordingly. Replace direct remove
calls and existence cleanup with flb_test_rmdir() for the test-specific
directory, while preserving the assertions that the output is created and the
fallback is absent.

---

Nitpick comments:
In `@tests/runtime/out_file.c`:
- Around line 643-645: Make the UTC assertions in
tests/runtime/out_file.c:643-645, tests/runtime/out_file.c:673-719, and
tests/runtime/out_file.c:721-778 timezone-sensitive by setting a controlled
non-UTC process timezone for each scenario, using timestamps that cross a
local-date boundary, and restoring the original timezone afterward; verify UTC
date expansion, distinguish standalone UTC timestamps from local time, and
preserve percent-value coverage while asserting UTC year and date components.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fe95adb7-b3b6-4fea-bedc-b01656d0ce3f

📥 Commits

Reviewing files that changed from the base of the PR and between 6174e0e and f21930c.

📒 Files selected for processing (2)
  • plugins/out_file/file.c
  • tests/runtime/out_file.c
🚧 Files skipped from review as they are similar to previous changes (1)
  • plugins/out_file/file.c

Comment thread tests/runtime/out_file.c
Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

filename placeholder support

1 participant