feat(datadog_logs): optionally truncate oversized log events - #26512
Conversation
Adds opt-in encoded-log and retained-message limits to the Datadog logs sink. When enabled, this preserves standard fields, mark shortened messages, and shrinks encoded messages to fit configured payload limits.
bruceg
added
type: enhancement
github-actions
Bot
added
docs review on hold
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![datadoghq-integration[bot]](https://avatars.githubusercontent.com/u/16866914?s=60&v=4){kind=small}
There was a problem hiding this comment.
More details
The opt-in truncation path preserves standard fields, accounts for encoded size and UTF-8 boundaries, and drops irreducible events without blocking later logs; no reportable defect was identified.
🤖 Bits Code Review · Commit e94cbee · @DataDog review to ask questions
tessneau
left a comment
tessneau
left a comment
There was a problem hiding this comment.
a couple comments, but deferring to @ArunPiduguDD for final OP review since he has the context
github-actions
Bot
added
the
docs review on hold
|
I have reworked the implementation to just focus on truncating the |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 22a148fe3c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8305c63d79
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
tessneau
left a comment
tessneau
left a comment
There was a problem hiding this comment.
one more suggestion but this already feels in a good state, thanks for keeping on this 🎉
thomasqueirozb
left a comment
thomasqueirozb
left a comment
There was a problem hiding this comment.
Nice! Approving to unblock. None of the comments are blocking
github-actions
Bot
removed
the
docs review on hold
thomasqueirozb
added
the
docs review on hold
github-actions
Bot
removed
the
docs review on hold
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1c2be34329
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
5 participants
Summary
Adds opt-in encoded-log and retained-message limits to the Datadog logs sink. When enabled, this preserves standard fields, mark shortened messages, and shrinks encoded messages to fit configured payload limits.
References
Vector configuration
How did you test this PR?
The unit tests exercise the new functionality, including an added captive HTTP server test.
Does this PR include user facing changes?
no-changeloglabel to this PR.Contributor Guidelines
@vectordotdev/vectorto reach out to us regarding this PR.git merge origin masterandgit push.