Repository navigation
Conversation
github-actions
Bot
added
the
domain: sources
There was a problem hiding this comment.
Pull request overview
Adds Datadog Agent v3 series metrics intake support to the datadog_agent source so Vector can ingest metrics from Agent versions that default to the v3 endpoint, and updates tests to exercise the new path.
Changes:
- Add Warp routing + decoding for
/api/intake/metrics/v3/seriesand/api/intake/metrics/v3beta/series, translating v3 payloads into Vector metrics. - Add unit test coverage for decoding a v3 series payload.
- Restore the floating Datadog Agent v7 leg in the Datadog metrics E2E test matrix and add the v3 proto to codegen.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/e2e/datadog-metrics/config/test.yaml | Unpins the v7 agent matrix leg to exercise current Agent behavior (v3 default). |
| src/sources/datadog_agent/tests.rs | Adds a v3 endpoint constant, a raw-bytes sender helper, and a v3 decode test. |
| src/sources/datadog_agent/mod.rs | Wires in the generated v3 metrics protobuf module. |
| src/sources/datadog_agent/metrics.rs | Adds v3 series route and decoding/translation logic. |
| proto/vector/dd_metric_v3.proto | Introduces the v3 intake proto definition used for decoding. |
| changelog.d/25790_datadog_agent_v3_series.feature.md | Adds a changelog fragment documenting the user-facing feature. |
| build.rs | Adds the v3 proto to the build-time proto compilation inputs. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5330fba0d3
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
github-actions
Bot
added
domain: sinks
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8be2d900a4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 584d5bcbee
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e2463c4444
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 42d4e8684f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 068c2391cb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4000346324
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
This comment was marked as outdated.
This comment was marked as outdated.
|
Hi @arfa79, thank you for this PR. Ignore my previous message. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8402a3b92a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
@pront This branch is current with |
…v3-series # Conflicts: # lib/datadog-proto/proto/datadog/metrics/intake_v3.proto # lib/datadog-proto/src/lib.rs
…v3-series # Conflicts: # src/sinks/datadog/metrics/encoder.rs # tests/e2e/datadog-metrics/config/test.yaml
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@pront Thanks again for the earlier review and for requesting removal of the v3beta route; that change has been addressed. I have now also resolved the latest master conflicts in commit |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d3369adda5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 519a2355c1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9feff534c6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3677136d5f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
pront
left a comment
pront
left a comment
There was a problem hiding this comment.
This code mostly looks good. We would love to include this PR in the next release. I will do another pass after all codex threads are resovled.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f8f9919a6a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
@pront Thanks — all review threads are now addressed and resolved. Commit f8f9919 adds the v1/v2 unit-expansion bound and moves the v3 implementation into its dedicated module; 0bbe585 preserves the no-index flag through v2 wire metadata as well. Focused v1/v2/v3 tests, formatting, and combined Datadog source/sink Clippy checks pass. The PR is ready for your next review pass. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0bbe585d8c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| fn decode_v3_strings(raw: &[u8], sanitize: bool) -> crate::Result<Vec<String>> { | ||
| let mut values = vec![String::new()]; | ||
| let mut expanded_bytes = std::mem::size_of::<String>(); |
There was a problem hiding this comment.
When a compressed v3 payload puts roughly 525,000 zero-length entries in each of dictNameStr, dictTagStr, dictUnitStr, dictResourceStr, and dictSourceTypeName, every invocation resets expanded_bytes and receives its own 16 MiB allowance. The per-entry fix therefore still lets about 2.6 MiB of extremely compressible input create four simultaneously live Vec<String> dictionaries with roughly 50 MiB of initialized headers and around 100 MiB of vector capacity, even with zero series, so concurrent unauthenticated requests can exhaust the process; pass one cumulative dictionary budget through all of these decodes instead of resetting it per field.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 22ba240. All five v3 string dictionaries now share one cumulative expansion budget, including each dictionary sentinel, and the resource dictionary participates in the same accounting. Added a regression test that distributes empty entries across every dictionary field; the full 23-test decode_v3 set and combined Datadog source/sink Clippy check pass.
…v3-series # Conflicts: # src/sinks/datadog/metrics/encoder.rs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 22ba240f71
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| /// Datadog metric unit received from an upstream Agent payload. | ||
| #[serde(default)] | ||
| pub(crate) datadog_metric_unit: Option<Arc<str>>, |
There was a problem hiding this comment.
When a source output uses a memory buffer configured with max_size, or feeds the Datadog metrics sink, multiple Agent requests can each carry a one-point series with a large distinct unit (up to the new 16 MiB per-request limit), but the owned Arc<str> added here is never reported by EventMetadata::allocated_bytes (metadata.rs:388-394 only reports finalizers). The source memory queue (limited_queue.rs:330-341) and the Datadog sink's 5 MiB byte batcher (sinks/datadog/metrics/sink.rs:125-135) both use ByteSizeOf, so these metrics are charged as tiny while retaining many megabytes each; repeated compressed requests can therefore bypass both backpressure limits and exhaust memory before payload splitting. Account for the unit allocation, without double-counting shared Arcs, in the byte-size path.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 734abdb. EventMetadata::allocated_bytes now includes the Datadog metric-unit allocation and amortizes it across shared metadata/COW owners, so cloned events collectively account for the unit without charging its full size to every clone. Added coverage for both shared Inner metadata and copy-on-write metadata retaining the same unit. The focused test and vector-core Clippy with warnings denied pass.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 734abdb47e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| let owners = Arc::strong_count(&self.inner) | ||
| .saturating_mul(Arc::strong_count(unit)) | ||
| .max(1); | ||
| unit.len().div_ceil(owners) |
There was a problem hiding this comment.
When a unit-bearing event is fanned out to multiple outputs and one memory-buffered branch stalls while siblings drain, this denominator makes each branch reserve only 1/N of the unit: lib/vector-core/src/fanout.rs:339-344 clones the EventArray, while lib/vector-buffers/src/topology/channel/limited_queue.rs:332-341 snapshots allocated_bytes() once for permits. After fast siblings drop their clones, the stalled queue owns the full string but its reservation is never recomputed, so repeated large units can exceed max_size by the fanout width and exhaust memory. Fresh evidence after the prior accounting fix is the new strong_count-based amortization; use stable or conservative ownership accounting instead.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 733cd3e. Unit size accounting is now conservative and stable: every event reports the full retained unit length instead of dividing by transient Arc owner counts, so a queue's snapshotted reservation remains sufficient after sibling fanout clones drain. The focused vector-core regression test passes.
| .flatten(); | ||
| let mut event_metadata = get_event_metadata(serie.metadata.as_ref(), metric_type); | ||
| if !serie.unit.is_empty() { | ||
| event_metadata.set_datadog_metric_unit(serie.unit.clone()); |
There was a problem hiding this comment.
When a unit-bearing Agent metric is passed through metric_to_log and then a configured log_to_metric, storing the unit only in metadata makes the derived metric inherit a stale unit: src/transforms/metric_to_log.rs:295-296 preserves the metadata, and src/transforms/log_to_metric.rs:379-387 clones it while replacing only the origin metadata. If that transform creates a metric with a different name or meaning, both Datadog encoders now emit the original metric's unit; clear the unit for configured metric creation while retaining it for the all-metrics reconstruction path.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 733cd3e. Configured metrics now explicitly clear inherited Datadog unit metadata, while the reconstruction path continues preserving it. Added focused regression coverage; the transform test and Clippy with warnings denied pass.
There was a problem hiding this comment.
Clarification: configured log_to_metric metrics clear inherited unit metadata; the all_metrics reconstruction path preserves it.
github-actions
Bot
added
the
domain: transforms
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 733cd3e590
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| let event_bytes = std::mem::size_of::<Event>() | ||
| .checked_add(name.len()) | ||
| .and_then(|total| total.checked_add(tag_bytes)) | ||
| .and_then(|total| total.checked_add(resource_bytes)) | ||
| .and_then(|total| total.checked_add(source_type.len())) |
There was a problem hiding this comment.
When a v3 series has at least one point and hundreds of thousands of distinct short tags, this estimate charges only the raw tag strings. Fresh evidence after the emitted-event-overhead fix is that decode_ddseries converts those strings into the MetricTags BTreeMap<String, TagValueSet> defined in lib/vector-core/src/event/metric/tags.rs:457-460 and deep-clones that map for every point, while neither its entry/node storage nor the analogous resource/source-type entries are included in size_of::<Event>(). Such a compact unauthenticated payload can therefore pass both 16 MiB guards while allocating substantially more live memory; include the per-entry container storage in the budget or cap the number of tags/resources.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 6fb0da3. The v3 event budget now includes conservative per-entry storage for every decoded tag, resource-derived tag, and source-type tag, in addition to string bytes and event overhead. Added a regression test with 200,000 distinct short tags. All 24 focused decode_v3 tests and the combined source/sink/transform Clippy check with warnings denied pass.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6fb0da3319
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| event_bytes | ||
| .checked_mul(point_count) | ||
| .ok_or("Datadog v3 expanded event size overflow")?, |
There was a problem hiding this comment.
When a v3 row has num_points == 0 and a large distinct tagset, multiplying the estimate by point_count contributes zero, yet decode_ddseries still calls into_metric_tags at src/sources/datadog_agent/metrics.rs:355 before iterating the empty points vector. Fresh evidence after the per-event tag-entry fix is that about 500,000 short metadata tags can remain under the 16 MiB metadata/tagset/series limits but materialize an unbudgeted tens-of-megabytes BTreeMap on this unauthenticated route even though no event is emitted; skip conversion for zero-point rows or charge the base map independently of point count.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 558d2c2542
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| source: DatadogAgentSource, | ||
| ) -> BoxedFilter<(Response,)> { | ||
| warp::post() | ||
| .and(path!("api" / "intake" / "metrics" / "v3" / "series" / ..).and(warp::path::full())) |
There was a problem hiding this comment.
When an Agent posts to the advertised compatibility route /api/intake/metrics/v3beta/series, this literal v3 segment cannot match it; the other Datadog Agent source filters also contain no v3beta handler, so Warp returns 404 and those payloads never reach the decoder. Register the beta alias alongside the stable v3 route.
Useful? React with 👍 / 👎.
| if inner.datadog_origin_metadata.is_none() { | ||
| inner.datadog_origin_metadata = other.datadog_origin_metadata; | ||
| } | ||
| if inner.datadog_metric_unit.is_none() { | ||
| inner.datadog_metric_unit = other.datadog_metric_unit; |
There was a problem hiding this comment.
When aggregate Latest/Min/Max selects a same-series sample whose metadata has no Datadog origin or unit after another sample did, replace_entry in src/transforms/aggregate/event_time.rs deliberately makes the selected sample the metadata base and then merges the discarded sample. These new fallbacks fill that base from the discarded sample, so the selected value can be forwarded as no-index or with a unit it never had; keep semantic attribution out of generic EventMetadata::merge or resolve it explicitly in the aggregate.
Useful? React with 👍 / 👎.
Summary
Adds support for the Datadog Agent v3 series metrics intake format used by default by Datadog Agent 7.81.0 and newer.
The
datadog_agentsource now accepts the dictionary-encoded protobuf payload at/api/intake/metrics/v3/seriesand/api/intake/metrics/v3beta/series, translates it into Vector metrics, and preserves metric names, tags, resources, units, intervals, timestamps, values, and origin metadata.The Datadog metrics e2e matrix also restores the floating Agent
7test leg so current Agent versions exercise the v3 path.References
Closes: #25790
Vector configuration
No new configuration is required. Existing
datadog_agentsource configurations accept the new intake endpoint automatically.How did you test this PR?
cargo check --lib --no-default-features --features sources-datadog_agentcargo test --lib --no-default-features --features sources-datadog_agent datadog_agent::cargo clippy --lib --no-default-features --features sources-datadog_agent -- -D warningscargo fmt --all -- --checkIs this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.