Skip to content

chore(config): make Core Agent config authoritative for V3 routing - #2226

Open
rayz wants to merge 4 commits into
mainfrom
rayz/remove-adp-v3-series-gate
Open

chore(config): make Core Agent config authoritative for V3 routing#2226
rayz wants to merge 4 commits into
mainfrom
rayz/remove-adp-v3-series-gate

Conversation

@rayz

@rayz rayz commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Remove the ADP-specific data_plane.metrics.v3.series.enabled gate.
  • Use the authoritative typed Agent configuration for metrics protocol and endpoint routing.
  • Match Core Agent behavior for use_v3_api.series.*, including the datadog_only default, per-endpoint overrides, and Datadog URL recognition.
  • Apply the same routing configuration across primary, OPW/Vector, MRF, and autoscaling metrics paths.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

References

@rayz
rayz requested a review from a team as a code owner July 28, 2026 21:14
@dd-octo-sts dd-octo-sts Bot added area/components Sources, transforms, and destinations. area/docs Reference documentation. encoder/datadog-metrics Datadog Metrics encoder. labels Jul 28, 2026

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

Removing the ADP-only gate consistently promotes the existing Agent-compatible V3 series setting, including its default, endpoint overrides, OPW/Vector routing, validation, and zlib fallback. No diff-only behavioral regression was identified; Rust tests could not execute because the sandbox could not fetch the pinned hyper-http-proxy git dependency.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Datadog Autotest · Commit e699c7d · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e699c7dfd0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 476 to 479
let series_mode = metrics_encoder_mode_for_config(
use_v3_series,
series_v3_can_be_enabled,
self.v3_api.series.validate,
metrics_v3_disabled_by_compressor,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep the Cluster Agent failover branch on V2

When autoscaling failover is enabled, add_autoscaling_failover_metrics_pipeline_to_blueprint constructs af_metrics_encode from this same configuration and sends its output through ClusterAgentForwarderConfiguration. Because use_v3_api.series.enabled defaults to true, this change makes that encoder V3-only, while the Cluster Agent mapper rewrites the series request to /series and its contract/test is based on an /api/v2/series payload. The branch therefore sends a V3 wire body where the Cluster Agent expects the V2 MetricPayload, breaking failover metric delivery unless users globally disable V3; use a branch-specific encoder configuration that forces V2 for the Cluster Agent path.

Useful? React with 👍 / 👎.

@pr-commenter

pr-commenter Bot commented Jul 28, 2026

Copy link
Copy Markdown

Binary Size Analysis (Agent Data Plane)

Baseline: 032b9ac · Comparison: 7c71a86 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 40.83 MiB (baseline) vs 40.91 MiB (comparison)
Size Change: +78.23 KiB (+0.19%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
tracing -112.98 KiB 35
saluki_common::resource_tracking::groups +106.28 KiB 23
prost -52.65 KiB 79
hyper +48.21 KiB 92
tonic_prost +45.91 KiB 16
hyper_util -43.39 KiB 10
figment -38.97 KiB 154
tokio +33.78 KiB 1182
saluki_components::common::datadog +31.41 KiB 114
hashbrown +18.33 KiB 139
axum -17.94 KiB 114
anon.1c766fa05dd13eba06da95c69824d6af.995.llvm.16845278109024732812 -17.36 KiB 1
anon.1c766fa05dd13eba06da95c69824d6af.995.llvm.2183139281678763661 +17.36 KiB 1
anon.1c5af32c575c0d14ae778d58023b5abf.0.llvm.10081054402879163701 +17.36 KiB 1
anon.ad88aefe8160b8dedfdf5305e59f030a.513.llvm.6948463008055664385 -17.27 KiB 1
std -14.42 KiB 110
serde_core +14.27 KiB 190
saluki_components::encoders::datadog +14.25 KiB 67
chrono -12.96 KiB 10
tokio_rustls -12.72 KiB 12
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +1.6%  +190Ki  +1.2%  +108Ki    [16255 Others]
  [NEW] +59.6Ki  [NEW] +59.5Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::h9313854c2f2a044f
  [NEW] +32.7Ki  [NEW] +32.4Ki    _<saluki_components::common::datadog::config::_::<impl serde_core::de::Deserialize for saluki_components::common::datadog::config::ForwarderConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::h215858157e52a210
  [NEW] +17.4Ki  [NEW] +17.3Ki    anon.1c766fa05dd13eba06da95c69824d6af.995.llvm.2183139281678763661
  [NEW] +17.4Ki  [NEW]     +76    anon.1c5af32c575c0d14ae778d58023b5abf.0.llvm.10081054402879163701
  [NEW] +13.0Ki  [NEW] +12.8Ki    _<saluki_common::resource_tracking::groups::Tracked<Inner> as core::future::future::Future>::poll::hb237d9782d5fe43d
 +24e2% +12.6Ki +40e2% +12.6Ki    _<saluki_components::transforms::trace_obfuscation::TraceObfuscation as saluki_core::components::transforms::SynchronousTransform>::transform_buffer::h4694da026393f90f
  [DEL] -12.2Ki  [DEL]     -81    anon.ad88aefe8160b8dedfdf5305e59f030a.665.llvm.6948463008055664385
  [DEL] -12.4Ki  [DEL] -12.3Ki    _<chrono::format::formatting::DelayedFormat<I> as core::fmt::Display>::fmt::h46581aaf02582bbc
  [DEL] -13.5Ki  [DEL] -13.4Ki    _<tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::h0a965208a434db48
 -84.0% -13.7Ki -84.8% -13.7Ki    _<hyper_util::server::conn::auto::Connection<I,S,E> as core::future::future::Future>::poll::h5670434c8a709418
 -86.2% -13.8Ki -87.0% -13.8Ki    _<hyper_util::server::conn::auto::Connection<I,S,E> as core::future::future::Future>::poll::he89a2edaa7fcd54b
  [DEL] -14.3Ki  [DEL] -14.1Ki    _<figment::value::de::ConfiguredValueDe<I> as serde_core::de::Deserializer>::deserialize_any::h2e163da9da896fb8
 -85.3% -14.8Ki -86.1% -14.8Ki    _<hyper_util::server::conn::auto::Connection<I,S,E> as core::future::future::Future>::poll::hca913f7324cfdd5b
  [DEL] -14.9Ki  [DEL] -14.8Ki    _<figment::value::de::ConfiguredValueDe<I> as serde_core::de::Deserializer>::deserialize_struct::ha36113e815a9c1ca
  [DEL] -17.0Ki  [DEL] -17.0Ki    prost::message::Message::encode::h28ad969e25341bd5
  [DEL] -17.3Ki  [DEL]     -76    anon.ad88aefe8160b8dedfdf5305e59f030a.513.llvm.6948463008055664385
  [DEL] -17.4Ki  [DEL] -17.3Ki    anon.1c766fa05dd13eba06da95c69824d6af.995.llvm.16845278109024732812
  [DEL] -19.6Ki  [DEL] -19.5Ki    saluki_components::transforms::apm_stats::ApmStats::process_trace::h0c7467fe110ac0a8
  [DEL] -24.3Ki  [DEL] -24.2Ki    saluki_components::transforms::trace_obfuscation::TraceObfuscation::obfuscate_span::hca56db06df8fb52f
  [DEL] -59.9Ki  [DEL] -59.7Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::h3e35c9d992da4100
  +0.2% +78.2Ki  +0.0% +8.08Ki    TOTAL

@pr-commenter

pr-commenter Bot commented Jul 28, 2026

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: 12e4057f-2435-408d-987b-f0d4a2ca9e3a
Baseline: 032b9ac5 · Comparison: 7c71a86c · diff

Optimization Goals: ❌ 2 regressions detected

experiment goal Δ mean % links
quality_gates_rss_dsd_heavy (erratic) memory 🔴 +216.55 metrics profiles logs
quality_gates_rss_dsd_ultraheavy (erratic) memory 🔴 +167.60 metrics profiles logs
Fine details of change detection per experiment (3)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
quality_gates_rss_dsd_medium (erratic) memory ⚪ +3.83 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ +1.98 metrics profiles logs
quality_gates_rss_idle memory ⚪ +0.30 metrics profiles logs
Bounds Checks: ❌ Failed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 0/10 ❌ 469 MiB ≤ 140 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 0/10 ❌ 53.2 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 0/10 ❌ 95.4 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 0/10 ❌ 521 MiB ≤ 200 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 31.4 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Pipelines

Unblock PR with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

DataDog/saluki | run-benchmarks-adp   View in Datadog   GitLab

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7c71a86 | Docs | Datadog PR Page | Give us feedback!

@dd-octo-sts dd-octo-sts Bot added the forwarder/datadog Datadog forwarder. label Jul 28, 2026
@rayz rayz changed the title chore(config): Remove ADP v3 series gate chore(config): make Core Agent config authoritative for V3 routing Jul 28, 2026
@rayz
rayz force-pushed the rayz/remove-adp-v3-series-gate branch from ca37cec to 7c71a86 Compare July 28, 2026 22:35
@tobz
tobz changed the base branch from rayz/fix-scalar-additional-endpoints to graphite-base/2226 July 29, 2026 00:51
@tobz
tobz force-pushed the graphite-base/2226 branch from 93b30d8 to 032b9ac Compare July 29, 2026 00:52
@tobz
tobz changed the base branch from graphite-base/2226 to main July 29, 2026 00:52

tobz commented Jul 29, 2026

Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/components Sources, transforms, and destinations. area/docs Reference documentation. encoder/datadog-metrics Datadog Metrics encoder. forwarder/datadog Datadog forwarder.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants