Skip to content

Runtime metrics ignore DD_DOGSTATSD_URL / DD_DOGSTATSD_SOCKET and are silently dropped on UDS-only setups #6107

Description

@ollym

Current behaviour

Datadog::Core::Metrics::Client#default_statsd_client always constructs the statsd client with explicit positional arguments:

Datadog::Statsd.new(default_hostname, default_port, **options)

where default_hostname/default_port come from DD_AGENT_HOST / DD_METRIC_AGENT_PORT, defaulting to 127.0.0.1:8125. Passing explicit arguments bypasses dogstatsd-ruby's own environment resolution (ConnectionCfg), so DD_DOGSTATSD_URL and DD_DOGSTATSD_SOCKET are ignored entirely for runtime metrics.

On UDS-only setups this makes runtime metrics vanish silently. The concrete case where we hit this: pods instrumented by the Datadog Admission Controller, which injects DD_DOGSTATSD_URL=unix:///var/run/datadog/dsd.socket and DD_TRACE_AGENT_URL=unix:///var/run/datadog/apm.socket but no DD_AGENT_HOST. Traces flow fine (the tracer honors DD_TRACE_AGENT_URL), but every runtime metric goes to UDP 127.0.0.1:8125 inside the pod, where nothing listens — no error, no warning, no metrics.

Reproduction (datadog 2.38.0, dogstatsd-ruby 5.7.1, same behaviour on master):

ENV['DD_DOGSTATSD_URL'] = 'unix:///var/run/datadog/dsd.socket'
require 'datadog'
require 'datadog/statsd'

client = Datadog::Core::Metrics::Client.new(telemetry: nil)
client.statsd
# => UDP transport to host="127.0.0.1" port=8125  (DD_DOGSTATSD_URL ignored)

Datadog::Statsd.new(single_thread: true)
# => UDS transport to "/var/run/datadog/dsd.socket"  (dogstatsd-ruby's own env resolution)

Expected behaviour

DD_DOGSTATSD_URL (and DD_DOGSTATSD_SOCKET) should be usable for runtime metrics without also setting DD_AGENT_HOST, consistent with dogstatsd-ruby's documented resolution ("DD_DOGSTATSD_URL has priority on other environment variables") and with how the admission controller configures workloads.

Workarounds

  • set DD_AGENT_HOST (e.g. downward API status.hostIP) so the UDP path reaches a node agent, or
  • pass a manually constructed client: c.runtime_metrics.statsd = Datadog::Statsd.new(single_thread: true).

I have a PR ready that delegates transport resolution to dogstatsd-ruby (>= 5.6, which introduced DD_DOGSTATSD_URL support) when those variables are set and no explicit DD_AGENT_HOST/DD_METRIC_AGENT_PORT is configured — deliberately conservative so existing host/port setups keep today's behaviour.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions