Skip to content

fix(topology): preserve component failure details - #26549

Open
gwenaskell wants to merge 9 commits into
masterfrom
yoenn.burban/socket-bind-error-context
Open

gwenaskell wants to merge 9 commits into
masterfrom
yoenn.burban/socket-bind-error-context

Conversation

@gwenaskell

@gwenaskell gwenaskell commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Preserve runtime errors returned by sources and sinks in topology task failures. Existing components that return only () still produce an opaque task error. Bind failures from TCP/UDP socket, syslog, and StatsD sources, and WebSocket server and Prometheus exporter sinks, now include their causes. Console encoding and write errors, Kafka and MQTT subscription errors, journald checkpoint open errors, and AMQP consumer errors also retain their causes.

Log string panic payloads in a structured panic field when a source, transform, or sink task panics. Concurrent transform child panics reach the same handler. The panic payload does not enter the log message or shutdown reason.

Example runtime log

With a TCP socket source named in whose port is already held by another listener, the topology emitted this line in the regression test on macOS:

2026-10-13T12:55:13.816424Z ERROR source{component_kind="source" component_id=in component_type=socket}: vector::topology: An error occurred that Vector couldn't handle: TCP bind failed: Address already in use (os error 48).

The OS error number and text vary by platform.

A remap transform named grok, fed the message foo=NaN, also demonstrates the structured panic field:

pattern = "%{data::keyvalue}"
.result = parse_groks!(.message, patterns: [pattern])

The topology emitted this line:

2026-10-13T13:07:04.603990Z ERROR transform{component_kind="transform" component_id=grok component_type=remap}: vector::topology: An error occurred that Vector couldn't handle: the task panicked and was aborted. panic="not a float: FloatIsNan"

This reproduction uses parse_groks!, which runs the Datadog Grok keyvalue filter. parse_grok! uses standard Grok, where %{LOGLEVEL:keyvalue} names a capture.

References

None.

Vector configuration

Regression tests build topologies with TCP/UDP socket, syslog, and StatsD sources, and WebSocket server and Prometheus exporter sinks, while another listener holds the configured address. A console sink unit test closes its output pipe to check that the sink retains the original I/O error.

How did you test this PR?

  • cargo test --no-default-features --features sources-socket,sinks-socket,sinks-websocket_server,test-utils --lib topology::test::crash:: -- --nocapture --test-threads=1 (7 passed)
  • cargo test -p vector --no-default-features --features sources-socket,sources-syslog,sources-statsd,sources-kafka,sources-mqtt,sources-journald,sources-amqp,sinks-socket,sinks-websocket_server,sinks-prometheus,test-utils --lib topology::test::crash:: -- --test-threads=1 (10 passed)
  • cargo test --no-default-features --features sources-socket,sinks-socket,sinks-websocket_server,test-utils --lib topology::test::panic:: -- --nocapture (2 passed)
  • cargo test --no-default-features --features sources-socket,sources-syslog,sinks-socket,sinks-websocket_server,test-utils --lib test_udp_syslog_bind_error_is_reported -- --nocapture (1 passed)
  • cargo test -p vector --no-default-features --features sinks-console write_error_retains_cause -- --nocapture (1 passed)
  • cargo test -p vector --no-default-features --features sinks-console sinks::console::sink::test::component_spec_compliance (1 passed)
  • FEATURES=sources-socket,sinks-socket,sinks-websocket_server,test-utils make check-clippy
  • FEATURES=sinks-console make check-clippy
  • FEATURES=sources-socket,sources-syslog,sources-statsd,sources-kafka,sources-mqtt,sources-journald,sources-amqp,sinks-socket,sinks-websocket_server,sinks-prometheus,test-utils make check-clippy
  • make check-clippy (all features)
  • Blackhole, Sematext logs, and Humio metrics smoke tests with sinks-blackhole,sinks-humio,sinks-sematext,aws-core,test-utils (3 passed)
  • Built Vector with sources-stdin,transforms-remap,sinks-console and ran the foo=NaN remap configuration above; verified the topology log includes the panic field
  • make check-fmt
  • make check-changelog-fragments
  • make check-markdown

Does this PR include user facing changes?

  • Yes. Added a changelog fragment.
  • No.

@github-actions github-actions Bot added domain: topology Anything related to Vector's topology code domain: sources Anything related to the Vector's sources domain: core Anything related to core crates i.e. vector-core, core-common, etc labels Oct 5, 2026
@gwenaskell
gwenaskell force-pushed the yoenn.burban/socket-bind-error-context branch from 1aada76 to c7982e2 Compare October 12, 2026 12:58
@github-actions github-actions Bot added the domain: sinks Anything related to the Vector's sinks label Oct 12, 2026
@gwenaskell gwenaskell changed the title fix(source): preserve failure details in task errors fix(topology): preserve component failure details Oct 12, 2026
@gwenaskell
gwenaskell marked this pull request as ready for review October 12, 2026 15:25
@gwenaskell
gwenaskell requested a review from a team as a code owner October 12, 2026 15:25
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T13:02:44.930330Z 0486b9a New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadoghq-integration datadoghq-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The completed static review found the error adapters consistent with existing failure and shutdown behavior. Detailed causes are retained, while panic payloads remain outside shutdown reasons.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit a676693

@gwenaskell
gwenaskell requested a review from bruceg October 13, 2026 13:15
@tessneau
tessneau self-requested a review October 13, 2026 16:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: sinks Anything related to the Vector's sinks domain: sources Anything related to the Vector's sources domain: topology Anything related to Vector's topology code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant