Skip to content

Use irtt streaming mode (-s) for unbounded pinger sessions - #395

Closed
ooonea wants to merge 1 commit into
lynxthecat:masterfrom
ooonea:feat/irtt-streaming-pinger
Closed

Use irtt streaming mode (-s) for unbounded pinger sessions#395
ooonea wants to merge 1 commit into
lynxthecat:masterfrom
ooonea:feat/irtt-streaming-pinger

Conversation

@ooonea

@ooonea ooonea commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Closes #394 — and, per the review discussion, gawk is now gone entirely: one irtt client -s -r --loose -i <interval> per reflector, parsed in the main loop like every other pinger method.

The raw stream carries neither the reflector nor a wall-clock timestamp, all pingers share the one response fd, and irtt client takes exactly one host per session (irtt_client.go:284) with no flag to tag its output — so a per-reflector plain-bash helper prefixes those two fields onto each line and handles the respawn (a server-side max duration clamps the negotiated session and the client just exits; rate-limited to one relaunch per second on quick deaths).

Per-line gating, now in the extraction arm: numeric rd/sd via glob class checks (no [[ =~ ]] on the hot path). That one check drops NaN OWDs (no server timestamps), duplicates (dup=1 lines carry NaN OWDs by construction) and negative OWDs (client and server clocks out of step — which to_us() used to reject, and the interim transcoder let through). ipdv is legitimately NaN on the first reply of every session and on seqno wrap (heistp confirmed the wrap behavior in heistp/irtt#45), so it never gates a sample. ms→µs uses the same printf %.3f idiom as the fping arm.

--loose: without it, any server-imposed restriction (e.g. a public server's min send interval) kills the client pre-test at every respawn — [ServerRestriction] server increased interval from 100ms to 1s, exit 1, zero samples, silent spin. Reproduced both ways against irtt server -i 1s.

Removed: the gawk + time-extension dependencies, the -d session bound, the WaitForPackets match, the unit-suffix parsing, and stdbuf (raw lines come from Go's unbuffered stdout — the old stdbuf -oL never had an effect for the same reason). In raw mode irtt routes events/notices to stderr, so stdout carries only sample lines. ping_prefix_string now wraps the irtt client itself, as in the other arms (it used to wrap gawk; the popen'd irtt never carried the prefix).

Validation: the gawk checks are gone; a time-bounded probe of irtt client -h checks for streaming-mode support — -s only exists since irtt 0.9.2, and without the probe an older irtt would reject -s at every respawn while the daemon silently produces no samples (min-rate pinning). No pipe to grep (the help goes to stderr and exits 2, which pipefail would turn into a false failure); read -t bounds it so a never-exiting irtt can't hang startup.

irtt_session_duration_m: deprecated-and-ignored rather than dropped, since dropping the key would break existing config files that set it — happy to drop outright instead, your call from the issue.

Also worth noting from heistp/irtt#45: a 1-hour 1 ms-interval stream (3.6M round trips) shows no leak, RSS stable ~12.5 MB.

Tested on a loopback rig (real irtt 0.9.2 server, wrapper and parser blocks extracted verbatim from the file): 9-field lines end-to-end, µs values matching irtt's own rd/sd within rounding, first sample kept, dup/NaN/negative/short/long lines dropped, respawn paced ~1/s across server restarts, group kill leaves no orphans, stdout stays pure across a server bounce, capability gate passes on 0.9.2 / fails on older / fails-not-hangs on a stuck binary; shellcheck finding count identical to master. Hot-path cost measured over 1M iterations: extraction arm 5.8→20.8 µs/sample (the ms→µs conversion moved in — same class of work the fping arm already does), helper 9.6 µs/line in its own background process vs 4.2 for the gawk stage; at -i 0.3s × 6 pingers that is under 0.1% of a core either way. Net diff vs master: −5 lines.

@lynxthecat

lynxthecat commented Aug 4, 2026

Copy link
Copy Markdown
Owner

But since irtt now outputs a stream like ping, tsping and fping, can't we forego gawk altogether and do what we do with ping, tsping and fping?

Why can't we do something much simpler like:

		fping)
			${ping_prefix_string} fping ${ping_extra_args} --timestamp --loop --period "${reflector_ping_interval_ms}" --interval "${ping_response_interval_ms}" --timeout 10000 "${reflectors[@]:0:${no_pingers}}" 2> /dev/null >&"${main_fd}" &
			pinger_pids[0]=${!}
			proc_pids['fping_pinger']=${pinger_pids[0]}

I'd like to dispense with gawk assuming it is no longer necessary and bring irtt in line with the ping, fping and tsping implementations.

@lynxthecat
lynxthecat self-requested a review August 4, 2026 12:23
@lynxthecat lynxthecat added the enhancement New feature or request label Aug 4, 2026
Replace the bounded-session irtt pinger (10-minute sessions relaunched
back-to-back, human-output parsing, gawk transcoding) with irtt >= 0.9.2
streaming raw mode (-s -r): one indefinitely-lived client per reflector
emitting fixed-unit machine-readable samples, parsed in the main loop
like every other pinger method.

gawk is gone entirely. The raw stream carries neither the reflector nor
a wall-clock timestamp, and all pingers share one response fd, so a
plain-bash helper prefixes both onto each line and respawns the client
(a server-side max duration clamps the negotiated session and the
client then just exits; respawn is rate-limited to one relaunch per
second on quick deaths). The client runs with --loose: a public
server's restrictions (e.g. a min send interval) otherwise kill the
client pre-test on every respawn, spinning forever with zero samples.
The main-loop extraction arm gates on numeric rd/sd with glob class
checks -- dropping NaN OWDs (no server timestamps), negative OWDs
(client and server clocks out of step, which the old to_us() also
rejected) and duplicates (dup=1 lines carry NaN OWDs by construction)
-- and converts ms to us with the same printf %.3f idiom as the fping
arm.

Gone: the gawk + time-extension dependencies, the -d session bound, the
WaitForPackets end-of-session match, the unit-suffix parsing, and
stdbuf (raw lines come from Go's unbuffered stdout; the old stdbuf -oL
never had an effect for the same reason). In raw mode irtt routes all
event/notice output to stderr, so stdout carries only sample lines.
ping_prefix_string now wraps the irtt client itself, as in the other
pinger arms; previously it wrapped gawk and the popen'd irtt never
carried the prefix.

Validation drops the gawk checks and probes the client help text for
streaming-mode support, time-bounded, so an irtt too old for -s (or one
that never exits on -h) is named up front instead of silently producing
no samples and pinning CAKE at the min rates. irtt_session_duration_m
is deprecated and ignored, retained so existing config files remain
valid.

Assisted-by: Claude Code
@ooonea
ooonea force-pushed the feat/irtt-streaming-pinger branch from 2e44368 to 6736fc2 Compare August 4, 2026 19:01
@ooonea

ooonea commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Done — gawk is gone entirely; the extraction arm now parses the raw stream like ping/tsping/fping (glob class checks, ms→µs via the same printf %.3f idiom as the fping arm). PR body updated to match.

One wrinkle kept me from dropping the helper process too: the raw lines carry neither the reflector nor a timestamp, all pingers share the one response fd, and irtt client takes exactly one host per session (irtt_client.go:284) with no flag to tag its output — so a per-reflector plain-bash loop still prefixes those two fields and handles the respawn. Plain bash, no gawk, no time extension.

Testing against a real 0.9.2 server surfaced two fixes now included: --loose (without it a server restriction, e.g. a public server's min send interval, kills the client pre-test at every respawn — silent zero-sample spin), and the glob gate rejecting negative rd/sd (unsynced clocks), which to_us() used to drop but the interim transcoder let through.

Hot-path numbers are in the PR body; short version: under 0.1% of a core either way at -i 0.3s × 6 pingers, and the diff vs master is now −5 lines with no non-ping dependency left on the irtt path.

@ooonea ooonea closed this Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Leverage new irtt streaming mode (-s).

2 participants