Skip to content

feat(cli,server): MAN-122 — log a startup banner and periodic status line - #136

Open
catalyst-cloud-connector[bot] wants to merge 1 commit into
mainfrom
MAN-122
Open

feat(cli,server): MAN-122 — log a startup banner and periodic status line#136
catalyst-cloud-connector[bot] wants to merge 1 commit into
mainfrom
MAN-122

Conversation

@catalyst-cloud-connector

@catalyst-cloud-connector catalyst-cloud-connector Bot commented Sep 7, 2026

Copy link
Copy Markdown

Summary

Before this change, manta listen --server-config said nothing on startup and nothing while
running — the first log line ever emitted was a per-connection message, and only once some
client happened to touch a listener. An operator watching stderr (or a journald unit) had no
way to tell the daemon came up correctly, or that it was still alive and decoding, short of
scraping /metrics by hand. This closes MAN-122's two scenarios:

  1. Startup banner — one INFO line, emitted as line 1 of the daemon's log, naming version,
    active source, sample rate, dial frequency, station callsign, and the three real bound
    addresses (telnet/JSON/metrics).
  2. Periodic status line — one rate-limited INFO line per configured interval (default 60 s),
    naming active track count, spots/min, spots total, per-protocol + total connected client
    count, and uplink connection state.

Both go to stderr through the tracing machinery MAN-59 already set up, so --json's stdout
stream (the determinism contract) is untouched.

Sample output (ANSI stripped):

2026-09-07T18:43:14.877308Z  INFO manta: manta 0.1.0 ready: source=file sample_rate_hz=48000 dial_freq_hz=14060000 station=W3XYZ telnet=127.0.0.1:38601 json=127.0.0.1:40259 metrics=127.0.0.1:40107
2026-09-07T18:43:14.879690Z  INFO telnet_client{peer=127.0.0.1:48882}: manta_server::telnet: telnet: client connected
2026-09-07T18:43:15.880213Z  INFO manta_server::status: manta status: uptime_s=1 tracks=0 spots_per_min=0.0 spots_total=0 clients=1 (telnet=1 json=0 ws=0) uplink=none

What changed

  • New manta-server::status module (crates/manta-server/src/status.rs) — the pure,
    synchronous formatting core: format_startup_banner, format_status_line, spots_per_min
    (a windowed derivative, not a lifetime average — answers "is it decoding now"), and
    spawn_status_line, the timer task itself.
  • Five new live-state getters on Metrics (spots_total, active_tracks, and the three
    per-protocol client-gauge readers) — Metrics was already the daemon's one shared,
    synchronously-readable handle on live state, but the only way to read most of it back out was
    string-parsing render_prometheus_text().
  • The banner is wired into start_spot_server (crates/manta-cli/src/main.rs), emitted after
    all three TcpListener::bind calls succeed but before any listener task is spawned — so a bind
    failure never produces a false "ready" line, and no per-connection line can race ahead of it on
    a multi-thread runtime. It reports local_addr(), not the configured port, so a *_port = 0
    (ephemeral) config still names the real bound address.
  • The status task is spawned from the same place, on a new status_interval_secs: Option<u64>
    config key (ServerConfig, backward-compatible default via #[serde(default)]; 0 disables
    the task entirely). It's raced against the shutdown watch channel so it can't emit a line into
    the middle of shutdown drain.
  • manta_active_tracks is no longer a permanent, documented 0. It was frozen because
    manta_engine::listen() never exposed a live track-count hook. Rather than ship a status line
    whose headline field silently lies on a healthy, actively-decoding node, this derives a real
    count from the DecoderEvent stream the CLI's on_event closure already observes — a
    HashSet<u32> of open track ids, keyed on any track-scoped event for insert and TrackClosed
    for remove, with no manta_engine API change. ARCHITECTURE.md §8 is corrected to match.
  • docs/DECISIONS/2026-09-07-man122-operator-liveness-logging.md (new) records the design
    decisions below with their rationale.

Design decisions

  • key=value, single line, no structured tracing fieldsgrep manta status: on a plain
    log yields the whole record, matching MAN-59's existing style.
  • uplink=none is distinct from uplink=disconnected — an operator with no [[rbn_uplink]]
    configured must not read a permanent "disconnected" as a fault. The distinction is computed from
    the count of enabled uplink configs, not inferred from the connected-count metric alone.
    Verified a dry-run uplink still reads connected (the dry-run suppression check is downstream of
    where the gauge is set).
  • "Rate-limited" means exactly one line per interval, full stop — no suppress-if-unchanged. A
    status line whose absence is meaningful (the daemon stopped logging) is more useful than one
    that goes quiet when nothing changes.
  • Default interval is 60 s, not 30 s — the quieter choice for a process meant to run for
    months on a Pi; status_interval_secs makes 30 s (or 0, for silence) a one-line config change.
  • No manta status subcommand, no /healthz, no CPU-percent field, no stdout change in text
    mode, no non-daemon (listen without --server-config) banner
    — all explicitly out of scope;
    see the decision record for why each belongs to a different ticket (MAN-44, MAN-128) or isn't
    backed by any existing accounting in the workspace.

Two real bugs fixed during implementation, beyond the original plan

  • The status task's shutdown-select arm now returns when the shutdown watch::Sender side has
    dropped (changed() returns Err), instead of busy-spinning a core forever — a real
    correctness bug the plan's own sketch would have shipped, and one that matters on this
    project's single-core Pi4 budget.
  • The live-track bookkeeping keys on any track-scoped event (CharDecoded, WordBoundary,
    SpeedUpdate, TrackMeta) for insert, not TrackMeta alone as originally planned — verified
    against TrackManager's has_emitted invariant that every event-emitting track id is
    guaranteed exactly one eventual TrackClosed, so the narrower TrackMeta-only rule would have
    undercounted short-lived tracks, and the shipped rule cannot leak.

Testing

Both ticket scenarios were reproduced live against the real manta binary in the implementation
container (not just asserted by unit test): the banner is confirmed as line 1 of stderr, naming
every field the Then clause requires, with real (non-zero, non-configured) ephemeral ports; the
status line was confirmed rate-limited to one line per configured interval, and tracks= was
confirmed genuinely live (moving 105→108→105 run to run against a synthetic CW replay) rather than
the previously-frozen 0.

Two new test files:

  • crates/manta-cli/tests/startup_banner.rs — end-to-end through the real binary: banner-is-line-1,
    real-port assertion, and a periodic-status-line-appears-during-replay case.
  • crates/manta-server/tests/status_line_acceptance.rs — the timer task really emits, really reads
    Metrics, really rate-limits, using a small custom MakeWriter to capture tracing output.

Plus unit tests for the two new formatters, the rate-arithmetic edge cases (zero window), the five
new Metrics getters, and the track-set bookkeeping (double-insert, unknown-close, non-lifecycle
events are all no-ops).

Full validation (validate-plan) ran the workspace's real CI-equivalent gate per-crate (this
container's disk can't hold every debug test binary from a single cargo test --workspace):
cargo fmt --all --check and cargo clippy --workspace --all-targets -- -D warnings both clean;
475 tests passed, 0 failed, 9 pre-existing ignored across all 9 crates, including the
determinism regression test (json_output_is_valid_and_deterministic_across_three_runs), confirming
stdout in --json mode is untouched. A reward-hacking scan over the diff found no suppressions,
no weakened assertions, and confined every added .unwrap()/.expect() to test code.

Not exercised in this environment

No SDR hardware, no reachable KiwiSDR, and no live RBN uplink endpoint were available in the
implementation or validation containers, so the following are read-verified against source but not
run end-to-end: source=/dial_freq_hz= against a real KiwiSDR/SoapySDR/HPSDR device (the code
path, IqSource::center_freq_hz(), is shared and already exercised by other tests); uplink=
flipping from connected to disconnected against a live target (the underlying gauge transitions
are covered by uplink_acceptance.rs against a mock); and a long-running (24 h) soak to bound-check
the open-track HashSet against leaks (source-level analysis shows it cannot leak — every event an
open track can emit sets has_emitted, and every TrackClosed path gates on it — but a live soak
remains the honest test). None of these are new gaps introduced by this change; they're the same
hardware-dependent gaps CLAUDE.md already tracks against M2/M4 acceptance.

Follow-up (non-blocking, per this repo's review-convergence policy)

Four informational findings from validation, none correctness or security, recorded for a
follow-up ticket rather than fixed inline:

  1. crates/manta-cli/tests/startup_banner.rs's periodic-status-line test derives its timing
    margin from unpaced file-replay speed (240 s fixture, 1 s interval) — measured margin varies
    2.8x–6.2x across containers in this fleet already, so a sufficiently fast CI runner could flip
    it red. Should be rewritten to not depend on replay speed.
  2. The decision record's per-event cost estimate for the track-bookkeeping HashSet predates the
    "any track-scoped event" fix above and should be corrected to reflect that it now runs on the
    CharDecoded hot path (cost is still negligible against per-event DSP work, but the written
    estimate is stale).
  3. spawn_status_line's returned JoinHandle is discarded at its only production call site (it's
    a detached background task, matching the existing reaper-task idiom, and shutdown is covered by
    the select! arm regardless) — the return value exists only for tests today.
  4. status.rs's uptime_s measures time since the status task started, not the process —
    indistinguishable today since the spawn happens microseconds after the daemon's binds, but
    would silently drift if the task were ever spawned lazily.

Scope notes

  • No wire-format change: telnet, JSON Lines, WebSocket, and Prometheus text output are unchanged,
    except that manta_active_tracks now reports real numbers — a correction of previously
    documented-broken behavior, not a contract change.
  • Config is backward compatible: status_interval_secs is optional; existing configs get the 60 s
    default unchanged.
  • Log volume increases from zero to ~1440 lines/day at the default interval — called out in the
    decision record so it isn't a surprise against a journald quota.

d07d835 feat: MAN-122 — An operator should be able to tell manta is alive and decoding from its own log output
044156f feat: MAN-122 — An operator should be able to tell manta is alive and decoding from its own log output

Catalyst-Replay-Squash: 49f05a4
@catalyst-cloud-connector catalyst-cloud-connector Bot changed the title feat: MAN-122 — An operator should be able to tell manta is alive and decoding from its own log output feat(cli,server): MAN-122 — log a startup banner and periodic status line Sep 7, 2026
@catalyst-cloud-connector
catalyst-cloud-connector Bot marked this pull request as ready for review September 7, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants