Metrics

Counters for a session and for a listener, grouped the way options are: what every transport has lives in common, and each transport gets its own group.

void SampleSession(PeerSession& session) {
  SessionMetrics m = session.metrics();

  // any transport
  uint64_t sent = m.common.messages_sent;
  uint64_t refused = m.common.send_failures;  // the transport dropped it

  // `connection_type` says which per-transport group carries real values
  if (m.connection_type == ConnectionType::ZDT) {
    uint32_t rtt_us = m.zdt.srtt_us;
    uint32_t window = m.zdt.cwnd;  // far below the cap means congestion control
    uint64_t retransmits = m.zdt.retransmits;
  } else {
    uint64_t writes = m.tcp.writes;
  }
}

void SampleServer(Server& server) {
  // ZDT listeners only: a TCP one tracks none of this and comes back zeroed
  ServerMetrics s = server.metrics();
  uint64_t accepted = s.connections_accepted;
  uint64_t rejected = s.zdt.cookies_rejected;
}

Groups that do not apply stay zeroed rather than holding garbage, so reading the wrong one is harmless, unlike a union, which would be undefined. A zero in m.zdt on a TCP session means "not that transport", not "nothing happened", which is what m.connection_type is there to disambiguate.

Pull, not push

metrics() takes a snapshot. The hot path only bumps plain members owned by the thread that already owns the object: no atomics, no locks, no per-packet callback. That is what makes them cheap enough to leave on.

The consequence is that you must sample on a timer, not per packet. Reading them in a hot loop costs more than the counting does. A second is a reasonable interval; anything under 100 ms is measuring your own sampling.

Values may be slightly stale, since the owning thread may be mid-update. For counters that does not matter. For the sampled gauges below it means an occasional reading that lags reality by a tick.

The same choice means reading them from a thread other than the session's owner is a data race, however harmless the result looks. Sample from inside a handler or a tick callback; see Threading Model. Server::metrics() is the exception: it takes the listener's lock, so any thread may call it.

Build with -DZNET_ENABLE_METRICS=OFF to compile them out entirely. The accessors stay, so calling code still compiles; metrics() returns a zeroed struct.

Common counters

Counter	Meaning
messages_sent / messages_received	Whole messages
message_bytes_sent / message_bytes_received	On the wire: after compression and encryption, before transport framing. Received is counted as the session pulls it from the transport, which it only does once a codec and handler are installed
payload_bytes_sent / payload_bytes_received	The serialized packet, before compression and encryption. Received only counts what reached a handler
wire_bytes_sent / wire_bytes_received	Including transport framing
send_failures	The transport refused a message that was already encoded
invalid_frames	Inbound frames that failed to decode. PeerSession::invalid_frames() returns the same count and survives ZNET_ENABLE_METRICS=OFF. See max_invalid_frames in Configuration Reference
outbound_queued	Current queue depth. Sampled, not accumulated

Both pairs are measured at the same two stages on either end, so one peer's message_bytes_sent matches its peer's message_bytes_received for the same message, and the same holds for the payload pair. The gap between the two pairs is what compression and encryption cost, so payload_bytes_sent against message_bytes_sent is the compression ratio the session actually achieved.

send_failures is the one to alert on, and it counts what was lost rather than what was pushed back. SendPacket refusing a full queue is not counted here: it returns Result::QueueFull, the caller still holds the packet, and nothing is gone. A message counted here was encoded and then dropped, because the transport had closed, because the frame was empty or over TCP's size limit, or because ZDT's own send queue was full. Nothing in the send path reports that back to you, so this counter is the only sign.

outbound_queued sitting near send_queue_capacity is the sign of the other problem, the application producing faster than the link drains.

The gap between message_bytes_sent and wire_bytes_sent is the transport's overhead: framing, headers, retransmits.

TCP counters

Counter	Meaning
writes	Buffers written whole, message frames and the transport's own control frames alike. A partial write is retried rather than counted again, so this is not a send() call count
reads	recv() calls that returned data

ZDT counters

Counter	Meaning
datagrams_sent / datagrams_received	UDP datagrams, not messages
retransmits	Reliable datagrams sent again
tail_probes	Resends fired because acks went silent, before any NAK or timeout could. A lost burst tail cannot be NAKed, so the probe breaks that stall
naks_sent / naks_received	Gaps reported to the peer, and by the peer
duplicates_dropped	Deduped by the receiver
inbound_dropped	Inbox was full, arrivals discarded
reassemblies_dropped	Incomplete, timed out, or over the cap

And these gauges, all sampled rather than accumulated:

Gauge	Meaning
srtt_us	Smoothed round-trip estimate
rtt_min_us	Windowed minimum, the baseline congestion is judged against
rto_us	Current retransmit timeout
cwnd	Congestion window, in datagrams
in_flight	Unacked reliable datagrams
mtu	What the handshake settled on

Reading them together

Is the link the bottleneck, or am I? Compare cwnd against max_datagrams_in_flight. A window sitting far below the cap means the controller is holding it there. A window at the cap with in_flight also at the cap means you are limited by the setting, not the network.

Is it congestion or corruption? ZDT backs off on queueing delay, so read srtt_us against rtt_min_us: a smoothed estimate well above the minimum is a queue building somewhere on the path. Retransmits with srtt_us near rtt_min_us is loss without queueing: a lossy link rather than a full one.

Am I dropping on the floor? inbound_dropped means arrivals were discarded because the inbox filled, which means the session is not being serviced fast enough. Usually a handler doing too much work on the session's thread; see Threading Model.

All of the above is ZDT. A TCP session populates common and tcp only, and has no window, no RTT estimate and no retransmit counters of its own, because the kernel owns all three.

Server counters

Server::metrics() is filled by the ZDT backend only. A TCP listener tracks none of it and returns a zeroed ServerMetrics, connection_type included, so per-session counters are all a TCP server has.

Counter	Meaning
connection_type	Which backend filled the struct
connections_accepted	Total ever accepted
connections_active	Currently held. Sampled, refreshed as connections are harvested

The rest live on the listener rather than on a session because ZDT rejects connections before a session exists:

Counter	Meaning
handshakes_started	First contact from a new address
handshakes_rejected	Version mismatch, server full, banned
cookies_rejected	Failed the return-routability check
rate_limited	Per-source handshake cap hit
datagrams_unroutable	Online datagram from an unknown peer
admission_rejected	Dropped by the allow/deny lists or the attempt throttle. See Configuration Reference

cookies_rejected and rate_limited rising together is the signature of a spoofed-source flood being turned away, which is what those mechanisms are for. Rising datagrams_unroutable on its own more often means peers whose NAT mapping changed than an attack.

Relay counters

RelayServer::metrics() returns a RelayMetrics, zeroed with metrics compiled out. The struct lives in znet/p2p/relay_server.h rather than metrics.h, since it is experimental with the rest of the relay. Peer-to-Peer says what an allocation and a bind are.

Counter	Meaning
allocations_active	Live allocations, bound or not. Sampled
allocations_total	Ever allocated
allocations_expired	Freed unbound after bind_timeout, idle after idle_timeout, or by Free()
binds_accepted	Sources that presented the right token to a free slot
binds_refused	Wrong token, or both slots already taken
datagrams_relayed, bytes_relayed	Forwarded between bound sources
datagrams_dropped	An unknown channel, a source that never bound it, or before the other side did
probes_answered	Reflect probes answered
probes_refused	Too short to be safe against amplification, or over the per-source throttle

allocations_total rising with allocations_expired close behind and binds_accepted at about twice the difference is the healthy picture: every pairing got a relay, most punched direct and let it expire, and the rest bound it. binds_refused rising on its own means someone is guessing tokens.