[SPIKE] Connections held beyond their liveliness

[mattyg]

Extracted from holochain/holochain#5376

I've seen some unexpected behavior around connections. Holochain's network metrics show an open connection long after the other conductor has been killed. We should review this behavior and either confirm it is expected and document, or create issues to resolve it.

There may be others, but these are the ones I noticed:

Behavior 1:

• A connection stays open indefinitely after receiving a webrtc event that the connection status changed to disconnected, failed or closed.
• This behavior didn't occur in datachannel, there appears to be inconsistency in the datachannel vs pion implementation.
• Potentially fixed by fix: drop connection on pion event that PeerConnectionState has changed to disconnected, closed, or failed #194

Behavior 2:
After a connection is closed, the connection is almost immediately re-established with webrtc: false. This can be reproduced by:

• launch 2 conductors using pion transport
• wait for them to establish a webrtc: true connection
• kill one conductor
• wait for the live conductor to remove the connection
• after a few more seconds, the connection is re-added, with webrtc: false

AC:

• Review behaviors above and confirm if behavior is expected
• If behavior is expected, document it
• If behavior is unexpected, create issues to resolve, or resolve it if spike time permits

[EthosengineBot]

Confirming Behavior 1 ("connection stays open indefinitely after a webrtc disconnected/failed/closed event") is a real, production-impacting off-heap memory leak, not just a stale liveliness metric — and that #194 fixes it.

On Holochain 0.6.x / tx5 0.8.1 the conductor's go-pion child grows Go-runtime memory monotonically to OOM (Rust [heap] flat, other anon climbing GBs). Cause: at v0.8.1 go_pion.rs ignored the state event (Evt::State(_) => ()), so dead-peer PeerConnections (ICE/DTLS/SCTP) are never peer_con_free'd; idle dead peers are never reaped because the only teardown path needs an active send. We built tx5 with #194+#199 (go-pion backend) and confirmed it — the teardown tests conn_dropped_on_peer_connection_state_{disconnected,closed,failed} pass with the fix and time out without it.

#194/#199 merged after the v0.8.1 tag and aren't in any release, so anyone pinned to tx5 0.8.1 (Holochain 0.6.x) is still affected. Given this spike's AC ("create issues to resolve"), would it make sense to (a) cut a tx5 patch release bundling #194/#199, and (b) track the remaining liveness hardening (the peer_map cleanup in #207)? We're carrying #194/#199 in a downstream fork in the meantime.

[EthosengineBot]

Correction to my earlier comment (#issuecomment-4732919972). I confirmed Behavior 1 as a real production off-heap memory leak and asked whether a tx5 patch release of #194/#199 was warranted to resolve it. The memory-OOM part of that was mistaken.

We deployed #194 + #199 fleet-wide (binary-verified) and the conductor's off-heap OOM persisted unchanged — so the zombie-PeerConnection mechanism was not the cause of our OOM. Reading per-VMA smaps, the growing anon is in glibc malloc secondary arenas (the 0x77xx mmap region) serving the conductor's native Rust/C allocations; the Go heap was flat ~52 MB throughout. (My earlier "flat Rust heap ⇒ Go runtime" read was wrong — glibc arenas are anon mmap classified other, and Rust defaults to glibc malloc, so off-heap anon is expected for Rust/C.) Switching the conductor's global allocator to jemalloc (unprefixed-malloc interposition for the C side) flattened it for many hours past the prior cadence — consistent with glibc retaining freed memory, not a never-freed tx5 leak.

To be clear, this doesn't make Behavior 1 a non-issue — connections lingering after a disconnected/failed event still looks like a genuine liveliness problem, and #194/#199 are a worthwhile dead-peer-reaping fix on their own merits. I just no longer have a memory-OOM argument for prioritizing a release; I defer entirely to your judgment there. We'll keep carrying #194/#199 downstream for connection hygiene.