Skip to content

cgka-engine: transport_group_id_index grows unbounded — never pruned on group leave/removal/wipe, and routing rotation is additive (inbound-routing hot-path memory growth) #896

Description

@erskingardner

Severity: LOW
Component: crates/cgka-engine

Summary

Engine::transport_group_id_index (HashMap<Vec<u8>, GroupId>) is insert-only. There is no .remove(...)/.retain(...)/.clear(...) anywhere in the crate, so it accumulates entries with no ceiling on a long-running device. It sits on the per-inbound-event routing hot path.

Location

  • crates/cgka-engine/src/engine.rs:206 — field definition.
  • Inserts: engine.rs:718, engine.rs:756, group_lifecycle.rs:302, group_lifecycle.rs:632.
  • Read on the inbound routing path: message_processor/ingest.rs:1746 (group_id_for_transport_group_id).
  • engine.rs:686reindex_transport_group_id is documented as additive on routing rotation: prior nostr_group_id → group_id entries are retained for the overlap window and never expire.

grep -rn "transport_group_id_index" crates/cgka-engine/src | grep -E '\.remove|\.retain|\.clear' returns nothing.

Two independent growth vectors

  1. Routing rotation: each reindex_transport_group_id adds a new transport_group_id → group_id mapping and keeps the prior one (overlap window), which never expires. A single long-lived group that rotates its Nostr routing component repeatedly accretes stale entries forever.
  2. Group churn: no engine path removes a group's entries on leave / self-eviction / removal / wipe. realize_self_eviction and the removal seams set removed = true but keep the index entries; do_join_welcome only re-inserts.

Impact

Steady, unbounded memory growth on always-on clients that churn groups or rotate routing. Stale mappings also outlive a group whose local record was later deleted at the storage layer (delete_local_group_data, #756); an inbound event on the stale routing id then resolves to a now-deleted group id and does an extra get_group miss per event on the hot inbound-routing path (wasted work, no crash).

Suggested fix

Prune the account's transport_group_id_index entries on group leave/removal/wipe (mirror the group-lifecycle removal seams), and drop superseded routing mappings once the rotation overlap window closes.

Dedup

Same fix shape as #868 (EngineMetrics::last_applied grows one entry per group, never pruned), but a distinct site and trigger: #868 is diagnostic telemetry keyed by group_id on the metrics path; this is a routing-critical index keyed by transport_group_id on the inbound hot path, with a second growth vector (routing rotation) that has no analogue in #868. No open/closed issue covers transport_group_id_index.


Filed from a meticulous automated code-review pass; verified against source and deduplicated against the existing open/closed issue set.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions