Agents view can exhaust file descriptors while scanning growing subagent sessions

[VincentBailly]

Summary

When the agents view refreshes while multiple subagents continuously append to their session JSONL files, Prime Agent can consume sustained CPU, accumulate thousands of file descriptors, and eventually make the UI unresponsive.

Observed behavior

On Linux with Node.js 22, one affected process showed:

• roughly 70% sustained CPU on the Node main thread
• approximately 3.5 GB RSS
• over 12,000 open file descriptors, increasing by about 280 per second
• thousands of descriptors pointing to the same six subagent JSONL files
• more than 1 TB of logical reads over several hours

The subagent processes themselves were not consuming significant CPU.

Reproduction

1. Start several long-running subagents that frequently append results to their session files.
2. Keep the agents view open so its session catalog refreshes periodically.
3. Observe the Prime Agent process's CPU and open descriptors.
4. The same growing JSONL files are opened repeatedly, and the UI progressively becomes unresponsive.

Root cause

readSessionInfo() records the file size with stat(), but previously opened an unbounded createReadStream().

For an actively growing file, the reader can continue following the moving EOF instead of completing. Because the metadata cache only stores completed scans, subsequent catalog refreshes start additional scans of the same file. This creates a feedback loop of overlapping streams, repeated parsing, and leaked descriptors.

Expected behavior

Each metadata scan should read a fixed snapshot of the file, and concurrent requests for the same snapshot should share one scan.

Proposed fix

• Bound each stream to the file size captured before scanning.
• Coalesce concurrent scans of an identical snapshot.
• Queue a follow-up scan when the file changes during an active scan, avoiding stale metadata and parallel readers.

I implemented and tested this in my fork:
VincentBailly#2

[sankks11]

I reproduced the same failure shape on Prime Agent 0.8.1 and confirmed it is an unbounded in-flight read stampede rather than a writer that permanently forgets to close a descriptor.

During a depth-2 RLM workload, one isolated worker reached:

• 20,928 open FDs; 20,781 were reads of nine child JSONLs and 17 rlm-subagent.json files
• 17,905 JSONL descriptors, with 16,457 below EOF and every sampled offset aligned to the 64 KiB createReadStream chunk size
• 7.4–7.6 GiB RSS and roughly four CPU cores
• 21.8 GB of additional logical reads in 157 seconds, while physical read_bytes stayed nearly flat

Handles advanced, closed, and were reused, but new scans were admitted faster than they completed. Attach timed out. One named rlm() admission took 7m26s before returning its handle; the next remained blocked for more than 18 minutes until manual abort.

The current path is:

1. each streamed child delta emits rlm_child_update;
2. the supervisor starts an uncoalesced worker-summary refresh for every update;
3. each worker list rebuilds passive RLM summaries and reads every display file and child transcript;
4. readSessionInfo() caches only completed scans, so concurrent misses each open a full createReadStream.

This independently validates the snapshot/single-flight direction in #1535. It also supports bounding the caller: allow at most one worker-summary refresh in flight, retain one dirty/trailing refresh, skip resident edges before transcript I/O, and single-flight readSessionInfo().

The affected journals contain private data, so I am providing only aggregate /proc and source-path evidence.