fix: correctness, resilience, and transport-safety bugs in the serve path#24
Open
ruminoob wants to merge 45 commits into
Open
fix: correctness, resilience, and transport-safety bugs in the serve path#24ruminoob wants to merge 45 commits into
ruminoob wants to merge 45 commits into
Conversation
Eagle/chain diagnostics
fix(ring): break tail bring-up handshake deadlock
Feat/topology select ring
The measured per-hop bottleneck is moving the bf16 activation over WAN (a 120KB round-trip is 6-8x the connect-ping; transfer dominates, not relay/compute). Transporting the inter-stage hidden state as fp8 (e4m3, per-tensor scale to keep the residual-stream outlier channels in range) halves bytes/hop -> ~2x tok/s on a transfer-bound ring. Opt-in: the bf16 path is byte-for-byte unchanged when off. Lossy but deterministic + verifiable: receipts hash the fp8 engine's activations, so trustless verification is intact (it just attests an fp8 engine, not bf16-exact). Quality is a measured question (residual-stream outliers can bite) -> validate with a bf16-vs-fp8 A/B on a real ring before claiming "usable M2.5 at 2x". - transport: register float8_e4m3fn/e5m2 in the codec dtype table - m25_pipe: _pack_h/_unpack_h + _hsend/_hrecv wrap the 5 activation wire sites - scatter launcher passes M25_FP8_WIRE through to stages Offline-validated: codec round-trips fp8 bit-exact, bytes exactly 0.50x, 2.26% per-hop quant error on an outlier-heavy activation (per-channel scaling is the tightening lever if the A/B shows quality loss).
…placement Consolidate the 2026-07-01 state onto master: everything (handshake, select_ring, EAGLE-chain, fp8) is now on master, branches deleted. Records: the measured fp8 A/B (+9% on high-bw vast, ~2x is the residential bytes-bound regime; quality preserved but not bit-exact); the 3-agent residential-bandwidth research (the bind is the sender's UPLOAD; decode survives, long-context prefill is the wall on cable/DSL but fine on fiber; you can't conjure upload, the fix is selection + use the download direction + fewer bytes/hops); and the admission-vs-placement decision (coarse proven floor in c0mpute + per-role capability matching in select_ring, not a join threshold). Next: make select_ring upload-aware, then the validation + residential-throttle runs.
RESUME HERE was current but the lower sections lagged: ROADMAP leyten#1 still said 'vLLM tree GO/NO-GO NEXT' (superseded), PROVEN lacked select_ring/fp8/residential, IN-FLIGHT still said 'reconfirm EAGLE accept'. Fixed: ROADMAP redirects to the RESUME HERE next-actions; PROVEN adds select_ring, fp8, residential-bottleneck rows and marks EAGLE merged; IN-FLIGHT marks EAGLE confirmed+merged.
select_ring minimized decode step-time only, blind to the residential wall: prefill ships the [S,H] activation (~100MB/hop @16k) over home uplinks and the bind is the SENDER'S UPLOAD, so a slow-cable node on a load-bearing hop costs minutes of TTFT while decode's tiny activation survives. The objective is now total request time (prefill + D*decode) with per-node upload a first-class, sender-bound cost. Prefill transport is the engine's chunked+ pipelined makespan (sum_fwd(u)+(C-1)*max_fwd(u))/C, interpolating serial-blob SUM and bottleneck MAX. The selector tails the lowest-upload node (the tail forwards nothing), drops nodes whose uplink would dominate prefill, and relegates them to off-critical roles (weight-seeder / aggregator / hot-standby / decode-only-replica / spot-check-verifier) derived from the drop reason, not a hard Mbps gate -- the placement half of admission-vs-placement. Stays pure; byte-identical to the old decode-only path when up_mbps is omitted (golden-snapshot tested). Validated offline (scratchpad/sim_network.py, volunteer/residential pool): aware/ oracle ~0.98 across context while upload-blind collapses 0.98->0.80 as context grows; TTFT speedup 2.5-5x (p95 19x); request-time 1.0->1.32x (2k->64k); the fat- uplink rental pool shows a smaller gap. tests/test_topology.py: 10 tests, all pass. Adversarial review also found a pre-existing false-"infeasible": the >TRIM latency funnel's must-set was subnet-blind, trimming feasibility-critical distinct-subnet cards when the fattest cards are co-located. Fixed (must-set keeps a distinct- subnet cover) + regression-tested.
…a-blocked Rented 8 real EU boxes for the residential upload A/B; measured real hop bandwidth heterogeneity (8-127 Mbps, i.e. residential-tier slow hops exist and would wall prefill), confirming the premise. The fully-automated per-node upload aware-vs-blind comparison was blocked by vast-container limits (no NET_ADMIN for tc, NAT hairpin, ssh-proxy rate-limiting under repeated runs). Boxes torn down (0 live, ~$4). Clean number needs one gentle non-rate-limited run; natural uplinks already span the residential range so no throttle needed.
M25_FP8_WIRE ships float8_e4m3fn tensors, but only shard/transport.py (the libp2p codec) had the fp8 entries in its dtype table — the raw-TCP PSK path rejected every fp8 frame as 'unknown tensor dtype' and reset the edge in a loop. Mirror the two entries; the tables must stay in lockstep (regression-tested in tests/test_eagle_draft.py).
… GQA broadcast The committed-context cache was a Python list of per-token [1,NKV,HD] tensors: every propose() re-cat'ed the WHOLE context + repeat_interleave GQA-expanded it, and extend() ran ~8 tiny kernels per position in a Python loop. With EAGLE at depth=1 all of that sits serially between recv and the next send, so it is pure per-traversal latency. Now: preallocated [CAP,NKV,HD] buffers (amortized doubling), one-shot batched extend, draft-chain k/v written into the scratch tail past ctx_len (next extend overwrites it — the committed cache is never mutated), and attention broadcasts the GRP query groups against NKV-head K/V instead of copying the context per step. CPU bench at toy dims, ctx 3000: prefill-extend 327ms -> 2.1ms (156x), decode round (extend K+1 + propose K) 16.4ms -> 4.35ms (3.8x); the GPU win is larger (the old path was launch-overhead-bound). Proposals are regression-locked to the original implementation (kept verbatim as the test reference) across rounds, dtypes, buffer growth and the legacy single-aux path: tests/test_eagle_draft.py, 6 tests. Also adds prefill_pair_tokens — the pure per-chunk EAGLE pairing the coordinator uses to feed the whole prompt (next commit).
… serial-path profile Three coordinator/stage-side costs on the EAGLE path (the ~7 tok/s reasoning product path), found by tracing one decode traversal: * aux rode every hop UN-quantized: 3 layers x [K+1,H] bf16 ~ 166KB/hop — 3x the activation payload itself — while h got fp8. _merge_aux now packs each stage's aux as [fp8,scale] (M25_FP8_AUX, defaults to the M25_FP8_WIRE setting); _unpack dequantizes once coordinator-side. Aux only feeds the DRAFTER, so losslessness is untouched by construction — only accept could move, and only when the flag is on. * the drafter saw only the LAST prefill chunk (default 512 tokens) of context: coordinate_pipe kept just the final chunk's aux. Every chunk now extends the EAGLE context as its response arrives (pairing = prefill_pair_tokens; the extend hides in the next chunk's WAN wait), so accept on long prompts stops being capped by a 512-token window. _eagle_aux_range also drops its per-position Python stack loop. * the decode wall was unattributed: t_draft/t_recv were measured but never returned. The result now carries decode_s/draft_s/ring_wait_s and coord prints the breakdown — the profile that ranks the next serial-path fix (measure first).
The 2026-06-27 warm sweep found K=8 ~2x over K=6 on the scattered EU ring, and every EAGLE measurement since ran K=8 — but both entry points still defaulted to 6, so any launch that forgot --K served below the known optimum. Defaults now match the receipts.
GraphRunner replays a captured graph and never re-runs run_block's aux capture, so with both flags set the EAGLE drafter silently feeds on stale prefill aux — accept craters and any measurement taken that way lies. The combination now refuses to start instead of quietly corrupting a run. (Found by the 2026-07-02 review fleet.)
…ram routing Two ways the hybrid wasted its drafting budget, both review-fleet finds: * every divergence (and the end-of-job drain) called d_fetch() on a request known to be stale — computing a full proposal (on the EAGLE path a K-step serial chain) just to discard it, then re-drafting. Drafters now expose cancel(); the coordinator uses it (falls back to fetch() for drafters without one). * NgramDrafter set matched=True on ANY ng-token anchor recurrence, even with zero preceding-context agreement (best_len=0). On novel/ reasoning text those are coincidences, so garbage n-gram drafts displaced the EAGLE drafts the hybrid exists for — accept 0 where EAGLE gets 2-4 tokens. matched now requires best_len >= min_match (default 1; M25_NGRAM_MINMATCH=0 restores any-anchor routing). The n-gram proposal itself is unchanged — only the routing signal moves; standalone n-gram deployments are unaffected. Needs the warm A/B to quantify the accept lift; the routing logic is regression-tested.
…findings, ranked next actions
… worktree-safe REPO Three measurement-integrity holes in m25_scatter_pipe, all review-fleet finds, all of which would have poisoned the upcoming branch-vs-master warm A/B: * M25_* flags were forwarded to the STAGES only — the coordinator and gateway processes got none, so e.g. M25_EAGLE=1 warmed aux-capturing stages while the coordinator silently drafted n-gram-only. One eng_env() prefix now feeds all three launch sites (only flags the operator actually set; unset ones fall through to the identical per-process defaults). Also brings the launcher's --K default to the measured 8. * push_code ignored scp failures — one flaky vast SSH silently launched a stale or MIXED-VERSION ring. Now retries once, then fails the launch loudly. * REPO was hardcoded to the main checkout — running the launcher from a worktree (the A/B: branch arm vs master arm) would push the WRONG tree's files. Derived from the launcher's own location now (SHARD_REPO overrides).
…rag-quote accept 13->44% Six arms on a scattered 6x5090 EU ring (rental-lottery order, same ring all arms). Serial-path branch vs master, EAGLE reasoning-hybrid, honest bench. Both warm orderings agree branch>master (fwd 4.0->5.9, rev 4.6->5.5) = +33% decode-weighted, jitter-robust. Accept identical on every cell EXCEPT rag-quote, where the whole-prompt drafter-context fix lifts accept 13->44% (g 2.0->4.5, ~4->10.5 tok/s) reproducibly across both branch passes. fp8-aux quality-safe (accept unchanged). min-match unproven on one pass.
…13->44%, merge-worthy
Perf/eagle serial path
…ail channel lifecycles The warm-ring wedge (re-warm before every coordinator, the session tax): the tail closed its predecessor whenever the coordinator-return died, the EOF cascaded upstream, and no stage ever re-dialed nxt_sock — one coordinator exit bricked the whole warm ring. Ports specpipe's proven recovery: head/middle deliberately drop the forward link on any edge error and rebuild it (dial-forever, 0.5s) before re-accepting a predecessor; the tail gives pred and ret independent lifecycles (ret death keeps pred+KV and accepts a new hello_return mid-session via select; pred death tears down both so a reset-ok can never reach a dead coordinator) and drops a dead job's in-flight replies until the next reset. Warm smoke-test: coord -> kill -> new coord on the same ring, no re-warm.
…not the receipts' CRITICAL trust hole (2026-07-02 fleet, 3 reviewers): verify_coverage's target layer_count was max(layer_end) over the receipts under test, so a ring that omitted layers still "tiled fully" and passed — skip-compute-and-still-get-paid. The m25 coordinator now passes S.cfg.num_hidden_layers (62) and fails CLOSED on an empty receipt list when SHARD_RECEIPTS is on; specpipe gains --n-layers and warns loudly when the target is self-derived (legacy path, coordinator never loads the model). tests/test_receipt_coverage.py pins the fixed semantics (truncated/gap/overlap/out-of-range/pinning/tamper all raise).
…live, guarded acks Two adversarial review passes on the churn-recovery port traced four real failure modes; all four are closed here: - A conn that speaks a JOB frame is adopted as the NEW predecessor with its frame queued (specpipe fill()'s semantic) instead of closed as junk — closing it kill-looped a stage replacement after a silent pred death, both at bring-up and mid-session. - TCP keepalive (60/20/3) on every ring socket bounds silent-peer death (no FIN: power loss, NAT expiry) to ~2min; the select()-based tail can't otherwise see a half-open predecessor now that the old 600s recv-timeout teardown (the idle wedge itself) is gone. - ret_ok sends are guarded at both loci: a coordinator dying between hello and ack was misclassified as predecessor death and cost a full ring cascade. - Pred-death keeps a JUST-reconnected coordinator's return channel (stale set, no reset consumed): the old-session EOF racing the gateway's retry used to kill the exact retry churn recovery exists for. Reset is only honored once a live ret exists; mid-session ret drops its greeting timeout (parity with bring-up); the pending pool is capped at 8. Also ports wire.py's malformed-frame guard to shard/transport.py recv_msg (the production path): the resilient tail now recv's stranger conns all session, so one unparseable frame crashing the stage process went from a bring-up window to a standing exposure. ConnectionError -> EDGE_ERRORS -> edge reset, not process death.
…al GQA kernel, receipts Port of eagle/tree-verify onto the merged serial-path base, with the 2026-07-02 fleet's payload verdict applied (the measured tok/s loss was ~6-7x self-inflicted wire bytes, not physics). What changed vs the measured v1: - propose_tree is best-first top-M by cumulative draft log-prob (M25_TREE_M/ _TOPB/_DEPTH; default 12 nodes) on the O(1) drafter internals — kills the fixed-2^d shape (62 nodes bought +0.7 g) that made the 1.23MB aux return. - Tree traffic rides _hsend/_hrecv: fp8 activations + fp8 aux per hop. - attn_tree always runs the manual broadcast-GQA kernel (tree_spec._gqa_masked_ attend): a dense mask knocks SDPA off flash on sm_120, and there is no repeat_interleave copy of the context; mask built once per stage-call on device in h's dtype. - Tree blocks are attested (signer.observe both branches + receipt sweep + coverage verify in coordinate_pipe_tree) — verification no longer silently off under M25_TREE, and the tree A/B carries the same receipt cost as chain. - Whole-prompt EAGLE prefill context (per-chunk extend), draft_s/ring_wait_s breakdown, loud no-aux guard, launcher forwards M25_TREE* + pushes tree_spec. - tests/test_tree_spec.py (8): mask ancestor property vs brute force, walk vs exhaustive, chain degeneracy, kernel-vs-reference, RoPE gather, and propose_tree topb=1 == propose() exactly (the offline losslessness gate).
…edge fixed) 6xEU 5090 ring: coord kill -9 mid-decode (depth=4 in flight) -> new coordinator completes on the same warm ring (6.70 -> 6.60 tok/s, weights never reloaded). Stage logs show the ported recovery choreography (drop+rebuild forward links, tail re-accepts channels). The re-warm-per-coordinator tax is gone.
…chain-tree The tree coordinator drove the EAGLE drafter directly, silently dropping the HybridDrafter's n-gram half; verbatim-reuse cells paid full EAGLE accept (2026-07-02 warm A/B: rag-quote 5.6 -> 3.0 tok/s while every novel cell won). Route per round: n-gram first, a real match verifies as a 1-wide tree (and now banks the +1 correction/bonus token the chain path drops); miss -> EAGLE tree.
…+18% Same-warm 6xEU ring, honest bench: chain 3.9 -> tree-hybrid 4.6 decode-weighted (reason-math 4.8->6.0, reason-logic 3.0->4.7, code-edit 4.5->6.1). Flips the v1 "tree loses tok/s on tight rings" verdict — the loss was wire payload, as the 2026-07-02 fleet concluded. Losslessness gate: 76 identical tokens then one near-tie kernel flip (documented class). rag-quote stays chain's win (pipelined n-gram vs sync tree round) — depth-aware hybrid is the measured next lever.
Fold the 2026-07-02 review fleet (79 confirmed) into a clear tiered roadmap: TIER 0 done (merged serial-path), TIER 1 perf (wedge, tree- verify, topology, cheap levers), TIER 2 trust/moat (the CRITICAL self- referential coverage check + 4 high), TIER 3 robustness (gateway/wire/ contained bugs), TIER 4 cleanup. Records the key signal: after the merge only 1 remaining HIGH is perf — risk moved off the single-stream hot path onto trust + gateway/wire. Adds the honest tok/s projection (~5.7-7 today -> ~10-12 good tight ring post-tree-verify+topology, ~5-6 global scatter; most findings are NOT tok/s). Full per-finding detail in .claude/plans/fleet-findings-20260702.md.
…erify wins EU +18% RESUME HERE now points at the two validated branches (wedge+trust, tree-v2), PROVEN gains the tree A/B + churn-survival rows, the trust row drops its self-referential warning, and TIER-1.1/1.2 + TIER-2.1 are ticked. Next: land both PRs, then the high-RTT global-scatter cell and the depth-aware hybrid.
leyten call (2026-07-02): steady-state rings are regional by construction (select_ring picks close subsets; a global ring is a placement failure). The global measure was go/no-go only while tree lost on tight rings; v2 wins the tight EU ring, so nothing hangs on a global number. NEXT re-ranked: depth-aware hybrid, then the RTT-ordered warm run for the absolute 10-12 check.
Fix/ring wedge receipt truth
Eagle/tree verify v2
…ith a pinned node select_ring's >_TRIM latency funnel built its feasibility cover require-blind: the pinned node's subnet slot went to the pool's fattest same-subnet card (which can never co-ring with require), and k_min stopped the cover walk before the other-subnet fat nodes a require-compatible ring needs — so the widened fallback searched a starved pool and returned None while a feasible ring existed. Fix: a second cover walk seeded with require (its subnet spent, its capacity counted), purely additive to the trimmed pool. Regression test fails pre-fix, passes post-fix; the legacy golden snapshot is byte-identical (the funnel never fires there). Unblocks the RTT-ordered launch (TIER 1.3).
fix(topology): require-aware trim funnel — no more false-infeasible w…
…ke-ring harness (leyten#14) Squash of perf/tree-depth-hybrid (11 commits): usability-report + CPU fake-ring harnesses, honest g + full-accept bonus, M25_STAGE_TIMING transport/compute split, select_ring head-first deployable orientation, depth-aware hybrid (pipelined n-gram bursts + sync tree rounds) with adversarial-review fixes, good-ring receipts (3 arms). 64 CPU tests green.
…he public internet The c0mpute technical report (author: leyten), final copy: page-one hero chart, the accept-gated pipelining law with a both-ends-calibrated Monte-Carlo, the depth-aware hybrid router, the transport/compute split and the CPU-launch-bound discovery, verification measured at 0.4% of stage compute, and a live coordinator-kill recovery timeline. Ships with the interleaved multi-arm bench harness (research/m25_paper_bench.py), four receipt sets from the test evening (docs/receipts/m25-paper-*), and every figure generated from those receipts (docs/paper/figures.py). 64 CPU tests green.
Paper published on Zenodo (CC-BY-4.0, v1.0). DOI added to the paper title block and the README, with the report linked from the top of the README.
Centered labels in the two segments collided where the segments were narrower than the text. Transport labels now sit outside the bar end, stage-compute labels stay inside their segment at a size that fits.
Reviewer feedback: one plain byline (leyten · c0mpute.ai · July 3, 2026) replaces the author and links lines; no links on the front page. Abstract now states the stages are untrusted, the defensible form of the trustless claim (Limitations still names the freshness/replay gaps). Title unchanged.
The concept DOI resolves to the latest report version; version DOIs stay valid for citations of a specific revision.
Reviewer feedback round two. 'Decentralized inference' placed in the intro and related work, 'decentralized AI' once in the close; three placements total, no stuffing. Conclusion gains a roadmap paragraph in which every future item traces to a measured finding (drafter training toward the 0.8 crossover, freshness/spot-recompute verification, CPU-aware selection and graph capture, right-sized models on the same fabric) and ends on the idle consumer-GPU supply plus a commitment to keep publishing receipts.
coordinate_pipe_tree started its output accumulator at `out = [cur]`, dropping every token committed before a mid-request node death. The chain coordinator already does `out = resume_ids + [cur]`; the tree path lost it, so under M25_TREE=1 a heal-and-resume returned only the post-failure tokens and continuation_preserved failed. Mirror the chain path exactly: seed `out` with resume_ids, and rebuild the draft prefix from prompt_ids + out (not gen_ids + out) so resume_ids is not double-counted. All three lines are no-ops when resume_ids is empty (gen_ids == prompt_ids), so the normal serve path is unchanged.
The tail and head/middle serve loops caught only EDGE_ERRORS (OSError/EOFError). Any other exception raised during a verify -- a RuntimeError from an over-context request (RoPE table / static-KV overflow), a CUDA OutOfMemoryError, a malformed frame -- propagated out of serve() and terminated the stage process, dropping its warm weights and taking the whole ring down until a manual relaunch. A request that is merely oversized, reachable through the unvalidated gateway, was enough to do it. Restore specpipe's proven "bad msg -> reset" guard: add a parallel except Exception to each loop that runs the same teardown as an edge close (reset layers, drop the channels, let the ring re-handshake on warm weights). EDGE_ERRORS is still matched first, so genuine edge deaths keep their existing predecessor/return-channel semantics.
shard/transport.py is the production receive path (SHARD_TRANSPORT=libp2p).
The libp2p link authenticates the peer but the engine authorizes nothing,
so any ring neighbour can stream a frame. Two unbounded allocations:
- recv_msg read a 64-bit length straight off the socket and buffered up
to 2**64 bytes with no cap.
- _unpack returned torch.empty(shape) for an empty blob, so a few header
bytes declaring shape [65536,65536] allocated 16 GB of host RAM -- a
memory-amplification bomb a frame-size cap alone would not catch.
Add SHARD_MAX_FRAME (default 2 GiB): a longer declared length is refused
before allocating, as a ConnectionError (an OSError -> a dead edge the
serve loop resets, not a crash). And require an empty blob to correspond
to a zero-element tensor; a non-empty shape with no bytes is rejected as
a malformed frame. Legitimate zero-element tensors still decode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three focused fixes in the serve path, each traced to a concrete failure it prevents. All are surgical and independent; the normal (non-failure) serve path is unchanged.
1. Preserve recovered tokens on tree-path FT resume (correctness)
coordinate_pipe_treeseeded its output accumulator without = [cur], dropping every token committed before a mid-request node death. The chain coordinator already doesout = resume_ids + [cur]; the tree path had lost it, so underM25_TREE=1a heal-and-resume returned only the post-failure tokens andcontinuation_preservedfailed (silent truncation of the user's output).Mirror the chain path exactly: seed
outwithresume_ids, and rebuild the draft prefix fromprompt_ids + outinstead ofgen_ids + outsoresume_idsis not double-counted. All three changed lines are no-ops whenresume_idsis empty (gen_ids == prompt_ids), so only the resume path is affected.2. Stage serve loops survive compute faults instead of crashing (resilience)
The tail and head/middle serve loops caught only
EDGE_ERRORS(OSError/EOFError). Any other exception raised during a verify — aRuntimeErrorfrom an over-context request (RoPE table / static-KV overflow), aCUDA OutOfMemoryError, a malformed frame — propagated out ofserve()and terminated the stage process, dropping its warm weights and taking the whole ring down until a manual relaunch. A request that is merely oversized, reachable through the gateway, was enough.Restore specpipe's proven "bad msg → reset" guard: a parallel
except Exceptionon each loop runs the same teardown as an edge close (reset layers, drop the channels, let the ring re-handshake on warm weights).EDGE_ERRORSis still matched first, so genuine edge deaths keep their existing predecessor/return-channel semantics.3. Bound transport frame size and reject empty-blob amplification (safety)
shard/transport.pyis the production receive path (SHARD_TRANSPORT=libp2p). The libp2p link authenticates the peer but the engine authorizes nothing, so any ring neighbour can stream a frame. Two unbounded allocations:recv_msgread a 64-bit length off the socket and buffered up to 2^64 bytes with no cap._unpackreturnedtorch.empty(shape)for an empty blob, so a few header bytes declaring shape[65536, 65536]allocated 16 GB of host RAM — a memory-amplification bomb a frame-size cap alone would not catch.Add
SHARD_MAX_FRAME(default 2 GiB): a longer declared length is refused before allocating, as aConnectionError(anOSError→ a dead edge the serve loop resets, not a crash). And require an empty blob to correspond to a zero-element tensor; a non-empty shape with no bytes is rejected as a malformed frame. Legitimate zero-element tensors still decode.Testing
64 passed(Python 3.13, CPU torch).socketpair: normal round-trip, a zero-element tensor, an oversized declared length, and the real empty-blob/huge-shape bomb — the bomb is rejected via thenumelguard as aValueError → ConnectionError, and the legit empty tensor still decodes.