feat(glm52): cross-engine P/D — vLLM TP8 prefill + openinfer EP8 decode, strict zero-prefill by xiaguan · Pull Request #657 · openinfer-project/openinfer

xiaguan · 2026-07-12T10:48:16Z

feat(glm52): cross-engine P/D — vLLM TP8 prefill + openinfer EP8 decode, strict zero-prefill

Carries the qwen3 vLLM-compat P/D groundwork directly (formerly stacked #540, now closed as superseded — its commits are part of this branch and were re-verified against current main). This PR takes the cross-engine P/D design to GLM5.2 across two 8×H200 nodes: P = vLLM TP8 (node A), D = openinfer EP8 (node B), pegaflow content-addressed KV with one-sided RDMA READ in between, and a router that forwards P's first generated token into D's context so D's first forward is a one-token decode — D never computes a single prompt position (strict mode: miss ⇒ reject ⇒ router retries through P).

What's in here

Admission (openinfer-glm52/src/scheduler/)

admit_vllm_pd: restore the full peer-prefilled prefix under vLLM's own block hashes (xxh3_128 over CBOR, seed-pinned), adopt the partial tail page under a derived tail key into the request's private page (never the radix), admit only at suffix == 1.
Park/deadline state machine: a racing registration parks the front request across step boundaries; a zero-hit miss window (--kv-pd-miss-wait-ms) and a 15s in-flight-fetch ceiling bound the wait; 3 consecutive exhausted windows open a miss breaker that shortens the wait to a 500ms probe window (fail over fast while a recovery stays automatic — see "failure injection" below).
vLLM-compat registration: arenas join the peer's pegaflow namespace under vLLM layer names with page-first layout (78 MLA layers × 656 B/token fp8_ds_mla + 21 index-K layers × 132 B/token, one host slot per block, offsets by layer-name lexicographic order).

RoPE layout fixup (openinfer-kernels/csrc/glm52/glm52_vllm_rope_fixup.cu)
vLLM (is_neox_style=False) stores rotated RoPE pairs interleaved (2i, 2i+1); openinfer's kernels are interleave-in/block-out [i, i+32]. Restored pages get a one-time deinterleave (MLA rows' 64 bf16 rope dims + index-K rows' first 64 fp8 bytes) between H2D landing and the page becoming matchable — exactly-once by construction (radix hits skip the restore leg entirely).

Verified (all measured on the two-node setup)

Token-identical output vs vLLM-direct baseline, aligned and tail-block prompt shapes, deterministic across repeats.
Zero prefill: every admission in every test window logs suffix=1; strict mode has no local-compute path.
Multi-turn delta reuse: turn N ≥ 2 pulls only the incremental blocks.
Accuracy: gsm8k 5-shot exact-match 0.960/0.970 (two rounds) vs 0.955 direct; NIAH passkey 36/36 at 4k/8k/16k across depths (16k = 256 blocks ≈ 845 MB per-request RDMA), parity with direct.
Failure injection: kill -9 on P ⇒ clean 502 in ~5ms, D unaffected, vLLM-only restart recovers. kill -9 on the metaserver ⇒ clean 502s while down; after restart the first fresh request re-closes the breaker automatically (0.18s, no manual action). The injection exposed a breaker deadlock — pegaflow's query() only starts an async fetch and reports a miss, so the old zero-wait open-breaker rejection could never observe a complete restore and the breaker never closed; fixed by the probe-window park plus closing the breaker on any full-window metadata hit.
Equal-card acceptance bench (16 vs 16 cards, 2× clean vLLM TP8 with session affinity as baseline; 8k first turn +2k/turn ×5, 32 conversations, byte-identical token streams on both sides, 160/160 success each): P/D holds TPOT p99 ≈ median (26–38 ms) through all turns while mixed spikes to 67–89 ms on turn 1 under prefill preemption, and P/D's turn-3+ TTFT median is 0.6–1.1 s vs ~1.7 s. Mixed wins raw throughput (340 vs 271 tok/s) and turn-1 TTFT on this prefill-heavy synthetic shape — a P:D=1:1 ratio mismatch (all 32×8k first-turn prefills queue on one node), not an architectural gap; under a "TPOT p99 ≤ 40 ms" SLO, P/D goodput is 100% while mixed violates in batches.

Full execution record, per-gate data and reproduction scripts: docs/models/glm52/pd-m2-execution.md.

Hard dependencies

vLLM 0.24.0 on P (0.23 builds indexer caches on all 78 layers → 156 registered layers vs openinfer's 99).
--kv-cache-dtype fp8_ds_mla --block-size 64 --prefix-caching-hash-algo xxhash_cbor + pinned PYTHONHASHSEED on P.
Pegaflow #395 (connector tail-save + router first-token forwarding) is merged, and pegaflow-core is pinned to merge revision 1473c5355d879b4fea23101760cb2a0074642ada.
Pegaflow #382 is not required by the supported normal configurations: stacked Qwen3 feat(qwen3): vLLM-prefill P/D — cross-engine KV compat, verified end-to-end #540 uses vLLM 0.23.0 default NHD, while GLM uses vLLM 0.24.0 page-first; both register the same single-segment block-local bytes OpenInfer restores.
feat(qwen3): vLLM-prefill P/D — cross-engine KV compat, verified end-to-end #540 is closed as superseded: its branch had gone CONFLICTING with main, while this branch already contains the same commits with the conflicts resolved (merge b843bd4a) and the full 12-check CI green on top. This PR lands as one unit.

Known limits / follow-ups

Metaserver metadata is in-memory: after a metaserver restart, previously saved prefixes are undiscoverable until P re-saves them (clean 502s, documented runbook: restart vLLM alongside). Proper fix is a pegaflow-side catalog re-publish on reconnect (tracked in the pegaflow repo).
openinfer's indexer RoPE follows the old-transformers half-split convention; NIAH parity up to 16k downgrades the risk but doesn't fully close it (doc has the details and the escalation path).
The three restore legs (host-tier, P/D full-window, P/D tail) share a query→lease→load→wait→commit shape and could fold into one helper; kept apart here to leave the verified paths untouched.
The miss-breaker constants are duplicated with qwen3's executor; both belong in openinfer-kv-offload once the semantics converge.

🤖 Generated with Claude Code

GLM prefill rides vLLM (openinfer glm52 kernel surface is decode-only); P->D readiness = D-side bounded fast-poll over pegaflow CPU P2P; cross- engine keys via a vLLM-hash-compat provider (xxhash_cbor + pinned PYTHONHASHSEED); tail-block connector extension + router t1 forwarding planned. Roadmap: qwen3 vLLM-P + openinfer-D smoke test first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…t window Decode-node support for a vLLM prefill peer over pegaflow CPU P2P: - openinfer-kv-offload::VllmBlockHasher replicates vLLM's xxhash_cbor prefix-cache hashing (canonical CBOR + xxh3_128, NONE_HASH from PYTHONHASHSEED, parent-chained full blocks + derivable tail keys); golden vectors captured from a real vLLM process. - Qwen3VllmCompatOptions switches cold-request offload queries to the vLLM key chain over the same [gpu_hit..cacheable) window (local GPU naming stays kvbm), joins the P side's connector namespace, skips LoRA requests (unreplicated extra_keys salting), and disables self-saves (kvbm keys are unfindable in a vLLM-keyed domain). - remote_fetch_action grows wait_on_miss: during the P/D handoff race a zero hit means the producer's registration hasn't landed yet, so the request stays parked until a bounded miss deadline instead of degrading to a local prefill. - Server flags: --kv-pd-vllm-seed / --kv-pd-vllm-namespace / --kv-pd-miss-wait-ms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…bility contract qwen3 vLLM-P + openinfer-D smoke passed on node 34: byte-identical outputs, delta reuse, TTFT overhead +14ms/+51ms/+147ms at 0.5k/1.8k/7k tokens. Distill the verified compatibility equation (namespace factors, block-size == D page size, per-layer slot counts, K/V segment layout absorbed by pegaflow#382) into section 3.1 and mark the remaining milestone-1 gaps (tail-block extension, strict no-prefill mode). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

… path Toxic-review findings on the vLLM-compat branch: every failure mode of the P/D handoff (wrong seed, wrong namespace, block-size mismatch, peer down) degraded into silent scratch prefills that read as "network is a bit slow". Address the unhappy path: - Warn when a request exhausts the zero-hit wait window (the single symptom all misconfigurations share), with blocks queried and time waited. - Log a startup fingerprint (seed, namespace, block size, NONE_HASH, KV geometry) an operator can diff against the vLLM peer's config. - Miss breaker: after 3 consecutive exhausted windows, new requests skip the wait — a dead or misconfigured peer no longer taxes every cold request the full window. Any remote hit re-arms waiting. - Throttle parked re-queries to one MetaServer RPC per 5ms per request so a miss storm cannot turn scheduler ticks into serial RPC pumps (the throttle the design doc promised but the code never had). - Reject miss-wait >= the 15s remote-fetch deadline at startup instead of silently capping it; validate seed (decimal u32) and namespace (8-hex) at flag parse time — an empty seed is a well-formed key space that can never match. - Drop the speculative include_tail parameter from key_chain (the tail golden vector stays covered via hash_block) and take the query window from the probe without materializing kvbm hashes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Failure injection (P down, 4 sequential cold requests) showed the 4th request still waiting the full miss window after the breaker opened: a transient Loading answer on the first query parks a request past the breaker gate, and the poll path recomputed wait-on-miss from the deadline alone. Make the poll consult the breaker too, and only emit the degradation warning (and count a miss window) when the request actually waited the window out, so breaker-shortened requests do not spam the log the breaker warning already covers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…tone table Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

A Loading-stuck peer rides requests to the hard deadline instead of the miss window; without counting those, the breaker never opens and every cold request pays the full 15s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…eat/glm52-pd # Conflicts: # docs/index.md # openinfer-kv-offload/src/lib.rs # openinfer-qwen3/src/executor.rs

…il-page adoption D-side of GLM5.2 P/D M2 (vLLM TP8 prefill + openinfer EP8 decode): - Glm52KvOffloadOptions grows p2p (pegaflow mesh) and vllm_compat (peer hash scheme) knobs; offload host joins the P2P mesh and the peer's namespace. - Admission in vLLM-compat mode restores the FULL peer-prefilled prefix under vLLM block hashes: full pages via the existing restore leg, the partial tail page via a derived tail key loaded into the request's own scheduled page (never the radix), leaving a single-token first forward (the router-appended t1) — zero prompt-position compute on the decode node. - A racing registration parks the front request at the queue and retries per step boundary (5ms idle throttle); an exhausted wait window REJECTS for the router to retry (miss breaker after 3 consecutive windows). --kv-pd-allow-local-prefill is the debug escape hatch. - Self-saves are disabled in compat mode (kvbm keys are unfindable in the vLLM-keyed domain). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The vLLM connector stores MLA-model blocks page-first (one host page per block, layer offsets by lexicographic layer name), so vllm-compat mode now registers each rank's arenas page-first under vLLM's own layer names (model.layers.N.self_attn.attn / .indexer.k_cache) instead of the native layer-first glm52.LN.* registration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

step_page_indices returns every page up to the step end, so the tail load was handing pegaflow all restored full-block pages as destinations for a one-block lease. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

vLLM (is_neox_style=False) stores rotated RoPE pairs interleaved at (2i, 2i+1); openinfer's kernels are interleave-in / block-out — same values, permuted dims. Restored pages read as structurally-wrong KV and decode derails. Rewrite each pulled page exactly once (before it becomes matchable / readable): the MLA rope-key dims at byte 528 and the first 64 fp8 dims of each index-K row, via a rank-worker command so the permute orders on the compute stream ahead of any step kernels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…wait reject pegaflow's query() only starts an async metaserver resolve + fetch and reports a miss until it lands (~50ms). The open-breaker fast-reject gave first-shot queries zero wait, so no remote restore could ever complete and the breaker never closed: three exhausted windows (e.g. a metaserver restart) put the decode node into permanent rejection even after the peer recovered. Found by failure injection. Open breaker now parks the front request with a 200ms probe window (covers the ~46ms P-side save-visibility pipeline plus the async fetch) instead of rejecting immediately; a complete restore still resets it, so recovery is automatic while a genuinely-down peer keeps failing over at probe cadence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…tics, breaker deadlock fix verified

…ard count

…er recovery, admission ownership Failure-injection review follow-ups on the vLLM-compat admission path: - saw_loading is per-attempt, not sticky: pegaflow's first query always starts an async fetch and reports Loading, so a sticky flag pinned every request to the 15s fetch deadline and --kv-pd-miss-wait-ms never bound - a full-window metadata hit closes the miss breaker immediately: restores larger than one probe window otherwise re-feed the breaker forever and it can never close after a metaserver blip - pegaflow query keys always come from the internal counter — a client- controlled request_id keyed pegaflow's prefetch state and failed-remote blacklist (duplicate ids cross-consume fetches; retries inherit a 5-minute blacklist entry) - launch validation enforces miss_wait < the 15s remote-fetch deadline (the config help text claimed this was enforced; only qwen3 did) - drop the gpu_hit + window == chain.len() debug assert: a retried block-aligned prompt legitimately overshoots cacheable by one (the plain path has always tolerated this state) - admit_from_queue moves to scheduler/admission.rs where intake lives (mod.rs back under 1k lines); the PD dispatch asserts the EP topology instead of carrying an unreachable mirrored branch - table tests for the park/window/breaker state machine Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4b5836a021

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-12T10:53:21Z

+            "kv_pd_vllm_seed",
+            "kv_pd_vllm_namespace",
+            "kv_pd_miss_wait_ms",
+            "kv_pd_allow_local_prefill",


Add the P/D flags to Qwen3's consumed list

When the detected model is Qwen3, Args::validate rejects any explicitly provided CLI id that is not listed in ModelType::Qwen3's consumed_args. This change wires kv_pd_vllm_seed, kv_pd_vllm_namespace, and kv_pd_miss_wait_ms into the Qwen3 launch path in load_engine, but the new ids are only accepted for GLM here, so a Qwen3 launch with the documented vLLM P/D flags fails validation before the engine is loaded (for example, --kv-pd-vllm-seed is not used by Qwen3).

Useful? React with 👍 / 👎.

Fixed in 7621edb: Qwen3's consumed_args now accepts kv_pd_vllm_seed / kv_pd_vllm_namespace / kv_pd_miss_wait_ms. Root cause: the validate() mechanism landed on main after the qwen3 vLLM-compat branch (#540), so only the merged branch hit the rejection. kv_pd_allow_local_prefill stays excluded on purpose — qwen3 never reads it (its miss path always falls back to a real local prefill), so rejecting it is the accurate behavior.

chatgpt-codex-connector · 2026-07-12T10:53:21Z

                            query_hashes,
-                            deadline: std::time::Instant::now() + REMOTE_FETCH_DEADLINE,
+                            deadline: now + REMOTE_FETCH_DEADLINE,
+                            miss_deadline: now + miss_wait,


Short-circuit Loading when the Qwen3 breaker is open

When Qwen3 vLLM-compat has opened the miss breaker, expect_remote only changes the zero-hit path; if the first query returns Loading (the async P2P fetch path), this branch still parks the request with the full 15s remote-fetch deadline. During a peer or MetaServer outage, new cold requests can therefore continue to stall on Loading until the hard deadline even after the breaker log says new requests will prefill immediately; use a short/open-breaker deadline here or avoid parking on Loading while the breaker is open.

Useful? React with 👍 / 👎.

Fixed in 7621edb: remote_fetch_action takes a park_on_loading flag — with the breaker open, a Loading answer folds to Scratch (immediate local prefill) instead of parking for the full 15s fetch deadline. The abandoned async fetch still lands in the local host tier, so a later request's first-shot query can answer Ready and re-arm waiting; plain offload mode (no prefill peer) still always parks. Covered by loading_scratches_when_parking_disabled.

xiaguan · 2026-07-12T13:05:29Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4b5836a021

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-12T13:11:01Z

+                if let Some(seed) = args.kv_pd_vllm_seed.clone() {
+                    offload =
+                        offload.with_vllm_compat(openinfer_qwen3::Qwen3VllmCompatOptions {


Include Qwen3 in vLLM P/D flag validation

When launching a Qwen3 decode node with --kv-pd-vllm-seed (and the required P2P/offload flags), Args::validate runs before load_engine and rejects every provided flag that is not listed in consumed_args(ModelType::Qwen3). The new kv_pd_* flags are only added to the GLM5.2 consumed list, so this Qwen3 branch is unreachable from the CLI: the process exits with --kv-pd-vllm-seed is not used by Qwen3 instead of enabling with_vllm_compat.

Useful? React with 👍 / 👎.

Fixed in 7621edb (same root cause as the config.rs comment) — the Qwen3 with_vllm_compat branch is now reachable from the CLI.

… the breaker Codex review follow-ups on #657: - Qwen3's consumed_args gains kv_pd_vllm_seed / kv_pd_vllm_namespace / kv_pd_miss_wait_ms: the validate() mechanism landed on main after the qwen3 vLLM-compat branch, so the merged branch rejected the documented flags before load_engine could enable with_vllm_compat (kv_pd_allow_local_prefill stays excluded — qwen3 ignores it by design) - remote_fetch_action takes park_on_loading: in compat mode the first-shot query always answers Loading (it only starts the async fetch), so an open breaker still stalled every cold request for the full 15s fetch deadline despite promising immediate local prefill; with parking disabled the abandoned fetch still lands in the host tier, so a later first-shot hit re-arms waiting Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

xiaguan · 2026-07-13T08:41:27Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9de922710d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-13T08:48:56Z

+        crate::runner::Glm52Worker::Remote(worker) => anyhow::bail!(
+            "GLM5.2 vLLM RoPE fixup cannot run on remote rank {}",
+            worker.rank()


Reject vLLM P/D when rank-host workers are present

When GLM5.2 is launched with --rank-hosts together with --kv-pd-vllm-seed (the CLI/launch path accepts both), any vLLM P/D restore assigned to a remote rank reaches this branch. admit_vllm_pd treats the returned error as an admission failure and the coordinator calls fail_step, so the first remote-rank restore can tear down the decode engine instead of failing fast at startup. Please either reject this configuration in launch validation or forward the RoPE fixup command to rank-host workers.

Useful? React with 👍 / 👎.

This configuration is already rejected in launch validation, before any engine state exists: --kv-pd-vllm-seed is structurally part of the --kv-offload option group (kv_offload.vllm_compat), and launch() has ensure!(kv_offload.is_none(), "GLM5.2 --kv-offload is not supported with --rank-hosts yet") whenever remote_ranks > 0. Both entry points (openinfer-server and glm52_step_bench) go through launch(), so admit_vllm_pd can never see a Glm52Worker::Remote — the bail! in vllm_rope_fixup is a defensive backstop for an unreachable state, not a live failure path.

xiaguan and others added 28 commits July 3, 2026 12:21

chore(kv-offload): commit lockfile entry for xxhash-rust

410fbbe

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(glm52): reflect miss-path observability and breaker in the miles…

9c43d66

…tone table Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/feat/pd-vllm-hash-compat' into f…

e1e14b7

…eat/glm52-pd # Conflicts: # docs/index.md # openinfer-kv-offload/src/lib.rs # openinfer-qwen3/src/executor.rs

test(glm52): update admit_from_queue contract test for the P/D parameter

38228e9

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(glm52): record P/D M2 implementation landing points and gate status

b200291

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(server): register GLM5.2 p2p/vllm-compat flags in consumed_args

1ce8efb

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(glm52): load P/D tail page into its own page only

9b75c33

step_page_indices returns every page up to the step end, so the tail load was handing pegaflow all restored full-block pages as destinations for a one-block lease. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(kernels): local CUresult helper in glm52 rope fixup TU

f37ddcb

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(glm52): borrow page_ids for rope fixup

f32a225

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(glm52): record P/D M2 cross-engine bring-up results and gates 0-3

c58d822

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(glm52): record P/D bring-up quick load-test snapshot

be658a2

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(glm52): P/D TTFT-delta and TPOT-stability A/B snapshot

ae1a6bb

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(glm52): multi-turn A/B and gsm8k accuracy gate results

521490f

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(glm52): NIAH long-context probe results, indexer risk downgraded

83e8ac7

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(glm52): failure injection results — kill-P/kill-metaserver seman…

77ffabd

…tics, breaker deadlock fix verified

docs(glm52): gate-5 acceptance benchmark — P/D vs 2x mixed at equal c…

eb3f32f

…ard count

docs(glm52): record reviewed tail-save trigger semantics

4b5836a

chatgpt-codex-connector Bot reviewed Jul 12, 2026

View reviewed changes

xiaguan and others added 4 commits July 12, 2026 09:41

fix(glm52): refresh P/D branch against current main

b843bd4

fix(pd): pin merged pegaflow dependency

992a533

docs(pd): record dependency CI result

9de9227

xiaguan mentioned this pull request Jul 13, 2026

feat(qwen3): vLLM-prefill P/D — cross-engine KV compat, verified end-to-end #540

Closed

chatgpt-codex-connector Bot reviewed Jul 13, 2026

View reviewed changes

xiaguan merged commit ad279af into main Jul 13, 2026
12 checks passed

Uh oh!

Conversation

xiaguan commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

feat(glm52): cross-engine P/D — vLLM TP8 prefill + openinfer EP8 decode, strict zero-prefill

What's in here

Verified (all measured on the two-node setup)

Hard dependencies

Known limits / follow-ups

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

xiaguan Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

xiaguan Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

xiaguan commented Jul 12, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

xiaguan Jul 12, 2026

Choose a reason for hiding this comment

Uh oh!

xiaguan commented Jul 13, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

xiaguan Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

xiaguan commented Jul 12, 2026 •

edited

Loading