Skip to content

[PCIe] add group-consistent i8_hier DCP transport - #1

Draft
yatesdr wants to merge 1 commit into
stack/pr105-base-20260801from
feature/pr105-i8hier-20260801
Draft

[PCIe] add group-consistent i8_hier DCP transport#1
yatesdr wants to merge 1 commit into
stack/pr105-base-20260801from
feature/pr105-i8hier-20260801

Conversation

@yatesdr

@yatesdr yatesdr commented Aug 1, 2026

Copy link
Copy Markdown
Owner

What

Adds i8_hier, a four-rank INT8 PCIe all-reduce mode for systems arranged as two fast GPU pairs with a slower inter-pair path.

  • the first reduce-scatter and final all-gather remain inside each fast pair;
  • only the pair-owned half of the compressed tensor crosses the slow boundary;
  • matching owners combine pair partials in a canonical order;
  • every rank reconstructs the final BF16 result from the same final INT8 payload;
  • unsupported world sizes fail closed.

The implementation uses the PCIe lifecycle supplied by upstream SparkInfer PR local-inference-lab#105. It does not add a rank-local fallback or alter PR local-inference-lab#105's initialization, replay, header-packaging, or teardown contracts.

Why

On four-GPU PCIe systems with a 2+2 topology, a flat ring sends more data across the slow boundary than necessary. i8_hier keeps the existing rank-consistent block-INT8 codec while matching the collective schedule to that topology.

Validation

  • includes focused mode-normalization, rank-consistency, and GPU collective coverage;
  • the exact files in this branch are byte-identical to the qualified r17 image;
  • the TP4/DCP4 production qualification selected B12X PCIe collectives on every rank with no PyNCCL fallback, collective divergence, capture failure, or restart;
  • cold 350k retrieval returned the exact expected value 738216;
  • MTP3/C16 completed at 234.05 aggregate tok/s with 16/16 requests active and zero errors.

Dependency

This draft is stacked on local-inference-lab#105. Its base branch is an immutable copy of that PR head so the review diff contains only i8_hier. Retarget to master after local-inference-lab#105 merges.

Review stack: #1

Assisted-by: OpenAI Codex
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant