Skip to content

HPC & Multi-Hardware Scaling (Post-Minimal) #36

Description

@yjmaxpayne

Overview

Tracks the longer-horizon hardware-scaling work: a CPU backend
(MKL-based sparse primitives, AVX-512, NUMA-aware threading), weak-scaling
multi-GPU ensembles (independent parameter-sweep runs distributed across
GPUs), strong-scaling multi-GPU (a single hierarchy computation partitioned
across GPUs), and eventually multi-node scaling. This sits behind the
BackendTraits abstraction, so each new backend implements the same
interface rather than re-deriving HEOM algorithm logic.

Background

Given this project's realistic delivery capacity, this scope is explicitly
sequenced after the single-GPU, single-maintainer-feasible core (traits
abstraction, bath redesign, external-consumer regression parity, packaging)
lands. It is tracked here for visibility and design continuity, not as a
near-term commitment — timeline is intentionally left open. The ordering
(ensemble multi-GPU before strong-scaling multi-GPU, single-node before
multi-node) reflects that ensemble/weak-scaling reuses far more of the
existing single-GPU implementation than strong-scaling does.

Core Objectives

1. CPU backend

MKL Inspector-Executor sparse primitives behind BackendTraits, AVX-512
intrinsics for elementwise hot paths, NUMA-aware OpenMP parallelization.
Functional correctness validated against the GPU path before any
performance target is pursued.

2. Ensemble multi-GPU (weak scaling)

An orchestration layer that runs N independent solver contexts across M
GPUs and aggregates results — reuses the existing single-GPU backend
unchanged, only adds a scheduling/aggregation layer.

3. Strong-scaling multi-GPU

Partitioning a single hierarchy computation across GPUs (candidate
strategies: ADO-block partitioning vs. hierarchy-depth partitioning), with
communication via GPU-to-GPU collectives and a host-staging fallback when
direct peer access isn't available.

4. Multi-node scaling (longest-horizon item)

Cross-node communication layered on top of the strong-scaling design,
explicitly without fault-tolerance/checkpoint scope.

5. Feasibility spike before committing to a partition strategy

A CPU prototype plus a 2-GPU partitioning proof-of-concept should precede
committing to a strong-scaling partition strategy, since a naive
partitioning approach is likely to produce incorrect results at hierarchy
boundaries on first attempt.

Success Criteria

Metric Target
CPU backend numerically matches GPU path within same tolerance used for bath/precision validation
Ensemble multi-GPU throughput scaling documented efficiency curve, no correctness regression
Strong-scaling partition feasibility spike go/no-go decision recorded with supporting data
All new backends implement BackendTraits without core-logic duplication verified by code review

Dependencies

Hard dependency on Backend Abstraction Architecture. Benefits from, but
does not strictly require, Bath Layer Redesign completion.

Note

This Epic intentionally carries lower priority than the API/backend/bath/
integration/test/packaging Epics; it should not be scheduled ahead of
external-consumer regression parity.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions