Pass -gpu= at link time so the Thrust fatbin keeps every architecture - #17
Merged
Conversation
The device-link step is where the final SASS is generated, so an arch passed only at compile time is discarded and the extension silently gets nvc++'s own default target instead of what was requested. Measured on NVHPC 26.1 (aarch64), compile with -gpu=cc90: .so linked without -gpu= -> sm_100 (the host default, not cc90) .so linked with -gpu= -> sm_90 _core_gpu_omp_offload already passes -gpu= at link, which is why only the Thrust backend was affected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hfwen0502
force-pushed
the
fix-thrust-fatbin-arch-at-link
branch
from
September 3, 2026 20:30
1b85c87 to
ae463c8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Thrust extension passes
-gpu=<arch>when compiling but not when linking.That is not a cosmetic omission. The device-link step is where the final SASS is generated —
nvc++keeps device IR in the object file and codegens at link. So without-gpu=there, the architecture requested at compile time is discarded entirely and the artifact gets whatever the toolchain's own default target is. No warning, exit 0.Measured on NVHPC 26.1 (aarch64, a cc100 host):
.sosm_100-gpu=cc90sm_100← asked cc90, got sm_100-gpu=cc90-gpu=cc90— this PRsm_90Note the middle row: it is not that a comma-separated list gets truncated, it is that any explicit request is overridden. A single-arch
cc90build is equally affected whenever the build host's default differs — which is why this surfaced as a Thrust-only failure on H100 from a build that had asked for the right architecture. Nothing masks it: these builds embed no PTX, so there is no JIT fallback._core_gpu_omp_offloadwas never affected — its extension already passes-gpu=inextra_link_args. That asymmetry is exactly why the symptom looked backend-specific rather than like a build-flag bug.Fix
One line, matching what the OMP-offload extension already does.
Effect on
mainmainresolves the arch as_resolve_gpu_arch(default='cc90'), and the README documents that default. With this PR that default becomes effective rather than nominal: an unsetSBD_GPU_ARCHnow genuinely produces ansm_90binary, as documented. Today it produces whatever the build host defaults to, which may not besm_90at all.So this PR makes
main's behaviour matchmain's README. It deliberately changes nothing else — no README edits, no change to the default.Verification on a real build of this package
Multi-arch, on GB200:
_core_gpu_thrustSASSsm_100sm_80 sm_90 sm_100Still numerically correct —
run_sbd_diag.py, h2o-1em3, 4 ranks (2×2):matching the published reference for that determinant set.
Notes
-gpu=ccall-majoralso works on the Thrust path, yieldingsm_80 sm_90 sm_100 sm_110 sm_120— a reasonable choice for distributed images instead of enumerating.Follow-ups (deliberately not in this PR)
build_extthat the architectures in the linked extension matchSBD_GPU_ARCH(cuobjdump --list-elf), so a silent arch drop fails the build instead of surfacing on someone else's GPU months later.SBD_GPU_ARCHoptional — unset meaning "target the build host", set meaning "honor exactly, including several generations" — is implemented onfix-conda-build-and-mpi-detectionand will arrive onmainwith that branch at the next PyPI release.