LAMMPS: workaround for ARM builds with CUDA and ARM - #4175
Conversation
![hound[bot]](https://avatars.githubusercontent.com/in/3598?s=60&v=4){kind=small}
…kos and cuda without gpu present
… on node without them
| # See https://github.com/kokkos/kokkos/issues/7483 | ||
| cuda_root = get_software_root('CUDA') | ||
| if get_cpu_architecture() == AARCH64 and LooseVersion(os.path.basename(cuda_root)) < '13.2.0': | ||
| self.cfg.update('configopts', '-D%s_ARCH_%s=no' % (self.kokkos_prefix, processor_arch)) |
There was a problem hiding this comment.
This will effectively result in using -DKOKKOS_ARCH_NATIVE=no, which seems like a big hammer to me, we're effectively falling back to a generic build then for the CPU part of Kokkos?
In this PR:
- Add nvidia Grace Architecture kokkos/kokkos#7158 (tactics slightly changed in Rework simd
neonandsvedetection kokkos/kokkos#8667)
the LAMMPS developers selectively disable the use of Neon by disabling KOKKOS_ARCH_ARM_NEON, wouldn't that be sufficient here?
I'm not sure what -DKOKKOS_ARCH_NATIVE=yes -DKOKKOS_ARCH_ARM_NEON=no does though, does that effectively disable Neon?
There was a problem hiding this comment.
Based on kokkos/cmake/kokkos_arch.cmake, I don't think that the combination of -DKOKKOS_ARCH_NATIVE=yes with -DKOKKOS_ARCH_ARM_NEON=no would work...
Another approach could be to replace all occurences of set(KOKKOS_ARCH_ARM_NEON ON) in kokkos/cmake/kokkos_arch.cmake with set(KOKKOS_ARCH_ARM_NEON ON).
That could be done with a (static) patch file, or with a runtime patch (a sed command in preconfigopts, or using apply_regex_substitutions in the LAMMPS easyblock.
There was a problem hiding this comment.
In general our decided stance on asking for native vs telling us which explicit arch is:
As decided in the dev meeting, this detection is now only active for Kokkos_ARCH_NATIVE. That also implies that for host archs that are specified, we expect the user to know that their toolchain supports the respective architecture and flags
Meaning, if you do tell us which exact arch you have (e.g. Kokkos_ARCH_ARMV84_SVE) we just assume that you also told us that your compiler combination can handle the respective instructions.
If you tell us to use NATIVE, we will do our best to detect it for you, but that only works when you are not cross compiling.
This also implies that if you compiler does not allow the simd extensions for the processor you are targeting there is not much that Kokkos can do for you.
Please do not try to manually manipulate any of our internal CMAKE variable. e.g. KOKKOS_ARCH_ARM_NEON
| cuda_root = get_software_root('CUDA') | ||
| if LooseVersion(os.path.basename(cuda_root)) < '13.2.0': | ||
| cxxflags = os.getenv('CXXFLAGS', '') | ||
| cxxflags += ' -march=armv8-a+nosimd' |
There was a problem hiding this comment.
Also here, this is a big hammer, we'd be losing quite a bit of stuff on an NVIDIA Grace system, for example...
Can we only disable Neon, but keep SVE?
@Thyre Any ideas?
There was a problem hiding this comment.
Let me do some checks if we can somehow isolate this to just Kokkos::simd
There was a problem hiding this comment.
so Kokkos does not allow to disable NEON but enable SVE:
#if defined(KOKKOS_ARCH_ARM_SVE)
#if !defined(__ARM_FEATURE_SVE_BITS) || !defined(__ARM_NEON)
#error \
"Both __ARM_FEATURE_SVE_BITS and __ARM_NEON must be definded for KOKKOS_ARCH_ARM_SVE"
#endif
#include <Kokkos_SIMD_SVE.hpp>
#endif
There was a problem hiding this comment.
AFAIU it would be possible to configure Kokkos without NEON support but compile downstream files with the -march=armv8 flag. But that produces potential mismatch if someone tries to infer the size of some Kokkos simd types via __ARM_NEON
boegel
changed the title
4 participants
(created using
eb --new-pr)