Skip to content

Native ARM64/NEON support (via SIMDe) - #931

Open
BenjaminDEMAILLE wants to merge 3 commits into
chhylp123:masterfrom
BenjaminDEMAILLE:arm64-neon-simde
Open

Native ARM64/NEON support (via SIMDe)#931
BenjaminDEMAILLE wants to merge 3 commits into
chhylp123:masterfrom
BenjaminDEMAILLE:arm64-neon-simde

Conversation

@BenjaminDEMAILLE

Copy link
Copy Markdown

Native ARM64/NEON support (via SIMDe)

This makes hifiasm build and run natively on aarch64/arm64 (Linux ARM64 and
Apple Silicon) in addition to x86, with the x86 build byte-for-byte unchanged.

Fixes #288. Supersedes #641: that PR used sse2neon and built fine, but this one
uses SIMDe instead, which also covers
AVX (not just SSE), is header-complete for the intrinsics hifiasm uses, and comes
with a cross-arch CI matrix that proves the x86 and ARM64 assemblies are identical.

Why the current build fails on ARM

The Makefile passes -msse4.2 -mpopcnt (the latter is a hard error on aarch64),
and Levenshtein_distance.h / ksw2_extz2_sse.c include x86 intrinsic headers
(<emmintrin.h>, <smmintrin.h>, <nmmintrin.h>, <immintrin.h>) that do not
exist on ARM.

What changed

All source changes are gated behind defined(__aarch64__) || defined(__arm__),
so on x86 the preprocessor takes the original path and codegen is unchanged.

  • Makefile: detect the target with uname -m. On aarch64/arm64, drop
    -msse4.2 -mpopcnt, use -march=armv8-a+simd, and add -Ithird_party/simde.
    ARCH and ARCH_FLAGS remain overridable, e.g.
    make ARCH_FLAGS="-mcpu=neoverse-v2" or make ARCH=aarch64. The x86 branch is
    identical to before (-msse4.2 -mpopcnt, empty INCLUDES).
  • Levenshtein_distance.h: on ARM, include the SIMDe x86/sse4.2.h and
    x86/avx2.h headers instead of the four x86 headers. One subtlety: GCC's x86
    _mm_srli_epi32 accepts a runtime shift count, but the NEON translation
    requires a compile-time constant, so the two variable-count uses go through a
    tiny HA_SRLI_EPI32_VAR macro that expands to the identical _mm_srli_epi32
    on x86 and to the variable-count shift on ARM.
  • ksw2_extz2_sse.c: provide the SSE2/SSE4.1 kernel via SIMDe and define
    __SSE2__/__SSE4_1__ so the body compiles on ARM. Note this file is not in
    the Makefile OBJS and its only caller is commented out, so it stays unlinked;
    it is ported (and verified to compile standalone on ARM) so it builds cleanly
    if it is ever re-enabled, with no change to either binary.
  • third_party/simde/: vendored SIMDe, pinned to v0.8.2
    (commit 71fd833d9666141edcd1d3c109a80e228303d8d7, MIT). Only the simde/
    header tree and COPYING are included (no tests/docs), so a plain
    git clone && make works with no submodule step.
  • .github/workflows/ci.yaml + test/gen_hifi_reads.py: CI now builds a
    matrix of Linux x86_64, Linux aarch64, macOS x86_64, and macOS arm64, and a
    smoke job assembles a tiny deterministic synthetic HiFi set on both Linux arches
    and asserts the x86_64 and aarch64 graphs are identical.

Notes on scope

No popcnt intrinsic (_mm_popcnt* / __builtin_popcount*) is used anywhere, so
dropping -mpopcnt on ARM needs no code change. And although <immintrin.h> was
included, no _mm256_*/AVX intrinsic is actually called; SIMDe's avx2.h is
still included on ARM to preserve the original include surface.

x86 is unchanged

  • Every source edit is inside an ARM #if; the #else branch is the original code.
  • The Makefile x86 branch reproduces the original flags and empty INCLUDES.
  • ksw2_extz2_sse.c stays out of OBJS, so the link line is unchanged on all arches.

Testing

make succeeds and produces a working binary on all four targets:

Target Result
Linux x86_64 CI: build + smoke assembly pass
Linux aarch64 CI: build + smoke assembly pass
macOS arm64 CI + local (Apple Silicon): build + smoke pass
macOS x86_64 local cross-compile passes (identical -msse4.2 -mpopcnt path, no SIMDe)

Determinism (the assembled graph is identical across architectures) is checked
two independent ways, both producing a byte-identical 67,868 B primary-contig GFA:

  • CI, Linux, GCC: x86_64 vs aarch64 (see the cross-arch graph determinism job).
  • Local, macOS, clang: x86_64 (under Rosetta) vs native arm64.

Locally on Apple Silicon, hifiasm -o test -t2 reads.fq.gz on a small synthetic
HiFi set assembles a single 59,609 bp contig, and repeated runs are byte-identical.

CI run (on my fork): https://github.com/BenjaminDEMAILLE/hifiasm/actions/runs/29027889900
The three Linux/macOS-arm64 build legs, both smoke legs, and the determinism job
are green; the macOS x86_64 leg was still waiting on a (scarce) Intel runner at
the time of writing and is additionally covered by the local cross-compile above.

BenjaminDEMAILLE and others added 3 commits July 9, 2026 16:59
Vendor the SIMD Everywhere (SIMDe) header tree, used to translate the x86
SSE/AVX intrinsics to ARM NEON on aarch64/arm64 builds.

Upstream: https://github.com/simd-everywhere/simde (MIT)
Pinned:   v0.8.2, commit 71fd833d9666141edcd1d3c109a80e228303d8d7

Only the simde/ header directory and COPYING are vendored (no tests/docs),
so a plain `git clone && make` works with no submodule step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Build hifiasm on aarch64/arm64 (Linux and Apple Silicon) in addition to x86.
All changes are gated behind `defined(__aarch64__) || defined(__arm__)`, so the
x86 build is byte-for-byte unchanged (same flags, same intrinsics).

- Makefile: detect the target via `uname -m`. On aarch64/arm64 drop the invalid
  `-msse4.2 -mpopcnt` (the latter is a hard error there), pass
  `-march=armv8-a+simd`, and add `-Ithird_party/simde`. `ARCH`/`ARCH_FLAGS`
  stay overridable (e.g. `make ARCH_FLAGS="-mcpu=neoverse-v2"`). No popcnt
  intrinsic is used anywhere, so dropping `-mpopcnt` needs no code change.

- Levenshtein_distance.h: on ARM include the SIMDe SSE4.2 + AVX2 headers instead
  of <emmintrin.h>/<nmmintrin.h>/<smmintrin.h>/<immintrin.h> (only SSE __m128i
  code is actually used; <immintrin.h> was included but no AVX intrinsic is
  called). GCC's x86 _mm_srli_epi32 accepts a runtime shift count; SIMDe/NEON
  requires a constant, so the two variable-count uses go through a small
  HA_SRLI_EPI32_VAR macro that expands to the identical intrinsic on x86.

- ksw2_extz2_sse.c: provide the SSE2/SSE4.1 kernel via SIMDe and define
  __SSE2__/__SSE4_1__ so the body compiles on ARM. This file is not in OBJS
  (its only caller is commented out), so it stays unlinked; it is ported so it
  builds cleanly if ever re-enabled, with no change to either binary.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Expand CI from a single x86_64 Linux build into a matrix that builds on
Linux x86_64, Linux aarch64, macOS x86_64, and macOS arm64. Add a smoke job
that assembles a tiny deterministic synthetic HiFi read set on both Linux
arches and asserts that the x86_64 and aarch64 assembly graphs are identical.

`-f0` disables the 16 GiB default bloom filter so the smoke test fits in a
standard runner's RAM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ARM compiling error

1 participant