[WIP] SIMD: Vectorized Transcendentals in Element Pushes

[ax3l]

Summary

With ImpactX_SIMD=ON, the elements that call sin, cos, sinh or cosh per particle got very little out of it: SIMD hardware has no transcendental instructions, so std::experimental::simd evaluates those one lane at a time. A vectorized ChrQuad push was barely faster than the scalar one.

This calls the amrex::Math transcendentals instead, which hand a whole SIMD register to a vector math library (glibc's libmvec).

Measured end to end on examples/apochromatic, 2·10⁶ particles, 4 slices, one thread pinned to a P-core on an otherwise idle machine, alternating runs, best of seven each:

	before	after
push::ChrQuad — sin, cos, sinh, cosh per particle	1.917 s	0.810 s	2.37×
push::ChrDrift — no transcendentals, the control	0.066 s	0.065 s	1.02×
whole run	2.783 s	1.648 s	1.69×

Run-to-run spread was 3–6 % across the seven repetitions.

ChrDrift is the useful part of that table: it goes through the same ParallelForSIMD machinery and does not change, which is what says the gain comes from the transcendentals rather than from anything else moving.

Correctness

Same example, same seed, the two builds differing only in whether the vector math library is reachable:

• the reference particle is bit-identical,
• the beam moments differ by at most 1.1e-10 relative over 50 steps, worst in dispersion_y — a quantity built from cancellation, which is where the few ULP a vector math library costs will concentrate.

libmvec documents a maximum error of 4 ULP where the scalar routines stay below 1, so results are no longer bit-wise reproducible against a scalar build. For a tracking code that is the usual trade; it is worth being explicit about it.

Why the calls are qualified

amrex::Math::sin(x), not sin(x). An unqualified call on a SIMD argument resolves to the SIMD library's own overload through argument-dependent lookup, and no using fixes it: the library's overload either ties with the AMReX one (ambiguous) or is more specialized and wins (compiles, silently slow). Both were verified. This is a property of ADL rather than of any particular implementation.

That is the same spelling amrex::Math::powi and amrex::Math::sincos already use here, so it reads consistently with the surrounding code.

Scope

36 call sites across six elements: ChrQuad, ChrPlasmaLens, ChrUniformAcc, ExactQuad, ExactSbend, ShortRF.

The rest needed nothing. Most elements precompute their trigonometry scalar-side in compute_constants, so their per-particle path is already pure arithmetic. And the SIMD kernels themselves contain no transcendentals: beamoptic.H only dispatches to the elements, and the beam-moments reduction behind ParticleReduceSIMD is sums of products. sqrt and abs are left unqualified on purpose — they map onto hardware instructions and the SIMD library already does the right thing.

Dependencies — this is why it is WIP

The two halves of this live upstream and are not merged yet, so ABLASTR.cmake and the CI dependency scripts point at branches:

• AMReX: ax3l/amrex@topic-simd-vecmath — [WIP] SIMD: Transcendental Math for amrex::Math (sinh, cos, exp, ...) AMReX-Codes/amrex#5644, adds the amrex::Math SIMD overloads
• vir-simd: ax3l/vir-simd@topic-vecmath — adds vir/simd_vecmath.h, which calls libmvec's entry points directly

Each is marked with a TODO to point back at the upstream release once they land. Nothing here needs compiler flags: the vector math calls go to libmvec's entry points by name, so there is no -ffast-math, no -fno-math-errno, and no dependence on the auto-vectorizer.

Where a vector math library is not available — another architecture, another libc, an older glibc, or an older vir-simd — everything still builds and simply evaluates one lane at a time, as today.

Testing

Built with ImpactX_SIMD=ON against both dependency branches; the resulting binary links libmvec.so.1 and contains 360 vector math call sites (143 _ZGVdN4v_cos, 137 _ZGVdN4v_sin, 30 each sinh/cosh, plus log and asin).

All simulation tests pass. Two caveats on that, since they matter for reading the CI result: solenoid.restart.run fails locally only because I configured without openPMD, and the Python .analysis/.plot tests could not run in my environment at all (ctest records an absolute python3 at configure time and that interpreter had no numpy). The numerical check above was done directly instead, by diffing the diagnostics of two builds.

To do

• Land the AMReX and vir-simd sides, then point the refs back at releases.
• Check whether any of the remaining elements would benefit from moving work out of compute_constants now that per-particle transcendentals are cheap.
• GPU builds are untouched by this and untested here: the SIMD overloads are host-only and the scalar ones are unchanged, but that deserves a CI confirmation rather than an argument.

🤖 Generated with Claude Code

[codspeed-hq]

Merging this PR will regress 1 benchmark

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 7 improved benchmarks
❌ 1 regressed benchmark
✅ 62 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	test_ShortRF[nospin]	246.2 µs	274.4 µs	-10.29%
⚡	test_ChrQuad[nospin]	2.4 ms	1.1 ms	×2.2
⚡	test_ChrQuad[spin]	3.6 ms	1.7 ms	×2.1
⚡	test_ChrPlasmaLens[spin]	2 ms	1.3 ms	+49.28%
⚡	test_ChrAcc[nospin]	743.4 µs	575.7 µs	+29.13%
⚡	test_ChrAcc[spin]	1.6 ms	1.3 ms	+24.27%
⚡	test_ExactSbend[nospin]	698.6 µs	582.8 µs	+19.87%
⚡	test_ExactSbend[spin]	1.2 ms	1.1 ms	+13.28%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ax3l:topic-simd-vecmath (55c90ed) with development (e11fd54)}