Skip to content

Avoid scaled-index table loads on the double path under -fPIC (fixes ~2x slowdown on AMD Zen4) - #21

Merged
xjb714 merged 1 commit into
xjb714:mainfrom
Antares0982:zen4-fix
Jul 23, 2026
Merged

xjb714 merged 1 commit into
xjb714:mainfrom
Antares0982:zen4-fix

Conversation

@Antares0982

Copy link
Copy Markdown
Contributor

Summary

Under -fPIC/-fPIE, the double path (xjb64) is ~2x slower on AMD Zen4
than it should be, while the float path and all other x86-64 microarchitectures
are unaffected. This PR folds two critical-path table loads into base+disp form,
recovering 1.55×–1.85× on Zen4 with no change to output and no change to
non-PIC builds.

Root cause

With -fPIC, static data uses RIP-relative addressing, so the double_table
base is pinned into a register and the hot pow10_double / exp_result_double
lookups compile to scaled-index loads:

; non-PIC:  movq  double_table+5184(,%r10,8), ...   (absolute disp)
; PIC:      movq  5184(%rbx,%r10,8), ...            (base + scaled index)

The pow10 load sits on the double path's critical dependency chain (its result
feeds the 128-bit multiply). On Zen4, a scaled-index load with a large
displacement misses the fast AGU path and adds load-use latency, stalling the
chain every call. Intel's AGU has no such penalty, which is why only Zen4
regresses — and why it reproduces on clang, gcc and icpx alike (it's the
addressing mode, not a compiler artifact). The float path is smaller and not
register-starved, so it's unaffected.

Fix

Fold the index into the pointer (moving address math off the critical load-use
chain) and use an empty asm barrier to stop the compiler from re-merging it
back into a scaled-index load:

const u64* pow10_ptr = t->pow10_double + 323 * 2 + 2 + k * 2;
#if defined(__x86_64__) && defined(__PIC__) && (defined(__clang__) || defined(__GNUC__))
    asm("" : "+r"(pow10_ptr));   // fold scaled index into base -> base+disp load
#endif
*pow10_hi = pow10_ptr[0];
*pow10_lo = pow10_ptr[1];

Same treatment for exp_result_double[e10 + 324].

Gated on PIC so static / LTO / -no-pie builds are byte-for-byte
unchanged (there the loads are already base+abs-disp, no index penalty).
znver4 can't be used because the affected builds are generic x86-64, where
that macro isn't defined.

Results — AMD Ryzen 7 7840H (Zen4), double, median ns/call

┌──────────────┬────────┬───────┬─────────┐
│ build │ before │ after │ speedup │
├──────────────┼────────┼───────┼─────────┤
│ clang SSE2 │ 16.68 │ 10.77 │ 1.55× │
├──────────────┼────────┼───────┼─────────┤
│ clang SSE4.1 │ 15.36 │ 8.88 │ 1.73× │
├──────────────┼────────┼───────┼─────────┤
│ gcc SSE2 │ 17.33 │ 10.63 │ 1.63× │
├──────────────┼────────┼───────┼─────────┤
│ gcc SSE4.1 │ 16.72 │ 9.08 │ 1.84× │
└──────────────┴────────┴───────┴─────────┘

perf stat (clang/SSE4.1): cycles 11.19G → 8.00G (−28%) at flat instruction
count and identical L1 load/miss counts, IPC 2.80 → 3.96 — i.e. pure load-latency
stall relief, exactly matching the diagnosis.

Cost

On Intel (i7-13700K) the extra leas cost ~4% under PIC (no scaled-index penalty
to recover). Non-PIC Intel builds are unchanged. The Zen4 win far outweighs this,
and xjb64 remains faster than the alternatives on Intel either way.

Correctness

Output is bit-identical to the current code over 30M random doubles plus edge
cases (0, -0, ±inf, nan, subnormals, 5e-324, 1e15/1e16 boundaries).
The asm barrier is a pure codegen barrier — semantically a no-op.

🤖 Generated with Claude Code

@xjb714
xjb714 merged commit 38d91db into xjb714:main Jul 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants