compilation from source issue on Mac (M3 specific?)

[Sebastian-Belkner]

Hi Martin,

I've been running into some funny behaviour.

Compiling DUCC from source,

pip3 install --no-binary ducc0 --user ducc0

I get wrong-ish results, here a minimum-working example

from lenspyx.lensing import get_geom
import healpy as hp
import numpy as np
from ducc0.sht.experimental import adjoint_synthesis

mask = np.zeros(hp.nside2npix(1))
mask[[1]] = 1
mask = hp.ud_grade(mask, nside_out=2048)
hp.mollview(mask)

geomlib = get_geom(('healpix', {'nside':2048}))
lmax = 1000

ebmaps = np.ones(shape=(2,hp.nside2npix(2048)), dtype=np.float64)

alme_ducc = adjoint_synthesis(map=np.atleast_2d(ebmaps[0]*mask), theta=geomlib.theta, lmax=lmax, mmax=lmax, nphi=geomlib.nph, spin=0, phi0=geomlib.phi0, nthreads=4, ringstart=geomlib.ofs)

hp.mollview(hp.alm2map(alme_ducc[0], nside=512))
plt.show()

plt.plot(hp.alm2cl(alme_ducc[0]))
plt.loglog()
plt.show()

Output is

There are some weird stripes and nonzero pixels in an area far away from what is unmasked

Instead, when I use the precompiled version,
pip3 install ducc0

this problem disappears,

I am sitting on a Mac, M3.

One thing that I already found about my metal machine is that longdouble is treated as if its just double. Not sure if that is causing it?

Additional observations

• The same stripping and non-zero (or non--372345) pixel behavior occurs symmetrically at the south pole.
• I don't think it’s related to the masking, as I observe similar anomalies on the full sky, with the poles appearing slightly different.

[mreineck]

Hi Sebastian, thanks for the report!

I'm pretty sure that longdouble isn't the issue here ... all SHT-related fnctionality in ducc only uses standard double.

The fact that the pixels oscillate quite a bit after the adjoint_synthesis is expected, but the stripes you see are definitely not OK. I have no immediate good explanation, except perhaps that trigonometric functions on that architecture have really bad accuracy when optimizing too much. But that seems quite far fetched.

I don't have access to the hardware, so if you have a bit of time to spare could you please run the following experiments for me?

• rerun the failing test, but with nthreads=1
• reinstall ducc using DUCC0_OPTIMIZATION=portable pip3 install --no-binary ducc0 --user ducc0 and retry the failing test
• reinstall ducc using DUCC0_OPTIMIZATION=none pip3 install --no-binary ducc0 --user ducc0 and retry the failing test (This might take a few minutes ...)

[mreineck]

One more thing: Can you do the pip install with a -v flag to find out the compiler that is used internally, and then run that compiler in turn with a -v flag, so that it prints its own version?

[mreineck]

Interesting: the stripes start exactly at the latitude where the number of pixels per ring starts to decrease towards the pole. Perhaps the issue isn't SHT-related, but rather FFT-related?

[Sebastian-Belkner]

Thanks for your input!

installation on Mac, M3 - clang++ 16.0.0	behaviour	pass
pip3 install --no-binary ducc0 --user ducc0, test on nthreads = 4	unusual striping	❌
pip3 install --no-binary ducc0 --user ducc0, test on nthreads = 1	unusual striping	❌
DUCC0_OPTIMIZATION=portable pip3 install --no-binary ducc0 --user ducc0	unusual striping	❌
DUCC0_OPTIMIZATION=none pip3 install --no-binary ducc0 --user ducc0	no issue	✅
pip3 install ducc0	no issue	✅

pip install -v gives,

`clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 ...

clang++ -v gives,

Apple clang version 16.0.0 (clang-1600.0.26.4)
Target: arm64-apple-darwin23.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

[mreineck]

Thanks for doing this!

Unfortunately your results make the whole thing even more bizarre ... the binary wheel on PyPI is actually built with DUCC0_OPTIMIZATION=portable, which fails when you do it locally :-/

At the moment I'm tempted to blame this on the compiler, which is still quite new (released September 2024), but of course this is not very satisfactory... Still, I have no good idea how to debug this problem further.

As long as you don't have to modify the ducc sources yourself, I think that you should be fine with the binary wheel from PyPI. The performance loss mentioned in ducc's README mostly concerns x86 CPUs; I don't think the difference is that big on Apple silicon.

In any case, let's keep this issue open, so other users can find it!

[Sebastian-Belkner]

Oh, that's also with =portable ..
..ok I just redid the =portable to make sure I didn't mix up things, also pip cache remove ducc0 (like before), to make sure its truly doing things, but yes, nothing changed.

I am good for now, precompiled is working for me :) The speed up I am loosing on my local machine is acceptable for the things I do there.

Ping me if you need more checks

[mreineck]

I'll probably make a new release soon, which contains many changes to the build machinery. It may be good to re-test then, altough I'm not very hopeful that things will spontaneously improve.

[mreineck]

Version 0.37 is out now!

[Sebastian-Belkner]

Great! Unfortunately, as you expected, that did not fix the problem when compiling from source, though. Precompiled 0.37 works, again.

[mreineck]

Thanks for trying it out! It's a relief at least that the binary wheel still works.