multiprocessing not using multiple threads

[strategos-nzr]

Hello,

I am trying to run a powder diffraction model using multiple threads on windows.

If i try and set config.NTHREADS to any number, it does not seem to make an observable difference on the run time, and from CPU load it does not appear any multiprocessing is going on.

I attempted to write my own multiprocessing script using Pool() but it appears that this cannot be done since the code as implemented already parallelizes over the 2theta angle.

for reference, i am running this as a python script with python=3.11 .

Is there a straightforward way to enable parallelization on windows?

Code is attached, cif file is arbitrary.

code

```

import matplotlib.pyplot as pp
import numpy as np

import xrayutilities as xu
import multiprocessing as mp
from xrayutilities import config

config.NTHREADS=16

def main():
    
    BB= xu.materials.Crystal.fromCIF("Crystal.cif")
        
    tt= np.arange(5,80,0.02)
    
    size=np.arange(1,51,1)
    Ntt=len(tt)
    
    intensities=  np.zeros((len(tt),len(size)))
    print("Frame Size",np.shape(intensities))

    def compute(s):
            BB_powder=xu.simpack.Powder(BB,volume = 1 ,crystallite_size_gauss=s*1e-9)
            pm = xu.simpack.PowderModel(BB_powder,I0=100)
            sim =  pm.simulate(tt)
            pm.close()
            return sim
        
    for s in size:
        print("Num: ",s)
        intensities[:,s-1]= compute(s)
    print("Complete")
    

    
    pp.figure(figsize=(12,3))
    
    pp.xlim(8,15)
    for s in range(5):
        pp.plot(tt,intensities[:,10*s-1],label=str(s*10))
    pp.legend(bbox_to_anchor=(1,.5))

if __name__ == '__main__':
    mp.freeze_support()
    main()

</details>

[dkriegner]

thanks for reporting this problem. I am however not sure it's a fully valid issue.

I believe the overhead of the multiprocessing is eating up any gain in this case.
There is, however, a lot of optimization potential in the code. You should reuse one PowderModel as much as possible. There is some internal caching going on which will speed up things considerable.

You can use something like:

import matplotlib.pyplot as pp
import numpy as np

import xrayutilities as xu
import multiprocessing as mp
from xrayutilities import config

config.NTHREADS = 1

def main():

    BB= xu.materials.GaN

    tt= np.arange(5, 80, 0.0002)

    size=np.arange(1, 11, 1)
    Ntt=len(tt)

    intensities=  np.zeros((len(tt),len(size)))
    print("Frame Size",np.shape(intensities))
    BB_powder = xu.simpack.Powder(BB,volume=1)
    pm = xu.simpack.PowderModel(BB_powder, I0=100)

    for s in size:
        print("Num: ",s)
        pm.materials[0].crystallite_size_gauss = 20e-9 + s*10e-9 # E: missing whitespace around operator
        pm.materials[0].a = pm.materials[0].a*0.99
        intensities[:,s-1] = pm.simulate(tt) # E: missing whitespace around operator
    print("Complete")
    pm.close()


if __name__ == '__main__':
    mp.freeze_support()
    main()

But I agree it's disappointing that the multiprocessing here does not give a visible benefit. I can reproduce the issue also on Linux. I may need to investigate a bit more the reason where the speedup is so negligible.