Exceptions in thread during Convolve gets stuck waiting for results instead of throwing an error in the main function call

[UlrikFriisJensen]

In cases where the given 2 theta range is too broad, an IndexError will be thrown in the thread spawned for Convolve. However, this error does not get thrown in the main process and the PowderModel.simulate function call will therefore keep waiting for a result of the convolution.

Exception from the thread:

Error after manually killing the process:

Error can be reproduced using this cif file (format changed to .txt for compatability with GitHub) when simulating scattering in a 2 theta range from 5 to 80.
qmof-4148b62.txt

[dkriegner]

thanks for the report.
I can, however, not reproduce this problem:

import xrayutilities as xu
import numpy
tt = numpy.arange(5, 80, 0.01)
m = xu.materials.Crystal.fromCIF("qmof-4148b62.txt")
p = xu.simpack.PowderModel(xu.simpack.Powder(m, 1))
inten = p.simulate(tt)

I get this

Can you specify: platform, python version? Does the problem appear also if running with less processes? Do you set any special settings on the PowderModel/PowderDiffraction objects?

[UlrikFriisJensen]

I am working in Ubuntu 20.04.6 (through WSL) and use Python 3.10.13 and xrayutilities 1.7.6.

I don't think I use any special settings. Here is the code snippet I use for generating the diffraction:

import numpy as np
import xrayutilities as xru
from xrayutilities.materials.cif import CIFFile
from xrayutilities.materials.material import Crystal

qmof_id = 'qmof-4148b62'
mof_struc_v2 = CIFFile(f'./qmof_database/relaxed_structures/{qmof_id}.cif')
mof_crystal = Crystal(name='MOF', lat=mof_struc_v2.SGLattice())
wavelength = 1.5406 # Cu Kα
tth = np.linspace(5, 80, 751)
powder = xru.simpack.smaterials.Powder(mof_crystal, 1)
pm = xru.simpack.PowderModel(powder, I0=100)
intensities = pm.simulate(tth)
pm.close()

I haven't tried running with less processes. What argument should I use to do that?

[dkriegner]

Ok, with your code snippet I can reproduce the problem.
I need to investigate.

What I observed so far: I get the error with the structure from your cif file.
With a slightly changed version of your snippet I get

import numpy as np
import xrayutilities as xru
from xrayutilities.materials.cif import CIFFile
from xrayutilities.materials.material import Crystal

qmof_id = 'qmof-4148b62'
mof_struc_v2 = CIFFile(f'{qmof_id}.txt')
mof_crystal = Crystal(name='MOF', lat=mof_struc_v2.SGLattice())
wavelength = 1.5406 # Cu Kα
tth = np.linspace(5, 100, 9510)
powder = xru.simpack.smaterials.Powder(mof_crystal, 1)
pm = xru.simpack.PowderModel(powder, I0=100)
intensities = pm.simulate(tth)

with the output:

bad parameters:   325.0964    0.0002   -0.0000  -18.0304    1.0000
7 [-5.68434189e-14  1.22430954e-11  9.28723765e-12  6.33137986e-12
  3.37552208e-12  6.25277607e-13]
Exception in thread Thread-620 (_send_work):
Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.11/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dk/tmp/python-venv/lib/python3.11/site-packages/xrayutilities/simpack/powder.py", line 2139, in _send_work
    results = handler.calc(run, ttpeaks)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in calc
  File "/usr/lib/python3.11/multiprocessing/managers.py", line 837, in _callmethod
    raise convert_to_error(kind, result)
ValueError: Bad axial helper parameters

This seems to be deep in the code which was contributed. I try to inquire if the original author can identify the issue quicker.

[mendenmh]

The problem lies in some debugging code I left in the axial_helper_function a long time ago. At line 733 powder.py is a block of code which does a relative error check to look for accidental negative intensities coming from roundoff errors. It happens that something in the setup (maybe computing very far into the wings of peaks) is putting the calculation into a region of extremely weak absolute intensity, so tiny absolute errors are large relative errors. The error that begins with:

bad parameters:   325.0964    0.0002   -0.0000  -18.0304    1.0000
7 [-5.68434189e-14  1.22430954e-11  9.28723765e-12  6.33137986e-12
  3.37552208e-12  6.25277607e-13]

shows the intensity array with intensities of 1e-11 and below.

For the interim, you could just delete the lines starting at 733:

        # intensities are never less than zero!
        if min(intg[:-1]) < -1e-10 * max(intg[:-1]):
            print("bad parameters:", (5 * "%10.4f") %
                  (peakpos, innerbound, outerbound, k, y0))
            print(len(intg), intg[:-1])
            raise ValueError("Bad axial helper parameters")

I will think about a more elegant solution soon.

Doing this, and plotting, gives me:

[dkriegner]

@mendenmh Is the fix implememented in #181 worth merging? Or do you think you will have a better fix soon? I am asking because I might do a release again soon when I set up building with numpy 2.0.

[mendenmh]

I think it can be merged. Marcus Mendenhall Materials Measurement Science Division National Institute of Science and Technology 100 Bureau Dr. stop 8370 (217/B115) Gaithersburg, MD 20899 USA Phone: +1-301-975-8631

…

On May 22, 2024, at 2:48 AM, Dominik Kriegner ***@***.***> wrote: @mendenmh Is the fix implememented in #181 worth merging? Or do you think you will have a better fix soon? I am asking because I might do a release again soon when I set up building with numpy 2.0. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>