[Python][RDF] Enable pure Python callbacks in RDataFrame

[siliataider]

This Pull request:

Now that #21261 landed, Python callables can be passed to C++ templated functions directly.

We apply this in RDataFrame: from now on, a Python function passed to Define or Filter without a decorator runs as it is in the Python interpreter. RDataFrame no longer compiles it with Numba behind the scenes. You can use any Python code in it, including ROOT classes and their methods, and any Python library:

import ROOT
from particle import Particle

def mass_difference(p: ROOT.Math.PtEtaPhiMVector, pdg: int) -> float:
    return p.M() - Particle.from_pdgid(pdg).mass / 1000  # PDG mass in GeV

df = (
    ROOT.RDataFrame(3)
    .Define("p", "ROOT::Math::PtEtaPhiMVector(40., 0., 0., 90. + rdfentry_)")
    .Define("pdg", "23")  # Z boson
    .Define("dm", mass_difference)  # the parameters `p` and `pdg` are read from the columns with the same names
)

The function needs type annotations: RDataFrame reads them to know the C++ types of the parameters and of the return value. These can be C++ type names, Python types or ROOT classes. A function without annotations raises.

Numba stays available as an opt in, with the same ROOT.Numba.Declare decorator as before:

@ROOT.Numba.Declare(["double"], "double")
def square(x):
    return x * x

df = df.Define("x", "(double) rdfentry_").Define("x2", square, ["x"])

Note

extra_args is now only supported for Numba decorated functions. Plain Python functions can use variables from the surrounding code instead.

[github-actions]

Test Results

    24 files      24 suites   4d 0h 58m 24s ⏱️
 3 888 tests  3 886 ✅ 0 💤 2 ❌
83 225 runs  83 223 ✅ 0 💤 2 ❌

For more details on these failures, see this check.

Results for commit 4035219.

♻️ This comment has been updated with latest results.

[siliataider]

Doesn't work out of the box because of compiler-research/cppjit#151

To land the functionality while that feature gets fixed, we will opt for explicit instanciation of Define by building the callback's function pointer signature using its annotations

[vepadulano]

A great improvement in ergonomics for Python users, thanks!