Skip to content

[Python] Sporadic failure of pyroot-pyz-ttree-branch in CI (vector proxy reference leak in vendored cppjit) #23677

Description

@guitargeek

Explain what you would like to see improved

Tracking issue: pyunittests-bindings-pyroot-pythonizations-pyroot-pyz-ttree-branch fails sporadically in CI on unrelated PRs, for example on #23649 (fedora43 and alma9 Debug jobs of run 37932249962). The cause is understood and fixed upstream; this issue exists so the failure is not reported twice. If you hit it on your PR, it is not caused by your change.

The failure looks like this:

test13_write_reference_proxy_branch (ttree_branch.TTreeBranch.test13_write_reference_proxy_branch) ... FAIL
test14_read_reference_proxy_branch (ttree_branch.TTreeBranch.test14_read_reference_proxy_branch) ... ERROR
...
AssertionError: 140174469364512 != 94329687085824 : branch 0 not bound to &MyHolder::myvec
...
TypeError: TObject object does not support indexing

test14 only fails as a consequence of test13 aborting before the file is written.

Root cause

vector_iter() in the vendored cppjit (bindings/pyroot/cppjit/src/cpyrt/Pythonize.cxx) takes two references to the container but the iterator releases only one, so every for x in vector_proxy leaks a reference to the proxy. The proxy, and the C++ object if Python owns it, are never freed.

In this test, the vector proxies leaked by test12 stay registered in cppjit's memory regulator under addresses that the closed file has already freed. When the new std::vector<double> of test13's MyHolder happens to land on one of those addresses, reading the pointer data member h.myvec returns the stale non-reference proxy instead of a fresh reference proxy, and the T** converter then binds the branch to the proxy's own pointer slot. Whether the allocator reuses one of the stale addresses depends on the heap state, hence the sporadic failures.

The leak entered ROOT with the cppjit vendoring (552d291, 2026-09-08). ROOT's previous cppyy copy was correct since #19017; the port of that change to the compiler-research fork (compiler-research/CPyCppyy#151) kept the original increment next to the new one, and cppjit inherited both.

Fix

Fixed upstream in compiler-research/cppjit@3df629f (compiler-research/cppjit#148). ROOT picks it up with the next cppjit sync; until then the test can fail on any PR.

ROOT version

master (since 552d291)

Installation method

Any build from source with PyROOT enabled

Operating system

Any (seen on fedora43 and alma9 in CI)

Additional context

The leak is not specific to this test: any iterated std::vector proxy stays alive, and if Python owns it, its C++ object stays alive too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions