Skip to content

[FreeBSD] roofit tests segfault #13200

Open
#22929
Open
[FreeBSD] roofit tests segfault#13200
#22929
Assignees
pcanal
Labels
bugin:Core Librariesin:RooFit

Description

@mkrzewic
mkrzewic
opened on Jul 6, 2023

Check duplicate issues.

  • Checked for duplicates

Description

roofit tests (test-stressroofit-batchmode-*) segfault, maybe it rings a bell with the experts I cannot find the culprit (yet):

304: Test 52 : B Physics p.d.f.s......................................OK
304: Test 53 : Automated MC studies...................................OK
304: Test 54 : MC Study with chi^2 calculator.........................OK
304: Test 55 : MC Study with param rand. and Z calc...................OK
304: Test 56 : MC Studies with aux. obs. constraints..................OK
304:
304:  *** Break *** segmentation violation
304: ******************************************************************
304: *  SYS: FreeBSD hack 13.2-RELEASE-p1 FreeBSD 13.2-RELEASE-p1 GENERIC
304: ******************************************************************
304: ******************************************************************
304: *  ROOTMARKS =4262.5   *  Root6.29/01   20221221/1150
304: ******************************************************************
304: Time at the end of job = 18.882812 seconds
304:  Generating stack trace...
304:  0x0000000826fefe68 in _ZNSt3__112__hash_tableINS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEENS_4hashIS6_EENS_8equal_toIS6_EENS4_IS6_EEE25__emplace_unique_key_argsIS6_JRKS6_EEENS_4pairINS_15__hash_iteratorIPNS_1 at /usr/include/c++/v1/__hash_table:2070 from /home/mkrzewic/build/root/lib/libTree.so
304:  0x0000000830314f3e in TClass::TDeclNameRegistry::AddQualifiedName(char const*) at /usr/include/c++/v1/stdexcept:153 from /home/mkrzewic/build/root/lib/libCore.so
304:  0x0000000830315169 in TClass::InsertTClassInRegistryRAII::~InsertTClassInRegistryRAII() at /usr/include/c++/v1/stdexcept:153 from /home/mkrzewic/build/root/lib/libCore.so
304:  0x000000083032618c in TClass::SetUnloaded() at /usr/include/c++/v1/stdexcept:153 from /home/mkrzewic/build/root/lib/libCore.so
304:  0x00000008302d88cb in ROOT::RemoveClass(char const*, TClass*) at /usr/include/c++/v1/stdexcept:153 from /home/mkrzewic/build/root/lib/libCore.so
304:  0x00000008303344d7 in ROOT::TGenericClassInfo::~TGenericClassInfo() at /usr/include/c++/v1/stdexcept:153 from /home/mkrzewic/build/root/lib/libCore.so
304: CMake Error at /home/mkrzewic/build/root/RootTestDriver.cmake:232 (message):
304:   error code: 129
304:
304:
1/1 Test #304: test-stressroofit-batchmode-cpu ...***Failed   22.19 sec

Reproducer

ctest -V -R test-stressroofit-batchmode-cpu

ROOT version

master

Installation method

source

Operating system

freeBSD 13.2p1

Additional context

No response

Activity

  1. mkrzewic
    added
    bug
    on Jul 6, 2023

  2. mkrzewic commented on Jul 6, 2023

    @mkrzewic
    mkrzewic
    on Jul 6, 2023
    ContributorAuthor

    it craps out while removing class TBaseClass, all strings seem valid, segfaults somewhere in unordered_set

  3. pcanal commented on Jul 6, 2023

    @pcanal
    pcanal
    on Jul 6, 2023
    Member

    Can you upload the output of valgrind --suppressions=$ROOTSYS/etc/valgrind-root.supp stressroofit ... other args .. ?

  4. mkrzewic commented on Jul 6, 2023

    @mkrzewic
    mkrzewic
    on Jul 6, 2023
    ContributorAuthor

    valgrind_roofit.log

  5. jalopezg-git
    assigned
    pcanal
    on Jul 7, 2023
  6. jalopezg-git
    added
    in:Core Libraries
    on Jul 7, 2023

  7. ferdymercury commented on Mar 13, 2025

    @ferdymercury
    ferdymercury
    on Mar 13, 2025
    Collaborator

    Thanks for the log. Could you retry to see if it still crashes with current master?

  8. mkrzewic commented on Apr 8, 2025

    @mkrzewic
    mkrzewic
    on Apr 8, 2025
    ContributorAuthor

    current master does not build with tests enabled. I have the feeling the build system is quite fragile anyway, even without tests enabled it sometimes needs restarting with -j1 to finish.
    Anyway, with tests enabled, after adding a few system libraries to the linker invocation for cling it craps out on an assertion in TClingUtils:

    `
    Assertion failed: (decl.isFirstDecl() && "Couldn't trace back include from a decl" " that is not from an AST file"), function GetFileName, file /home/mkrzewic/src/root/core/clingutils/src/TClingUtils.cxx, line 3511.
    #0 0x000000082ade5008 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) Signals.cpp:0:0
    #1 0x000000082ade2b87 llvm::sys::RunSignalHandlers() Signals.cpp:0:0
    #2 0x000000082ade5658 SignalHandler(int) Signals.cpp:0:0
    #3 0x000000082bede410 (/lib/libthr.so.3+0x1a410)
    #4 0x000000082bedd9cb (/lib/libthr.so.3+0x199cb)
    #5 0x000000082134c2d3 ([vdso]+0x2d3)
    #6 0x000000082fb641ba thr_kill (/lib/libc.so.7+0x1381ba)
    #7 0x000000082fadd5d4 _raise (/lib/libc.so.7+0xb15d4)
    #8 0x000000082fb90b59 abort (/lib/libc.so.7+0x164b59)
    #9 0x000000082fac07c1 (/lib/libc.so.7+0x947c1)
    #10 0x000000082632a415 ROOT::TMetaUtils::GetFileName(clang::Decl const&, cling::Interpreter const&) /home/mkrzewic/src/root/core/clingutils/src/TClingUtils.cxx:3512:10
    #11 0x000000082648bc8d RecordDecl2Headers(clang::CXXRecordDecl const&, cling::Interpreter const&, std::__1::set<clang::CXXRecordDecl const*, std::__1::less<clang::CXXRecordDecl const*>, std::__1::allocator<clang::CXXRecordDecl const*>>&) /home/mkrzewic/src/root/core/dictgen/src/rootcling_impl.cxx:3158:25
    #12 0x000000082648b2b1 RecordDecl2Headers(clang::CXXRecordDecl const&, cling::Interpreter const&, std::__1::set<clang::CXXRecordDecl const*, std::__1::less<clang::CXXRecordDecl const*>, std::__1::allocator<clang::CXXRecordDecl const*>>&) /home/mkrzewic/src/root/core/dictgen/src/rootcling_impl.cxx:3108:43
    #13 0x000000082648b2b1 RecordDecl2Headers(clang::CXXRecordDecl const&, cling::Interpreter const&, std::__1::set<clang::CXXRecordDecl const*, std::__1::less<clang::CXXRecordDecl const*>, std::__1::allocator<clang::CXXRecordDecl const*>>&) /home/mkrzewic/src/root/core/dictgen/src/rootcling_impl.cxx:3108:43
    #14 0x000000082648c07b ExtractHeadersForDecls(std::__1::vector<ROOT::TMetaUtils::AnnotatedRecordDecl, std::__1::allocatorROOT::TMetaUtils::AnnotatedRecordDecl> const&, std::__1::vector<clang::TypedefNameDecl const*, std::__1::allocator<clang::TypedefNameDecl const*>>, std::__1::vector<clang::FunctionDecl const*, std::__1::allocator<clang::FunctionDecl const*>>, std::__1::vector<clang::VarDecl const*, std::__1::allocator<clang::VarDecl const*>>, std::__1::vector<clang::EnumDecl const*, std::__1::allocator<clang::EnumDecl const*>>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>, std::__1::list<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>>>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator> const, std::__1::list<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>>>>>>&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>, std::__1::list<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>>>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator> const, std::__1::list<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>>>>>>&, cling::Interpreter const&) /home/mkrzewic/src/root/core/dictgen/src/rootcling_impl.cxx:3238:41
    #15 0x000000082649ac4c RootClingMain(int, char**, bool) /home/mkrzewic/src/root/core/dictgen/src/rootcling_impl.cxx:4979:7
    #16 0x00000008264a109a ROOT_rootcling_Driver /home/mkrzewic/src/root/core/dictgen/src/rootcling_impl.cxx:6247:16
    #17 0x0000000000201d7a main /home/mkrzewic/src/root/main/src/rootcling.cxx:43:4
    #18 0x000000082fab1c3a __libc_start1 (/lib/libc.so.7+0x85c3a)
    PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
    Stack dump:
    0. Program arguments: /home/mkrzewic/build/root/bin/rootcling -rootbuild -v2 -f G__ROOTGpadv7.cxx -cxxmodule -s /home/mkrzewic/build/root/lib/libROOTGpadv7.so -m Core.pcm -m RIO.pcm -excludePath /home/mkrzewic/src/root -excludePath /home/mkrzewic/build/root/ginclude -excludePath /home/mkrzewic/build/root/externals -excludePath /home/mkrzewic/build/root/builtins -writeEmptyRootPCM -compilerI/usr/include/c++/v1 -compilerI/.bastille/usr/lib/clang/18/include -compilerI/usr/include -compilerI/.bastille/usr/lib/clang/18/include -compilerI/usr/include -I/home/mkrzewic/build/root/include -I/usr/local/include -I/home/mkrzewic/src/root/graf2d/gpadv7/inc -I/home/mkrzewic/src/root/core/unix/../clib/res -I/home/mkrzewic/src/root/core/unix/inc -I/usr/local/include -I/home/mkrzewic/src/root/core/zstd/inc -I/home/mkrzewic/src/root/core/lz4/inc -I/home/mkrzewic/build/root/include -I/home/mkrzewic/src/root/core/lzma/inc -I/home/mkrzewic/src/root/core/zip/inc -I/home/mkrzewic/src/root/core/thread/inc -I/home/mkrzewic/src/root/core/textinput/src -I/home/mkrzewic/src/root/core/textinput/inc -I/home/mkrzewic/src/root/core/rint/inc -I/home/mkrzewic/src/root/core/meta/inc -I/home/mkrzewic/src/root/core/gui/inc -I/home/mkrzewic/src/root/core/foundation/res -I/home/mkrzewic/src/root/core/foundation/inc -I/home/mkrzewic/src/root/core/cont/inc -I/home/mkrzewic/src/root/core/clingutils/res -I/home/mkrzewic/src/root/core/clingutils/inc -I/home/mkrzewic/src/root/core/clib/inc -I/home/mkrzewic/src/root/core/base/v7/inc -I/home/mkrzewic/src/root/core/base/inc -I/home/mkrzewic/build/root/ginclude -I/home/mkrzewic/src/root/io/io/inc -I/home/mkrzewic/src/root/core/clib/res ROOT/RCanvas.hxx ROOT/RFrame.hxx ROOT/RMenuItems.hxx ROOT/RColor.hxx ROOT/RDisplayItem.hxx ROOT/RAttrMap.hxx ROOT/RAttrBase.hxx ROOT/RAttrAggregation.hxx ROOT/RAttrAxis.hxx ROOT/RAttrBorder.hxx ROOT/RAttrLine.hxx ROOT/RAttrFill.hxx ROOT/RAttrFont.hxx ROOT/RAttrMarker.hxx ROOT/RAttrMargins.hxx ROOT/RAttrText.hxx ROOT/RAttrValue.hxx ROOT/RAxisDrawable.hxx ROOT/RPalette.hxx ROOT/RPaletteDrawable.hxx ROOT/RDrawable.hxx ROOT/ROnFrameDrawable.hxx ROOT/RDrawableRequest.hxx ROOT/RStyle.hxx ROOT/RPadDisplayItem.hxx ROOT/RPadExtent.hxx ROOT/RPadBase.hxx ROOT/RPad.hxx ROOT/RPadLength.hxx ROOT/RPadPos.hxx ROOT/RPave.hxx ROOT/RVirtualCanvasPainter.hxx ROOT/TObjectDisplayItem.hxx ROOT/TObjectDrawable.hxx /home/mkrzewic/src/root/graf2d/gpadv7/inc/LinkDef.h
    Subprocess aborted
    gmake[2]: *** [graf2d/gpadv7/CMakeFiles/G__ROOTGpadv7.dir/build.make:147: graf2d/gpadv7/G__ROOTGpadv7.cxx] Error 1
    gmake[1]: *** [CMakeFiles/Makefile2:64372: graf2d/gpadv7/CMakeFiles/G__ROOTGpadv7.dir/all] Error 2
    gmake: *** [Makefile:166: all] Error 2

    `

  9. mkrzewic commented on Apr 8, 2025

    @mkrzewic
    mkrzewic
    on Apr 8, 2025
    ContributorAuthor

    btw. the crash happens even before adding libprocstat and libutil (those are used somewhere to get the program name on FreeBSD as procfs is deprecated)

  10. guitargeek
    added
    in:RooFit
    on Nov 28, 2025

  11. guitargeek commented on Jul 27, 2026

    @guitargeek
    guitargeek
    on Jul 27, 2026
    Contributor

    Thanks for the detailed traces! This looks like an static initialization order fiasco. I have opened a PR that should fix it:

  12. guitargeek
    added a commit that references this issue on Jul 29, 2026

    [core] Make TClass decl-name registry an immortal singleton

    89dc2d4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugin:Core Librariesin:RooFit

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions