Integrate symgc in druntime - #4
Conversation
0711d28 to
eed3016
Compare
| fi | ||
| cd runtime/symgc | ||
| # FIXME: hitting LLVM assertion with -g | ||
| DFLAGS="-O -gline-tables-only -singleobj" dub build -v --compiler=$PWD/../../../bootstrap-ldc/bin/ldc2 |
There was a problem hiding this comment.
-singleobj compiles the whole symgc project to a single object file (in the produced static lib) - which is then LLVM-optimized as a single unit (=> implicit cross-module optimizations).
|
Okay, seems to work with both shared and static druntime:
extern(C) void _d_register_sdc_gc();
void main() {
import std.stdio;
writeln("Hello world!");
auto dummyAlloc = new int;
auto dummyRef = &_d_register_sdc_gc; // make sure to link symgc from static druntime
} |
ada4d81 to
db4ea0e
Compare
|
Some metrics for the
|
|
And monorepo test runs with an unoptimized REPL:
|
Okay, the test runner was just hanging for the 5th consecutive run of 56 consecutive runs passed with an additional |
Okay, looking better now with the thread hooks: I must admit I was expecting something like ~2GB (starting from ~3.4 GB with old GC), based on old numbers from months ago (but the tests might have changed significantly since then too...). |
eb29994 to
d18fb28
Compare
Mainly to work around frontend assertions when building SIL.
Apparently needed for static druntime, where the strong `register_default_gcs()` symbol in the symgc object doesn't seem to override the default @weak one in druntime. Possibly because that would require dragging in the symgc object via an undefined symbol *before*.
…built with SDC GC To trivially switch to the new GC via env var `DRT_GCOPT=gc:sdc` without breaking any programs using a druntime library without SDC GC support (incl. LDC itself).
or terminating, to do necessary initialization or cleanup. Move blkcache destruction to the the cleanup function from the conservative GC.
|
Updated metrics for the
|
|
The stress loop |
|
And monorepo test runs with an unoptimized REPL:
|
|
Some stdlib unittest runner numbers from my local Windows qemu VM (no Sentinel), with 24 virtual CPU cores, using a little PowerShell helper to gather and print process stats: $args = @("-j1", "-q", "--DRT-gcopt=gc:sdc", "--DRT-gcopt=parallel:7")
foreach ($i in 1..3) {
Write-Host "Run $i with arguments:" $args
$p = Start-Process "bin\ut.exe" -ArgumentList $args -WorkingDirectory "C:\Users\packer\dev\SIL\stdlib\core" -PassThru -WindowStyle Hidden
$rss = 0
while (-not $p.HasExited) {
$rss = $p.PeakWorkingSet64
sleep 0.25
$p.Refresh()
}
$p.WaitForExit()
if ($p.ExitCode -ne 0) {
Write-Host -ForegroundColor Red " Exit code: $($p.ExitCode)"
}
Write-Host " Overall runtime: " ($p.ExitTime - $p.StartTime)
Write-Host " Peak working set (MB): " ($rss / 1MB)
Write-Host " Total CPU time (s): " $p.TotalProcessorTime.TotalSeconds
}
The RSS seems to vary wildly, with the old GC too. The new-GC runtime overhead seems similar to Linux, with similar speedups for the tweaked When it comes to multi-threading, I see mostly hangs when trying to run with higher |
|
And monorepo test runs on Windows with an unoptimized REPL - seems like the RSS is more stable than on Linux, and significantly smaller with the new GC: Edit: And as on Linux, the RSS seems to get worse with higher |
|
Okay the Windows hang seems gone now with the latest update. There was a single (unittest) failure in 250+ runs with |
|
The impact on RSS for the monorepo tests on Linux is very interesting, with absolutely brilliant numbers with
|
…HA images. Remove GHA for setting up ninja.
…r when appending to a large array that could potentially use extend, when the requested array size is within a certain size for a multiple of pages. The calculation had the wrong sign for the large padding (the 2 size_t + 1 byte), which made it end up sending a number very close to size_t.max into the extend function. This affected both appending and setting length performance, though appending also had a grow factor which may have mitigated this somewhat. (dlang/dmd!21616)
…rms where it's supported.
thread is no longer in the active thread list (and cannot be paused)
… _loadedDSOs' This error currently happens when a D DSO is unloaded in a thread the DSO wasn't registered with. So for example, loading a D Python extension DLL via dlopen/LoadLibrary in a Python worker thread (dragging in the druntime and Phobos DLLs etc., and initializing those in the worker thread), and one of the D DLLs then being unloaded in the Python *main* thread at Python exit. druntime has no way of registering the unknown main thread with druntime (registering the OS-initialized TLS data with the D GC, running the TLS module ctors etc.). [Normally, if you spawn a new thread via druntime, then that new thread inherits all the DSOs registered with the parent thread.] What we currently do in case a DSO is unloaded in such a thread is still running the TLS module dtors (as well as the shared module dtors etc.) of that DSO, but then printing that 'not in sync' message and aborting the process, so not finalizing any remaining DSOs. Try to handle such scenarios more gracefully by * emitting a stderr warning (per unloaded DSO) instead of aborting (and so not skipping any remaining DSOs anymore, as well as not tampering with the exit code), * emitting the warning *before* the DSO finalization, so that any issues there are easier to troubleshoot, and * excluding the TLS module dtors for such not-registered-in-current-thread DSOs, as the TLS module ctors most likely haven't run either in that thread.
The macOS 13 image isn't available anymore.
Limit the number of platforms that this is done on. A inspection of some libc implementations of fork has identified the main culprits, don't need to apply this to any others. MacOS testsuite also regressed as a result on calling this code, it's not clear why, but the backtrace is: ``` * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0) * frame #0: 0x00007ff81abe6ee3 libsystem_platform.dylib`_os_unfair_lock_recursive_abort + 23 frame #1: 0x00007ff81abe12da libsystem_platform.dylib`_os_unfair_lock_lock_slow + 247 frame #2: 0x00007ff81abccd44 libsystem_pthread.dylib`_pthread_atfork_prepare_handlers + 48 frame #3: 0x00007ff825dc2705 libSystem.B.dylib`libSystem_atfork_prepare + 25 frame #4: 0x00007ff81aac17e1 libsystem_c.dylib`fork + 24 frame #5: 0x0000000101f730ee test_runner`core.internal.backtrace.dwarf.resolveAddressesWithAtos(Location[]) + 210 ```
No description provided.