Lots of improvements#35
Open
oschulz wants to merge 24 commits into
Open
Conversation
The `run_everywhere` branch of `add_procinit_code` was unreachable (placed after a try/finally that always returned) and referenced an undefined variable `pids`. As a result, `@always_everywhere` only stored the init code for future workers instead of also running it on the present ones. Created by generative AI.
The `@deprecate` rules used `1::n` (a type-assert) instead of `1:n`, so calling the deprecated functions threw a `TypeError`. Created by generative AI.
Created by generative AI.
`_threading_run` computed task indices as offsets from the first selected thread ID, which only works for contiguous ascending ranges. Selections like `[1, 3]` or reversed order caused a BoundsError. Created by generative AI.
`worker_start_command` passed the user-supplied `julia_flags` twice: once directly and once within `additional_julia_flags`. Also adds tests for the SLURM/HTCondor worker start command generation and the SLURM flag/environment parsing. Created by generative AI.
On worker-init failure the occupancy count was decremented explicitly and then again by the subsequent `put!`, corrupting the occupancy state of the removed worker. Created by generative AI.
`memory_limit!(soft_limit)` threw a MethodError on non-Linux systems. Created by generative AI.
The docstring and `read_files` both use `cache_dir`; the implementation accidentally named the keyword `cache_dirname`, so the documented keyword was not accepted. Created by generative AI.
Fixes a broken string interpolation (`$err.n_tries`) in a debug message and removes two no-op statements. Created by generative AI.
Fixes wrong docstring headers and signatures, stale references, unclosed code fences, missing `(@ref)`s, duplicated see-also blocks and various typos. No functional changes. Created by generative AI.
Adds tests for the exception helpers and printover/display, wires the new slurm/htcondor test files into the test suite, tests write_worker_start_script generation, single-argument memory_limit! and immediate @always_everywhere propagation to existing workers. Created by generative AI.
Covers workpart input validation and scalar/vector worker selections, non-exception task failures, SLURM flag-parsing error branches, write_files/read_files edge cases (partial pre-existing files, on-demand cache-directory creation, double close, files appearing during write, closed FilesToRead), the @onthreads current-thread fast path, non-retriable exceptions in onworker and the ElasticManager pool callback. Line coverage now at ~95%. Created by generative AI.
The previous top-level const was evaluated at precompile time, baking the Julia executable path and flags of the build environment into the package image. Created by generative AI.
An argument that startswith "-<opt>" but has no value characters is always caught by the preceding exact-match branch. Created by generative AI.
It tested a README example (DistributedArrays-based parallel histogramming) that no longer exists, was not part of the test suite and its dependencies are not in the test environment. Created by generative AI.
With interactive threads present (julia -t N,M), the default threadpool
does not start at thread ID 1, so allthreads() selected the wrong
threads and ThreadLocal's nthreads()-sized value vector caused a
BoundsError when indexed by threadid().
allthreads() now returns the thread IDs of the default threadpool,
ThreadLocal sizes its value vector by Threads.maxthreadid(), and
getallvalues returns the values of the default-threadpool threads (in
thread-ID order). Behavior is unchanged when no interactive threads are
present. Also documents that ThreadLocal{T}() initializes the value on
each thread to T().
Created by generative AI.
One write_files testset created its target files in the current working directory instead of its mktempdir. Created by generative AI.
Created by generative AI.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #35 +/- ##
===========================================
+ Coverage 67.43% 83.85% +16.42%
===========================================
Files 20 20
Lines 1394 1375 -19
===========================================
+ Hits 940 1153 +213
+ Misses 454 222 -232 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
On 32-bit systems, memory sizes of 2 GiB and above overflowed the native Int, e.g. producing a zero heap-size-hint for workers. Created by generative AI.
The upcoming Test version (Julia nightly) misparses a leading begin/end block as the log pattern. Created by generative AI.
Created by generative AI.
Also removes the obsolete pre-1.10 branch (Julia compat is >= 1.10). Created by generative AI.
Splits _on_worker_impl into a single-attempt function that classifies each attempt into an outcome (succeeded, failed, timed out, worker unusable) and a retry-policy loop over outcomes. Previously, failures surfaced through different code paths depending on whether a timeout was set and whether the worker was local or remote, requiring duplicated retry logic and an outer catch that also handled its own inner throws. Worker losses still don't count against tries, but are now capped at 3 * tries instead of being tolerated indefinitely. Worker death is detected whether it surfaces as a thrown exception or as the future's value. The two onworker methods collapse into one and the return type is inferred via Base.promote_op instead of Core.Compiler.return_type. Created by generative AI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.