Skip to content

(PE-45366) Add pe_run_in_parallel option, serial by default on macOS - #309

Merged
Magisus merged 1 commit into
mainfrom
worktree-configurable-parallel-execution
Jul 30, 2026
Merged

(PE-45366) Add pe_run_in_parallel option, serial by default on macOS#309
Magisus merged 1 commit into
mainfrom
worktree-configurable-parallel-execution

Conversation

@Magisus

@Magisus Magisus commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

What's this PR do?

Adds a pe_run_in_parallel switch controlling whether beaker-pe's PE install/upgrade steps fork a child process per host or run serially, and changes the default to serial on macOS.

Several install/upgrade steps act on many hosts by fork()ing a child per host, each with its own Net::SSH connection. On macOS, forking a Ruby process that has already used Net::SSH deadlocks the child, so those steps hang until beaker's watchdog kills them after 1800s (RuntimeError: Child process ran longer than timeout of 1800). This never reproduces on Linux/CI.

A new private helper pe_run_in_parallel? on PEUtils resolves a single boolean, precedence highest first:

  1. BEAKER_PE_RUN_IN_PARALLEL env var (1/true/yes = parallel, 0/false/no = serial)
  2. options[:pe_run_in_parallel] option
  3. Default: false on macOS, true on every other platform

The 5 hardcoded :run_in_parallel => true literals in lib/beaker-pe/install/pe_utils.rb now call the helper. Mac developers get serial execution out of the box, so the deadlock no longer bites; a developer on any platform can override the default either direction via the option or env var.

Should any of this be tested outside the normal PR CI cycle?

CI runs on Linux, where the default resolves to true, so behavior is byte-for-byte unchanged there. The macOS serial path is exercised by unit specs (the platform default, option, and env var all resolve to false and reach the call sites); the underlying deadlock itself only reproduces on a Mac and isn't covered by CI.

Any background context you want to provide?

  • The default is derived from RUBY_PLATFORM (the machine running beaker, which is the process that forks), matching /darwin/.
  • The spec suite is made hermetic against both a developer's exported BEAKER_PE_RUN_IN_PARALLEL and being run on a Mac, by stubbing the env var and RUBY_PLATFORM for every example. Pre-existing :run_in_parallel => true specs still pass because they run against the pinned non-macOS platform.
  • README gains a ## Configuration section documenting the option, env var, precedence, and the macOS default.
  • Labeled enhancement for the changelog generator (feature → minor version bump).

🤖 Generated with Claude Code

@Magisus
Magisus requested review from a team as code owners July 28, 2026 00:17
@Magisus
Magisus requested review from a team as code owners July 28, 2026 00:17
@Magisus Magisus changed the title (maint) Add pe_run_in_parallel option to run install steps serially Add pe_run_in_parallel option to run install steps serially Jul 28, 2026
@Magisus
Magisus force-pushed the worktree-configurable-parallel-execution branch from fff2d7c to 28f6d95 Compare July 28, 2026 00:24
@Magisus Magisus changed the title Add pe_run_in_parallel option to run install steps serially (PE-45366) Add pe_run_in_parallel option to run install steps serially Jul 28, 2026
Several PE install/upgrade steps act on many hosts by fork()ing a child
process per host, each with its own Net::SSH connection. On macOS, forking a
Ruby process that has already used Net::SSH deadlocks the child, so those
steps hang until beaker's watchdog kills them after 1800s
("RuntimeError: Child process ran longer than timeout of 1800"). The hang
never reproduces on Linux/CI. Because beaker always holds SSH connections
before forking, the only reliable workaround is to run serially, and the 5
parallel sites were hardcoded to :run_in_parallel => true with no supported
way to turn them off.

Add a private pe_run_in_parallel? helper on PEUtils that resolves a single
boolean, precedence highest first:

  1. BEAKER_PE_RUN_IN_PARALLEL env var (1/true/yes = parallel,
     0/false/no = serial)
  2. options[:pe_run_in_parallel]
  3. default: false on macOS, true on every other platform

The 5 hardcoded :run_in_parallel => true literals in
lib/beaker-pe/install/pe_utils.rb now call the helper. macOS defaults to
serial so Mac developers no longer hit the deadlock out of the box; Linux/CI
behavior is unchanged. A developer can override the default either way via the
option or env var.

Also document the option, env var, precedence, and macOS rationale in the
README, and stub BEAKER_PE_RUN_IN_PARALLEL and RUBY_PLATFORM in the pe_utils
spec so the suite stays hermetic against a developer's exported value and
against being run on a Mac.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Magisus
Magisus force-pushed the worktree-configurable-parallel-execution branch from 28f6d95 to 94a6e2a Compare July 29, 2026 23:42
@Magisus Magisus changed the title (PE-45366) Add pe_run_in_parallel option to run install steps serially (PE-45366) Add pe_run_in_parallel option, serial by default on macOS Jul 29, 2026
@Magisus
Magisus merged commit 63fcfac into main Jul 30, 2026
3 of 4 checks passed
@Magisus
Magisus deleted the worktree-configurable-parallel-execution branch July 30, 2026 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants