Make the SDK build parallelism configurable - #293
Open
artem-from-ua wants to merge 2 commits into
Open
Conversation
The SDK was built strictly serially: neither `build.sh tools` nor
`build.sh distribution` was ever given a `-j`, so both ran one job at a
time regardless of how many cores the build host had. On a four-core
NetBSD guest the `distribution` phase took 63 minutes at a load average
of about 1.
Pass `-j "$SDK_JOBS"` to both phases and default `SDK_JOBS` to 1, so the
serial behaviour is unchanged unless the caller asks for more:
SDK_JOBS=4 ./build/bootstrap.sh distribution
`build.sh` turns `-j N` into `parallel="-j N"` and hands it to make, so
this is the interface NetBSD already expects for this.
Samba needed no change: `SAMBA4X_JOBS` (default 2, via `SAMBA4_JOBS`)
was already overridable from the environment. The two knobs are now
symmetric and independent — `SDK_JOBS` for the cross toolchain and
sysroot, `SAMBA4X_JOBS` for Samba and its GnuTLS dependency chain.
Nothing here picks a value automatically. Parallel NetBSD builds have
historically been racy on some subtrees, so raising the default is left
to the caller, who knows the host.
artem-from-ua
marked this pull request as ready for review
August 31, 2026 01:53
Owner
|
'SDK_JOBS' defaulting to 1 means this now always passes '-j 1'. NetBSD This is not behavior-preserving and may break previously working makefiles, especially in the NetBSD 4 lanes, which I'm not going to fully audit. Leave 'SDK_JOBS' unset by default and add '-j' only when explicitly configured. |
artem-from-ua
marked this pull request as draft
September 3, 2026 12:34
The previous version defaulted SDK_JOBS to 1, which meant build.sh was
always given -j 1. That is not behavior-preserving: any -j turns off
make(1) compatibility mode, and the whole script for a target is then fed
to a single shell instead of one shell per line.
SDK_JOBS is now unset by default and the flag is built conditionally, so
an unset SDK_JOBS reproduces the original invocation exactly. Confirmed
against the real tree with build.sh -n:
SDK_JOBS unset -> nbmake-evbarm obj-tools
SDK_JOBS=4 -> nbmake-evbarm -j 4 obj-tools
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
artem-from-ua
marked this pull request as ready for review
September 3, 2026 12:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Neither
build.sh toolsnorbuild.sh distributionwas ever passed a-j, so the SDK builds one job at a time no matter how many cores the host has. On a four-core guest thedistributionphase takes over an hour at a load average of about 1.This adds
SDK_JOBS, passed through tobuild.sh -j, defaulting to 1 so behaviour is unchanged unless the caller opts in:build.shturns-j Nintoparallel="-j N"and hands it to make, so this is the interface NetBSD already expects.Samba needs no change —
SAMBA4X_JOBSwas already overridable from the environment. The two knobs are now symmetric and independent:SDK_JOBSfor the cross toolchain and sysroot,SAMBA4X_JOBSfor Samba and its GnuTLS dependency chain.Measured
Two full runs from a pristine NetBSD 10.1/aarch64 image under QEMU with hvf acceleration,
-smp 4, 4 GB RAM, on an Apple M1 Pro. Same image, same mirrors, nothing preinstalled in either — the bare starting point was asserted rather than assumed.SDK_JOBS=1,SAMBA4X_JOBS=2SDK_JOBS=4,SAMBA4X_JOBS=4bootstrap.sh toolsbootstrap.sh distributionsamba4x.shSource download is excluded from both columns:
git cloneis single-threaded either way and varies with the network, so it says nothing about parallelism.Parallelism was confirmed at runtime rather than inferred from the flag — 2-3 concurrent
cc1and 6-7nbmakeduring the distribution phase, at a load average of 3.13 against roughly 1.0 serially.The
samba4x.shrow moves becauseSAMBA4X_JOBSdiffers between the two runs, not because of anything this change does.Why the default stays at 1
Parallel NetBSD builds have historically been racy on some subtrees. Both phases completed cleanly at
-j4across several runs here, but that is evidence, not proof — and the caller knows their host better than the script does. Anyone on a host where a parallel build misbehaves simply does not set the variable.