Skip to content

Add builder flags for CI and resumed builds - #276

Open
maralcbr wants to merge 1 commit into
omacom:masterfrom
maralcbr:omacom/builder-flags
Open

Add builder flags for CI and resumed builds#276
maralcbr wants to merge 1 commit into
omacom:masterfrom
maralcbr:omacom/builder-flags

Conversation

@maralcbr

@maralcbr maralcbr commented Sep 2, 2026

Copy link
Copy Markdown

Summary

Three opt-in knobs for bin/build, plus one bug fix. The first matters most for aarch64 builds on the x86_64 host: under QEMU a full pass takes hours rather than minutes, so being able to resume an interrupted run instead of starting over is what makes that practical. The other two let the same script drive a multi-job pipeline (the fork's ARM CI today, or a prebuilt-upload path later) without diverging. Every default keeps today's behaviour; an unadorned bin/build is unchanged.

  • OMARCHY_KEEP_BUILD_WORKSPACE=1 keeps build-output/$MIRROR/$ARCH instead of wiping it, and build/build.sh now folds any packages already in the workspace into omarchy-build.db even when no database exists yet. A job can seed the workspace with packages built elsewhere (or resume after an interruption) and they take part in dependency resolution for what builds next. The version check still consults the published tree only, as today.
  • OMARCHY_SKIP_BUILDER_IMAGE=1 uses the omarchy-pkg-builder:latest-$ARCH-$MIRROR image already present instead of building it. A workflow builds the image once with an external BuildKit cache and fans out over package jobs that all run the same bytes. A missing image is an error, not a silent rebuild.
  • OMARCHY_DEFER_RUNTIME_DEPS=true builds the omarchy / omarchy-settings pair with --nodeps, installing only their makedepends and checkdepends (including the _$CARCH variants) explicitly. The pair depends on each other and on packages a sharded pipeline builds in other jobs, so they cannot resolve in isolation; the consumer of this mode installs the assembled set in one verified transaction afterwards. It is refused for anything but exactly that pair (or the -dev pair), checked both on the host before Docker starts and again inside the container.
  • make_dir_writable fix in helpers/docker-helpers.sh: chown -R can succeed on part of the tree and fail on files a previous container left behind as another uid. The old || chmod fallback only ran when chown failed outright, leaving those files unwritable. It now always follows with chmod.

This is the second of the Apple Silicon series (first: #275). Next is the aarch64 release plumbing: a separate aarch64 timer on the build host, arch-aware check-versions/auto-release/clean-repo, and the keyring bootstrap for an unpublished aarch64 tree. If you would rather not carry OMARCHY_DEFER_RUNTIME_DEPS or OMARCHY_SKIP_BUILDER_IMAGE on a single-host setup, say so and I will drop them from this PR.

Testing

  • bin/build --dry-run in an Arch Linux ARM container: a bad OMARCHY_DEFER_RUNTIME_DEPS value, no --package, a single half of the pair, and an unrelated package are each refused with a specific message and exit 1; the pair itself is accepted and reports the deferral.
  • On a MacBook Pro M1 Pro with native Docker: a plain bin/build --arch aarch64 --package retroarch-joypad-autoconfig-git builds the aarch64 edge image and the package; rerunning it with OMARCHY_SKIP_BUILDER_IMAGE=1 OMARCHY_KEEP_BUILD_WORKSPACE=1 uses the prepared image without rebuilding it and folds the kept package into the build database before building; OMARCHY_SKIP_BUILDER_IMAGE=1 against a mirror whose image does not exist fails before Docker runs; and OMARCHY_DEFER_RUNTIME_DEPS=true builds omarchy-settings + omarchy with --nodeps in the prepared image, installing only their makedepends first. For that run the Dockerfile's keyring bootstrap was pointed at a local copy of omarchy-keyring, because pkgs.omarchy.org/edge/aarch64 does not exist yet (Publish an aarch64 tree at pkgs.omarchy.org #199); the aarch64 image otherwise cannot build. The bootstrap for an unpublished tree is part of the follow-up PR.
  • These flags have run every Apple Silicon package build in the fork's CI since July.

Three opt-in knobs for bin/build, each defaulting to today's behaviour:

OMARCHY_KEEP_BUILD_WORKSPACE=1 keeps build-output/$MIRROR/$ARCH instead
of wiping it, and build/build.sh now folds any packages already there
into omarchy-build.db even when no database exists yet, so packages
built by an earlier job (or a previous, interrupted run) resolve as
dependencies of what builds next.

OMARCHY_SKIP_BUILDER_IMAGE=1 uses the omarchy-pkg-builder image already
present instead of building it, so a workflow can build the image once
with an external BuildKit cache and fan out over package jobs that all
run the same bytes. A missing image is an error, not a silent rebuild.

OMARCHY_DEFER_RUNTIME_DEPS=true builds the omarchy/omarchy-settings pair
with --nodeps, installing only their makedepends and checkdepends
explicitly. The pair depends on each other and on packages a sharded
pipeline builds in other jobs, so they cannot resolve in isolation; the
assembled set is installed in one verified transaction downstream. The
request is refused for anything but exactly that pair, on the host
before Docker starts and again inside the container.

Also fix make_dir_writable: chown -R can succeed on part of the tree
and fail on files a previous container left behind as another uid, and
the old `|| chmod` fallback only ran when chown failed outright. Always
follow with chmod.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant