Add multi-arch builder pipeline for amd64 + arm64 support - #314
Open
jvulgan wants to merge 5 commits into
Open
Conversation
Reviewer's GuideIntroduces a new Tekton multi-architecture builder pipeline that fans out per-platform builds via buildah-remote-oci-ta, assembles a multi-arch image index, and adjusts the builder image scripts/Containerfile to auto-detect the target architecture when PLATFORM/AUDITWHEEL_ARCH are not explicitly set. The existing PR/push pipelines are re-pointed to this new multiarch pipeline and configured to build amd64 and arm64 images. Sequence diagram for the new multi-arch Tekton build-multiarch pipelinesequenceDiagram
participant Pipeline as build_multiarch
participant init
participant clone_repository as git_clone_oci_ta
participant prefetch as prefetch_dependencies_oci_ta
participant build_images as buildah_remote_oci_ta
participant build_index as build_image_index
Pipeline->>init: task init
init-->>Pipeline: result build
Pipeline->>clone_repository: task git-clone-oci-ta
clone_repository-->>Pipeline: results commit, SOURCE_ARTIFACT
Pipeline->>prefetch: task prefetch-dependencies-oci-ta
prefetch-->>Pipeline: results SOURCE_ARTIFACT, CACHI2_ARTIFACT
loop [for each PLATFORM in build-platforms]
Pipeline->>build_images: task buildah-remote-oci-ta (matrix PLATFORM)
build_images-->>Pipeline: result IMAGE_REF
end
Pipeline->>build_index: task build-image-index
build_index-->>Pipeline: results IMAGE_URL, IMAGE_DIGEST (multi-arch index)
Flow diagram for AUDITWHEEL_ARCH and PLATFORM auto-detection in builder imageflowchart LR
A[Start build_scripts.sh] --> B{AUDITWHEEL_ARCH set?}
B -- Yes --> C[Use AUDITWHEEL_ARCH]
B -- No --> D[Set AUDITWHEEL_ARCH = uname -m]
C --> E[Set AUDITWHEEL_PLAT = AUDITWHEEL_POLICY + '_' + AUDITWHEEL_ARCH]
D --> E
E --> F[export AUDITWHEEL_ARCH, AUDITWHEEL_PLAT]
subgraph Containerfile
G[ARG PLATFORM]
G --> H{PLATFORM provided?}
H -- Yes --> I[Use given PLATFORM in build]
H -- No --> J[Builder scripts derive arch from AUDITWHEEL_ARCH]
end
F --> J
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- The
build-platformsdefault inbuild-multiarch-pipeline.yamlislinux/x86_64but the PR wireslinux/amd64andlinux/arm64in the plumbing pipelines; consider aligning the default with the values actually used to avoid confusing or non-working defaults. - In the
build-imagesmatrix, thePLATFORMparam is set viavalue: - $(params.build-platforms); for a Tekton array parameter matrix this likely needs the[*]expansion ($(params.build-platforms[*])) so each platform is fanned out correctly rather than being passed as a single list element. - Clearing the
PLATFORMARG in theContainerfileand relying onAUDITWHEEL_ARCH=$(uname -m)is a behavior change; double-check any existing build paths or scripts that assumed the previous defaultx86_64ARG so they don't regress whenPLATFORMis now empty by default.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `build-platforms` default in `build-multiarch-pipeline.yaml` is `linux/x86_64` but the PR wires `linux/amd64` and `linux/arm64` in the plumbing pipelines; consider aligning the default with the values actually used to avoid confusing or non-working defaults.
- In the `build-images` matrix, the `PLATFORM` param is set via `value: - $(params.build-platforms)`; for a Tekton array parameter matrix this likely needs the `[*]` expansion (`$(params.build-platforms[*])`) so each platform is fanned out correctly rather than being passed as a single list element.
- Clearing the `PLATFORM` ARG in the `Containerfile` and relying on `AUDITWHEEL_ARCH=$(uname -m)` is a behavior change; double-check any existing build paths or scripts that assumed the previous default `x86_64` ARG so they don't regress when `PLATFORM` is now empty by default.
## Individual Comments
### Comment 1
<location path=".tekton/build-multiarch-pipeline.yaml" line_range="170-171" />
<code_context>
+ workspace: git-auth
+ - name: netrc
+ workspace: netrc
+ - name: build-images
+ matrix:
+ params:
+ - name: PLATFORM
</code_context>
<issue_to_address>
**issue (bug_risk):** Matrix parameter definition likely creates a nested array instead of a flat list of platforms.
In the `build-images` task, `PLATFORM` is defined as:
```yaml
matrix:
params:
- name: PLATFORM
value:
- $(params.build-platforms)
```
This makes `value` a single element that is itself an array, so the matrix will iterate once over the whole list instead of per platform. To get one iteration per platform, pass `$(params.build-platforms)` directly as the flat list (e.g. `value: $(params.build-platforms)`).
</issue_to_address>
### Comment 2
<location path=".tekton/build-multiarch-pipeline.yaml" line_range="77-81" />
<code_context>
+ default: 'false'
+ description: Enable cache proxy configuration
+ type: string
+ - default:
+ - linux/x86_64
+ description: List of platforms to build on. Available values depend on multi-platform-controller configuration.
+ name: build-platforms
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Default platform identifier (`linux/x86_64`) is inconsistent with the values used in plumbing configs (`linux/amd64`).
The `build-platforms` param defaults to `linux/x86_64`, while the plumbing YAML uses `linux/amd64` and `linux/arm64`. If the underlying task/controller expects the `linux/amd64` convention, this inconsistency could lead to unexpected behavior when the default is used. Please align the default with the values used elsewhere, or clearly document/normalize the accepted platform identifiers in this pipeline.
```suggestion
- default:
- linux/amd64
description: List of platforms to build on (for example: linux/amd64, linux/arm64). Available values depend on multi-platform-controller configuration.
name: build-platforms
type: array
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
The builder image Containerfile already handles multiple architectures via AUDITWHEEL_ARCH, but the CI pipeline was hardcoded to single-arch buildah-oci-ta. This adds a new build-multiarch pipeline that uses buildah-remote-oci-ta with Tekton matrix fan-out, and makes the Containerfile auto-detect the target architecture from uname -m when PLATFORM is not explicitly set. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
The rustup-init binary is architecture-specific, so each platform has a different sha256 hash. Split the single RUST_HASH into per-arch variables (RUST_HASH_x86_64, RUST_HASH_aarch64) and select the correct one based on AUDITWHEEL_ARCH. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Multi-arch builds require the image index to assemble per-platform images into a manifest list. The param defaulted to "false" (carried over from the single-arch pipeline), causing build-image-index to skip generation when multiple images were supplied. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
When PLATFORM is empty, ENV sets AUDITWHEEL_PLAT to "manylinux_2_28_" (trailing underscore). The := operator treated this as non-empty and skipped recomputation, so auditwheel repair failed with an invalid platform tag. Always recompute AUDITWHEEL_PLAT after resolving AUDITWHEEL_ARCH. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
jvulgan
force-pushed
the
multiarch-builder
branch
from
June 30, 2026 11:20
219703e to
e655cf0
Compare
jvulgan
changed the title
The auditwheel repair step used hardcoded *linux_x86_64.whl patterns, so native wheels on aarch64 (linux_aarch64) would be silently skipped. Derive the glob from AUDITWHEEL_ARCH / uname -m so the script works on any architecture. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The builder image Containerfile already handles multiple architectures via AUDITWHEEL_ARCH, but the CI pipeline was hardcoded to single-arch buildah-oci-ta. This adds a new build-multiarch pipeline that uses buildah-remote-oci-ta with Tekton matrix fan-out, and makes the Containerfile auto-detect the target architecture from uname -m when PLATFORM is not explicitly set.
Assisted-by: Claude Opus 4.6 noreply@anthropic.com
Summary by Sourcery
Introduce a multi-architecture container image build pipeline and wire CI to use it for builder images.
Enhancements:
Build: