Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 45 additions & 28 deletions .github/workflows/build-images.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,17 @@ jobs:
needs: validate-config
permissions:
contents: read
id-token: write
strategy:
fail-fast: false
matrix:
include:
- target_platform: linux/arm64
target_arch: arm64
runner: ubuntu-24.04-arm
yq_arch: arm64
- target_platform: linux/amd64
target_arch: amd64
runner: ubuntu-24.04
yq_arch: amd64

Expand Down Expand Up @@ -76,45 +79,59 @@ jobs:
FIREWORK_GITHUB_TOKEN: ${{ secrets.FIREWORK_GITHUB_TOKEN }}
run: make build

- name: Resolve upload buckets
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
env:
S3_IMAGES_BUCKET: ${{ vars.S3_IMAGES_BUCKET }}
S3_IMAGES_BUCKET_ARM64: ${{ vars.S3_IMAGES_BUCKET_ARM64 }}
S3_IMAGES_BUCKET_AMD64: ${{ vars.S3_IMAGES_BUCKET_AMD64 }}
GCS_IMAGES_BUCKET: ${{ vars.GCS_IMAGES_BUCKET }}
GCS_IMAGES_BUCKET_ARM64: ${{ vars.GCS_IMAGES_BUCKET_ARM64 }}
GCS_IMAGES_BUCKET_AMD64: ${{ vars.GCS_IMAGES_BUCKET_AMD64 }}
run: |
case "${{ matrix.target_arch }}" in
arm64)
echo "S3_IMAGES_BUCKET=${S3_IMAGES_BUCKET_ARM64:-${S3_IMAGES_BUCKET:-}}" >> "$GITHUB_ENV"
echo "GCS_IMAGES_BUCKET=${GCS_IMAGES_BUCKET_ARM64:-}" >> "$GITHUB_ENV"
;;
amd64)
echo "S3_IMAGES_BUCKET=${S3_IMAGES_BUCKET_AMD64:-}" >> "$GITHUB_ENV"
echo "GCS_IMAGES_BUCKET=${GCS_IMAGES_BUCKET_AMD64:-${GCS_IMAGES_BUCKET:-}}" >> "$GITHUB_ENV"
;;
esac

- name: Upload images to S3
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && matrix.target_platform == 'linux/arm64'
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: ${{ vars.AWS_REGION }}
S3_IMAGES_BUCKET: ${{ vars.S3_IMAGES_BUCKET }}
run: make push

- name: Save amd64 rootfs artifacts
if: matrix.target_platform == 'linux/amd64'
uses: actions/upload-artifact@v4
with:
name: rootfs-amd64
path: "*-rootfs.ext4"
if-no-files-found: error

upload-gcs:
needs: build
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-24.04
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v4

- uses: actions/download-artifact@v4
with:
name: rootfs-amd64
run: |
if [ -z "${S3_IMAGES_BUCKET:-}" ]; then
echo "Skipping S3 upload for ${{ matrix.target_platform }}; no bucket configured"
exit 0
fi
make push-s3

- name: Authenticate to GCP
if: >-
github.event_name == 'push' &&
github.ref == 'refs/heads/main' &&
(
(matrix.target_arch == 'arm64' && vars.GCS_IMAGES_BUCKET_ARM64 != '') ||
(matrix.target_arch == 'amd64' && (vars.GCS_IMAGES_BUCKET_AMD64 != '' || vars.GCS_IMAGES_BUCKET != ''))
)
uses: google-github-actions/auth@v3
with:
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
project_id: ${{ secrets.GCP_PROJECT_ID }}

- name: Upload images to GCS
env:
GCS_IMAGES_BUCKET: ${{ vars.GCS_IMAGES_BUCKET }}
run: make push-gcs
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: |
if [ -z "${GCS_IMAGES_BUCKET:-}" ]; then
echo "Skipping GCS upload for ${{ matrix.target_platform }}; no bucket configured"
exit 0
fi
make push-gcs
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Project

This is the example GitOps input repo for Firework. It defines tenant service YAML and config overlays used to build Firecracker-ready rootfs images and publish them (ARM64 to S3, amd64 to GCS). Public routing is provider-neutral via `metadata.subdomain`; there is no provider-specific runtime config tree.
This is the example GitOps input repo for Firework. It defines tenant service YAML and config overlays used to build Firecracker-ready rootfs images and publish both ARM64 and amd64 artifacts to S3 and GCS via architecture-specific buckets. Public routing is provider-neutral via `metadata.subdomain`; there is no provider-specific runtime config tree.

## Layout

Expand All @@ -12,7 +12,7 @@ This is the example GitOps input repo for Firework. It defines tenant service YA
- `configs/<service>/` and `configs/<tenant>-<service>/`: rootfs overlays; tenant-specific overlays take precedence.
- `scripts/build-images.sh`: resolves `fc-init` and builds all tenant rootfs images.
- `scripts/docker-to-rootfs.sh`: converts Docker images into ext4 rootfs images.
- `scripts/push-images.sh`: uploads generated rootfs images to S3.
- `scripts/push-images.sh`: uploads generated rootfs images to the selected object store bucket.
- `scripts/fc-init/`: fallback bundled `fc-init` source for CI.

## Conventions
Expand Down
15 changes: 13 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> This is an example deployment intended for demonstration and learning purposes only. It is not hardened, audited, etc.

Example GitOps repository for [Firework](https://github.com/artemnikitin/firework), focused on building Firecracker-ready rootfs images and publishing ARM64 images to S3 and amd64 images to GCS.
Example GitOps repository for [Firework](https://github.com/artemnikitin/firework), focused on building Firecracker-ready rootfs images and publishing both ARM64 and amd64 images to S3 and GCS via architecture-specific buckets.

## Related Repositories

Expand All @@ -24,11 +24,22 @@ The `build-images` workflow runs on every pull request and every push to `main`.
It builds the tenant rootfs images twice, once for `linux/arm64` and once for
`linux/amd64`.

On pushes to `main`, each matrix build job publishes its architecture to the
configured S3 bucket and, when configured, authenticates to GCP in the same job
and uploads that architecture to its GCS bucket. Keep the buckets
architecture-specific: the generated `*-rootfs.ext4` filenames are the same
across architectures, so sharing one bucket would cause overwrites.

Legacy variables keep their original meanings: `S3_IMAGES_BUCKET` is the arm64
S3 bucket and `GCS_IMAGES_BUCKET` is the amd64 GCS bucket. Configure
`S3_IMAGES_BUCKET_AMD64` and `GCS_IMAGES_BUCKET_ARM64` to enable the extra
cross-backend uploads.

### CI config validation

Before building images, the `validate-config` CI job runs Firework's
`cmd/configcheck --require-remote-routing` against this repository's root, using
the exact enricher of a pinned core version.
the exact enricher of a pinned core version.

Local platform-specific builds:

Expand Down
Loading