Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 41 additions & 13 deletions .github/workflows/docker-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@ name: Docker Release
# render empty (and are skipped, stevedore >= v0.0.5) outside CI so local
# builds are unaffected.
#
# Remaining gap vs the matrix implementation: no Dispatch publish notification
# yet (blairham/stevedore#6) — the EventBridge ECR-push backstop still covers
# deploy correlation. Callers should keep `no-push: true` (the default) or stay
# on the pre-stevedore pin for real publishes until that is wired.
# Dispatch publish notification: stevedore's release summary (a `summary` step
# output since v0.0.6, blairham/stevedore#6) drives one notify-dispatch call
# per PUSHED image after the release — behavior parity with the matrix
# implementation. The EventBridge ECR-push backstop still covers a missed call.

on:
workflow_call:
Expand All @@ -50,10 +50,15 @@ on:
type: string
default: ""
no-push:
description: "Build without pushing (validation only). Default true until the Dispatch notification gap closes."
description: "Build without pushing (validation only)."
required: false
type: boolean
default: true
default: false
only:
description: "Comma-separated image ids to release (stevedore --only); empty releases per change detection. Feeds workflow_dispatch service pickers."
required: false
type: string
default: ""
parallel:
description: "Build up to N images concurrently."
required: false
Expand All @@ -63,11 +68,10 @@ on:
description: "stevedore release to use."
required: false
type: string
# v0.0.5 = plan/matrix mode (v0.0.4) + skip-empty cache entries
# (blairham/stevedore#8), which keeps LOCAL builds of cache-wired
# configs working; in CI the cache env below renders the entries
# non-empty either way.
default: "v0.0.5"
# v0.0.6 = skip-empty cache entries (v0.0.5, blairham/stevedore#8)
# + release summary with pushed/reason fields as a step output
# (blairham/stevedore#9) — the Notify Dispatch step consumes it.
default: "v0.0.6"

env:
AWS_REGION: us-east-1
Expand Down Expand Up @@ -107,11 +111,13 @@ jobs:
NO_PUSH: ${{ inputs.no-push }}
PARALLEL: ${{ inputs.parallel }}
CHANGED_SINCE: ${{ inputs.changed-since }}
ONLY: ${{ inputs.only }}
run: |
set -euo pipefail
args="--parallel ${PARALLEL}"
[ "${NO_PUSH}" = "true" ] && args="${args} --no-push"
[ -n "${CHANGED_SINCE}" ] && args="${args} --changed-since ${CHANGED_SINCE}"
[ -n "${ONLY}" ] && args="${args} --only ${ONLY}"
echo "value=${args}" >> "$GITHUB_OUTPUT"

# buildx's type=gha cache backend reads ACTIONS_CACHE_URL /
Expand All @@ -122,9 +128,10 @@ jobs:
uses: crazy-max/ghaction-github-runtime@v3

- name: Stevedore
id: stevedore
# uses: refs cannot be expressions — pin the action; the `version`
# input below selects the binary that actually runs.
uses: blairham/stevedore@v0.0.5
uses: blairham/stevedore@v0.0.6
env:
# ECR host, e.g. 123456789.dkr.ecr.us-east-1.amazonaws.com — the
# .stevedore.yaml repositories template this as {{ .Env.REGISTRY }}.
Expand All @@ -145,6 +152,28 @@ jobs:
install-syft: "false"
install-grype: "false"

# One publish notification per PUSHED image, from stevedore's release
# summary (platform-gitops#1201). pushed=false (a no-push validation
# build) and skipped entries notify nothing. Never fails the release —
# the EventBridge ECR-push backstop covers a missed call.
- name: Build Dispatch batch
id: dispatch-batch
if: ${{ steps.stevedore.outputs.summary != '' }}
env:
SUMMARY: ${{ steps.stevedore.outputs.summary }}
run: |
set -euo pipefail
batch=$(jq -c '[(.images // [])[] | select(.pushed and (.skipped | not)) | {kind: "image", service: .id, version: .version}]' <<<"${SUMMARY}")
echo "value=${batch}" >> "$GITHUB_OUTPUT"
echo "Pushed images to notify: $(jq -r 'length' <<<"${batch}")"

- name: Notify Dispatch
if: ${{ steps.dispatch-batch.outputs.value && steps.dispatch-batch.outputs.value != '[]' }}
uses: pinpredict/.github/actions/notify-dispatch@main
with:
batch: ${{ steps.dispatch-batch.outputs.value }}
secret: ${{ secrets.CI_WEBHOOK_SECRET }}

# ── How a repo calls it ──────────────────────────────────────────────────────
#
# jobs:
Expand All @@ -153,5 +182,4 @@ jobs:
# uses: pinpredict/.github/.github/workflows/docker-release.yml@main
# with:
# changed-since: ${{ github.event.before }} # or the PR base SHA
# no-push: true # flip to false to publish
# secrets: inherit
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Both workflows use a three-shot retry pattern (try / sleep 30 / retry / sleep 60

- **ECR is the version record** (platform-gitops#1201): both release workflows resolve the next version as the highest strict-`X.Y.Z` tag in the service's ECR repo (image or `charts/<name>` OCI) plus one, then probe-and-bump past any existing candidate (ECR tags are immutable). Git tags are never read for versioning.
- **Release marker refs**: `refs/releases/image/<name>` and `refs/releases/chart/<name>` — one mutable ref per service, force-advanced to the released SHA on every successful push. They are the change-detection baseline for `discover-services` and chart-release's prepare job (legacy `image|chart/<name>/X.Y.Z` tag is the fallback until a service releases once with the marker in place). Not tags, so they don't feed Kargo's tag enumeration or the GitHub `create` webhook. Readers must fetch them explicitly (`+refs/releases/*:refs/releases/*`).
- **Dispatch publish notification**: after each successful push, `chart-release.yml` and the pinned `docker-release.yml@pre-stevedore` call `actions/notify-dispatch` — the stevedore `docker-release.yml` on main does not yet (blairham/stevedore#6; its `no-push` defaults `true` until that closes). The notification is a signed POST (service, version, full SHA, run URL) through the public webhook-forwarder `/dispatch/ci` route, HMAC'd with the org `CI_WEBHOOK_SECRET` (reaches reusable workflows via `secrets: inherit`). Warn-only on failure: the EventBridge ECR-push backstop covers a missed call.
- **Dispatch publish notification**: after each successful push, both release workflows call `actions/notify-dispatch` — chart-release per chart, the stevedore docker-release once with a `batch` built from the release summary (only `pushed: true` images; blairham/stevedore#6). The notification is a signed POST (service, version, full SHA, run URL) through the public webhook-forwarder `/dispatch/ci` route, HMAC'd with the org `CI_WEBHOOK_SECRET` (reaches reusable workflows via `secrets: inherit`). Warn-only on failure: the EventBridge ECR-push backstop covers a missed call.
- **Chart sha alias**: chart-release aliases the pushed OCI chart as `X.Y.Z-<sha7>` (via `aws ecr put-image` on the same manifest — allowed under tag immutability) so the ECR-push backstop can recover commit provenance for charts, mirroring the image tag pair. Semver-prerelease form, so Kargo chart Warehouses ignore it.
- **No `image/*` / `chart/*` git tags and no GitHub Releases are minted** (platform-gitops#1201). The historical tags remain in service repos as the *frozen* fallback change-detection baseline — do not prune them until every service has released once with marker refs (the readers fall back to them). Do not reintroduce tag minting: it spams Kargo's git tag enumeration and the GitHub `create` webhook.
- Per-service config tag (Kargo freight for `<svc>-config` Warehouse): `vX.Y.Z+<svc>` — semver **build metadata** form (`+`), not pre-release (`-`). The `+` form passes `semverConstraint: ">=0.0.0"` cleanly; the pre-release form would require `>=0.0.0-0`. **This tag family stays** — it is genuine Kargo freight.
Expand Down
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Why `.github` and not a dedicated `github-actions` repo: `.github` is *the* GitH

| File | Purpose |
|---|---|
| `docker-release.yml` | Image build + release via [stevedore](https://github.com/blairham/stevedore), driven by the caller repo's `.stevedore.yaml` — **no `matrix` input**. Single job by design (pinpredict/.github#39): one runner + one BuildKit, so a build-once Dockerfile compiles once for every image. Version = highest `X.Y.Z` tag in the ECR repo + 1 (ECR is the version record — platform-gitops#1201, resolved by stevedore); advances the `refs/releases/image/<id>` marker refs. No git tags or GitHub Releases. Exports `STEVEDORE_CACHE_FROM/TO` (one `type=gha` scope) for configs that opt into layer caching. `no-push` defaults `true` until the Dispatch notification gap (blairham/stevedore#6) closes. Existing callers are pinned to `@pre-stevedore` (the frozen matrix implementation, which also carried the `private-modules` BuildKit-secret input) — see the tagging exception below. |
| `docker-release.yml` | Image build + release via [stevedore](https://github.com/blairham/stevedore), driven by the caller repo's `.stevedore.yaml` — **no `matrix` input**. Single job by design (pinpredict/.github#39): one runner + one BuildKit, so a build-once Dockerfile compiles once for every image. Version = highest `X.Y.Z` tag in the ECR repo + 1 (ECR is the version record — platform-gitops#1201, resolved by stevedore); advances the `refs/releases/image/<id>` marker refs. No git tags or GitHub Releases. Exports `STEVEDORE_CACHE_FROM/TO` (one `type=gha` scope) for configs that opt into layer caching. Notifies Dispatch once per **pushed** image from stevedore's release summary (`no-push` builds notify nothing). Existing callers are pinned to `@pre-stevedore` (the frozen matrix implementation, which also carried the `private-modules` BuildKit-secret input) — see the tagging exception below. |
| `chart-release.yml` | Auto-discovers `charts/*/`, skips charts unchanged since their `refs/releases/chart/<name>` marker ref, resolves the next version from the ECR OCI repo, packages, pushes (+ `X.Y.Z-<sha7>` provenance alias), advances the marker, notifies Dispatch. No git tags or GitHub Releases. No caller inputs. |
| `tag-config.yml` | Tags merges to main that touch `.platform/services/<svc>.yaml` with `vX.Y.Z+<svc>` (per-service Kargo `<svc>-config` Warehouse freight), then dispatches `service-config-tag` to platform-gitops so missing pointer files get seeded. |
| `actionlint.yml` | Lints GitHub Actions workflow YAML with [`actionlint`](https://github.com/rhysd/actionlint) at a pinned version. Self-runs on this repo when PRs/pushes touch `.github/workflows/**` or `actions/**/action.yml`; callers reuse it via `uses: pinpredict/.github/.github/workflows/actionlint.yml@main`. |
Expand All @@ -19,7 +19,7 @@ Why `.github` and not a dedicated `github-actions` repo: `.github` is *the* GitH

| Action | Purpose |
|---|---|
| `notify-dispatch` | POSTs a signed publish notification (service, version, SHA, run URL) to Dispatch's public `/dispatch/ci` route after a successful ECR push. Called by both release workflows; warn-only on failure (the EventBridge ECR-push backstop covers a missed call). |
| `notify-dispatch` | POSTs a signed publish notification (service, version, SHA, run URL) to Dispatch's public `/dispatch/ci` route after a successful ECR push — one call per artifact, or a `batch` JSON array (used by docker-release's stevedore summary). Called by both release workflows; warn-only on failure (the EventBridge ECR-push backstop covers a missed call). |
| `configure-aws-with-retry` | Wraps `aws-actions/configure-aws-credentials` in a three-shot retry (try / sleep 30 / retry / sleep 60 / retry) to survive a first-colocation race with Crossplane minting the per-service `xp-<svc>-gha-push` role (pinpredict/trading#616). Inputs: `role-to-assume` (required), `aws-region` (default `us-east-1`). Shared by both release workflows. |
| `discover-services` | Reads `.platform/services/*.yaml` and emits a docker matrix of services whose docker-relevant files changed since their last release (baseline = `refs/releases/image/<name>` marker ref; legacy `image/<name>/*` tag fallback). Also emits `charts_changed`. |
| `validate-platform-service` | Pre-merge static + render check for added/modified `.platform/services/*.yaml`. Renders each via `charts/service-template` for every env in `environments[]` with all `renderXxx` flags forced on; verifies `repositories.chart` resolves to a real `charts/<x>/Chart.yaml`. Closes the gap from platform-gitops#544 — every dis-opticodds-props-streamer failure mode would have failed CI here. |
Expand Down Expand Up @@ -141,7 +141,6 @@ jobs:
uses: pinpredict/.github/.github/workflows/docker-release.yml@main
with:
changed-since: ${{ github.event.before }}
no-push: true # flip to false once the Dispatch notification lands
secrets: inherit

chart-release:
Expand Down
60 changes: 44 additions & 16 deletions actions/notify-dispatch/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,25 @@ description: >-

inputs:
kind:
description: '"image" or "chart"'
required: true
description: '"image" or "chart" (single mode; ignored when batch is set)'
required: false
default: ""
service:
description: Service / chart name (the tag's middle segment)
required: true
description: Service / chart name (single mode; ignored when batch is set)
required: false
default: ""
version:
description: Released version (X.Y.Z)
required: true
description: Released version X.Y.Z (single mode; ignored when batch is set)
required: false
default: ""
batch:
description: >-
JSON array of {kind, service, version} objects to notify in one call —
for releases that publish several artifacts at once (e.g. the stevedore
docker-release summary). When set, the kind/service/version inputs are
ignored. Empty array is a no-op.
required: false
default: ""
secret:
description: >-
HMAC secret (org Actions secret CI_WEBHOOK_SECRET, reaches reusable
Expand All @@ -37,6 +48,7 @@ runs:
KIND: ${{ inputs.kind }}
SERVICE: ${{ inputs.service }}
VERSION: ${{ inputs.version }}
BATCH: ${{ inputs.batch }}
SECRET: ${{ inputs.secret }}
URL: ${{ inputs.url }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
Expand All @@ -47,17 +59,33 @@ runs:
exit 0
fi

body=$(printf '{"repo":"%s","kind":"%s","service":"%s","version":"%s","sha":"%s","run_url":"%s"}' \
"${GITHUB_REPOSITORY}" "${KIND}" "${SERVICE}" "${VERSION}" "${GITHUB_SHA}" "${RUN_URL}")
sig=$(printf '%s' "$body" | openssl dgst -sha256 -hmac "${SECRET}" | awk '{print $NF}')
notify() {
local kind=$1 service=$2 version=$3
local body sig status
body=$(printf '{"repo":"%s","kind":"%s","service":"%s","version":"%s","sha":"%s","run_url":"%s"}' \
"${GITHUB_REPOSITORY}" "${kind}" "${service}" "${version}" "${GITHUB_SHA}" "${RUN_URL}")
sig=$(printf '%s' "$body" | openssl dgst -sha256 -hmac "${SECRET}" | awk '{print $NF}')

status=$(curl -sS -o /tmp/dispatch-resp.json -w '%{http_code}' -X POST "${URL}" \
-H "Content-Type: application/json" \
-H "X-Dispatch-Signature: ${sig}" \
--data "$body" || echo "000")

status=$(curl -sS -o /tmp/dispatch-resp.json -w '%{http_code}' -X POST "${URL}" \
-H "Content-Type: application/json" \
-H "X-Dispatch-Signature: ${sig}" \
--data "$body" || echo "000")
if [ "$status" -ge 200 ] && [ "$status" -lt 300 ]; then
echo "Dispatch notified: ${kind} ${service} ${version}"
else
echo "::warning::Dispatch notification returned ${status}: $(cat /tmp/dispatch-resp.json 2>/dev/null || true) — ECR-push backstop covers this release"
fi
}

if [ "$status" -ge 200 ] && [ "$status" -lt 300 ]; then
echo "Dispatch notified: ${KIND} ${SERVICE} ${VERSION}"
if [ -n "${BATCH}" ]; then
count=$(jq -r 'length' <<<"${BATCH}")
echo "Notifying Dispatch for ${count} artifact(s)"
jq -c '.[]' <<<"${BATCH}" | while IFS= read -r entry; do
notify "$(jq -r .kind <<<"$entry")" "$(jq -r .service <<<"$entry")" "$(jq -r .version <<<"$entry")"
done
elif [ -n "${SERVICE}" ]; then
notify "${KIND}" "${SERVICE}" "${VERSION}"
else
echo "::warning::Dispatch notification returned ${status}: $(cat /tmp/dispatch-resp.json 2>/dev/null || true) — ECR-push backstop covers this release"
echo "Nothing to notify (no batch, no service)"
fi