fix(ci): wire missing UTILITIES_DIGEST into promote, add real nightly-status reporting

[djdomi]

CLD-1787880000

Transparency notice: This contribution was developed with AI assistance. Reviewed by the maintainer before merge.

Summary

Fixes two real, live bugs found while investigating why PR #1659 (a pure GitHub Actions dependency bump touching zero product code) still failed CI: promote has hard-failed on every run since utilities was added to the service matrix (missing digest wiring), and nightly-refresh.yml has been reporting false daily "success" for at least 9 days while the actual nightly images sat frozen.

Linked Issues

Refs #1095 | Refs #1556

What This Actually Changes

Bug 1 (the actual blocker): merge-manifests already computes utilities_digest per-service internally (its loop reads the shared CI_BUILD_SERVICES list, which includes utilities), but never re-exposed it in the job's own outputs: block, and promote never read it either. Every promote run has hard-failed with Missing or malformed candidate digest for utilities: '' since utilities (#1556) was added -- blocking channel promotion for all 8 services at once (promote's own gate is all-or-nothing), not just utilities. Live-confirmed: watchdog:nightly's org.opencontainers.image.revision label was frozen at a 2026-08-15 commit, 9 days stale, and its compose-expected compiled binary didn't even exist in that old image (a real, separate incompatibility this staleness caused).

Bug 2 (why nobody noticed): nightly-refresh.yml's only job dispatches build-push.yml -f channel=nightly via the GitHub API and reports its own success -- which only means the dispatch request was accepted, never that the dispatched run itself succeeded. It has reported green every single day for weeks while today's actual dispatched run (and, per Bug 1, likely most others) failed at promote.

What This PR Fixes / Adds

• build-push.yml: wires UTILITIES_DIGEST into merge-manifests's outputs: and promote's env/candidate_digests map, matching the existing 9-service pattern exactly.
• New .github/actions/nightly-status composite action + report.sh: files, updates, or closes one standing nightly-broken issue based on a nightly dispatch's real result. Adapted from wiki-mod/distcc-ng's already-proven nightly-status action (same design: one reused standing issue across consecutive failures, closed automatically on the next success, DRY_RUN support, Bug issue-type assignment, explicit project-board add via PROJECT_AUTOMATION_PAT since GITHUB_TOKEN-authored issues suppress the issues:opened webhook add-to-project.yml relies on).
• New report-nightly-status job in build-push.yml, gated to workflow_dispatch + channel == 'nightly' only (never fires on push/PR), needs: [build, build-arm64, merge-manifests, full-setup-validate, promote] with if: always() so it fires even when an earlier stage fails, computing the real outcome from those jobs' actual results.

What Changed In Code

• .github/workflows/build-push.yml: 3-line digest fix (Bug 1) + new report-nightly-status job (~45 lines, appended at file end).
• .github/actions/nightly-status/action.yml + report.sh: new composite action, ported from distcc-ng.

Why This Matters For Users / Operators

No operator-visible runtime change -- this is CI-internal. It does fix a real, currently-live gap: operators/maintainers evaluating nightly were getting a 9-day-stale, now-incompatible image, with the daily automation actively reporting "all fine" throughout. This also unblocks every future nightly/manual-dispatch promote attempt, not just today's.

Scope Boundaries

Deliberately does not: apply the same real-outcome-reporting fix to the daily latest-channel schedule: cron (a separate trigger, not investigated here) or to any push-triggered promote failure (those already have PR/commit-status visibility; the standing-issue mechanism specifically targets off-hours/unattended scheduled runs where nobody is watching in real time, matching distcc-ng's own scope). Does not touch promote's all-or-nothing gate itself (tracked separately as G16 on #1095) -- this PR only unblocks the one concrete cause that gate is currently tripping on.

Risk / Rollback / Follow-up

• Rollback: plain revert; the digest fix only adds previously-missing env wiring, and the new job/action are purely additive and gated to a narrow trigger.
• Follow-up (tracked on ci: rework of the entire CI infrastructure (rebuild/scan/promote pipeline, workflow consolidation, runner stabilization) #1095, not blocking): same real-outcome reporting for the latest-channel daily cron; G16 (promote's all-or-nothing gate).

Local Scope Evidence

.github/actions/nightly-status/action.yml | 41 ++++++++++
.github/actions/nightly-status/report.sh  | 152 +++++++++++++++++++++++++++++
.github/workflows/build-push.yml          | 48 +++++++++-
3 files changed, 239 insertions(+), 2 deletions(-)

Validation

All run for real inside the pinned build-tools container (AG-VAL-016) on a self-hosted runner (lancache-229), except the gh-dependent parts of report.sh (that CLI is not installed in the build-tools image -- these composite-action steps run natively on the runner, same as every other gh-calling composite action in this repo, e.g. pr-tracking-metadata-fetch-and-validate):

• bash scripts/tracked/check-file-headers.sh, check-review-chronology-comments.sh (AG-CODE-012 duplicate-reference guard), check-workflow-line-limit.sh -- all exit 0.
• actionlint .github/workflows/build-push.yml -- exit 0, resolves the new report-nightly-status job and its nightly-status action reference cleanly.
• shellcheck --severity=warning .github/actions/nightly-status/report.sh -- exit 0.
• report.sh run for real (not just syntax-checked) against the live repo with DRY_RUN=true and a real GH_TOKEN: both the "failure, no existing standing issue" path (correctly previews gh label create + gh issue create with the right body/title) and the "success, no existing standing issue" path (correctly no-ops) verified against real gh issue list API reads.
• Root-caused Bug 1 live against real history: merge-manifests's create-trusted-manifests/create-pr-staging-manifests steps both already emit utilities_digest via their generic per-service GITHUB_OUTPUT loop (confirmed by reading the actual printf line); only the job-level outputs: re-export and promote's own env/map were missing.

bash scripts/tracked/check-file-headers.sh
bash scripts/untracked/check-review-chronology-comments.sh
bash scripts/untracked/check-workflow-line-limit.sh
actionlint .github/workflows/build-push.yml
shellcheck --severity=warning .github/actions/nightly-status/report.sh
GH_TOKEN="$(gh auth token)" REPO=wiki-mod/lancache-ng OUTCOME=failure SCOPE="nightly channel promote (current_dev)" RUN_URL="https://example.invalid/run/1" FAILED_JOBS="promote (failure)" DRY_RUN=true bash .github/actions/nightly-status/report.sh

Type of change

• Bug fix
• New feature

Changelog

Fixes promote hard-failing on every run since utilities was added to the service matrix (missing UTILITIES_DIGEST wiring), which had frozen the nightly channel 9 days stale. Adds a distcc-ng-style standing-issue mechanism so a future nightly-dispatch failure is actually visible, instead of nightly-refresh.yml reporting a false daily "success". No operator-visible runtime change.