Skip to content

NCBC-4283: Let FIT runs test a branch's performer image - #163

Merged
davidkelly merged 1 commit into
masterfrom
dk/ncbc-4283
Aug 12, 2026
Merged

NCBC-4283: Let FIT runs test a branch's performer image#163
davidkelly merged 1 commit into
masterfrom
dk/ncbc-4283

Conversation

@davidkelly

@davidkelly davidkelly commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

NCBC-4283

Problem

fit-testing-dotnet.yml hardcoded performer: 'dotnet-fit-performer:main', so every FIT run tested master's SDK.

The SDK under test is baked into the performer image (built by publish-fit-performer.yml from a chosen ref), not taken from the calling workflow's checkout. Selecting a branch in the Actions UI therefore changed which YAML ran but not which code was tested — the UI implies otherwise, so a PR could pass FIT without its changes ever being exercised. The workaround was a throwaway commit editing the hardcoded tag on the branch.

Change

Adds an optional performer_tag dispatch input, defaulting to the branch the workflow is run from.

Deriving that tag needs shell — dk/ncbc-4283 is not a valid Docker tag — and a uses: job cannot also have steps, so a small performer-tag job resolves it and passes it via job outputs. It mirrors the two rules the publish side already applies:

  • sdk-docker-build-action renames mastermain
  • docker/metadata-action collapses runs of [^a-zA-Z0-9._-] to -

Verified against the tags actually published to ghcr.io today:

GITHUB_REF_NAME performer tag
master (nightly) main
dk/ncbc-4300 dk-ncbc-4300
dk/pe-ramp-bootstrap dk-pe-ramp-bootstrap
NCBC-4271 NCBC-4271
release/3.9.x release-3.9.x

The resolved image is echoed into the job log, so a run states what it tested.

The input is passed via env: rather than interpolated into the script body, since ${{ }} is substituted before bash parses it. The tag job gets permissions: {} — it has no need for the OIDC token the fit job requires.

Nightly is unchanged

schedule passes no inputs and GITHUB_REF_NAME is master, which resolves to main exactly as before.

Usage

Build the image first, or the run fails to pull it:

  1. Publish FIT Performer Image with ref=<your-branch>
  2. Run FIT-CLI from <your-branch> — no performer_tag needed

Pass performer_tag: main to test master's SDK from a branch.

🤖 Generated with Claude Code

Motivation
----------
The FIT caller hardcoded `performer: 'dotnet-fit-performer:main'`, so
every run tested master's SDK. The SDK under test is baked into the
performer image rather than taken from the workflow's own checkout, so
selecting a branch in the Actions UI changed which YAML ran but not
which code was tested — a PR could pass FIT without its changes ever
being exercised. Testing a branch meant a throwaway commit editing the
hardcoded tag.

Changes
-------
Add an optional `performer_tag` dispatch input which defaults to the
branch the workflow is run from, so a branch run tests that branch.

Deriving the tag needs shell (branch names like dk/ncbc-4283 are not
valid Docker tags), and a `uses:` job cannot also have `steps`, so a
small `performer-tag` job resolves it and hands it over via outputs. It
mirrors the two rules the publish side applies: sdk-docker-build-action
renames master to main, and docker/metadata-action collapses runs of
[^a-zA-Z0-9._-] to '-'. The resolved image is echoed into the log.

The nightly is unchanged: `schedule` passes no inputs and GITHUB_REF_NAME
is master, which resolves to main as before.

Note the performer image must already exist — build it first via
"Publish FIT Performer Image" with ref=<your-branch>.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the FIT test workflow so manual runs can test the correct SDK code by selecting a matching dotnet-fit-performer image tag (instead of always using main).

Changes:

  • Adds a performer_tag workflow_dispatch input to select the performer image tag (defaults to the branch-derived tag).
  • Introduces a performer-tag job which normalizes GITHUB_REF_NAME into a valid Docker tag and exposes it as a job output.
  • Updates the fit reusable-workflow invocation to use the resolved tag for its performer input.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

preset: ${{ vars.FIT_CLI_PRESET || inputs.preset || 'op-multi-lite' }}
collect_extra_diagnostics: ${{ inputs.collect_extra_diagnostics == '' && true || inputs.collect_extra_diagnostics }}
performer: 'dotnet-fit-performer:main'
performer: dotnet-fit-performer:${{ needs.performer-tag.outputs.tag }}
@emilienbev

emilienbev commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

We had a "Run FIT-CLI (Shared)" (fit-testing-dotnet.yml called it, passing the hardcoded dotnet-fit-performer:main) which was the base workflow where we could pass a custom performer image, not sure what happened to that one.
Edit: Ah found it, it was removed with #157 .

LGTM 👍 after Copilot's comment on the needs expression if it's valid.

@davidkelly

Copy link
Copy Markdown
Contributor Author

Thanks both.

@Copilot's needs point is a false positive — hyphens are fine in needs.<job-id> dot notation. Two disproofs:

  1. This exact commit (3fb3a7864) already ran green: run 31523670346, where the fit job received performer: dotnet-fit-performer:dk-ncbc-4283 — which can only come from needs.performer-tag.outputs.tag. An invalid expression fails evaluation rather than resolving correctly.
  2. apidocs.yml and release.yml already use needs.validate-inputs.outputs.* in production.

@emilienbev re the old fit-testing-shared.yml: correct that #157 removed it, but it wouldn't have helped — it took a performer_image_name input, and fit-testing-dotnet.yml passed the hardcoded 'dotnet-fit-performer:main' into it. The shared fit-cli workflow likewise exposes performer as a required input today; the gap was always that the caller never surfaced it.

Also smoke-tested end to end: built dotnet-fit-performer:dk-ncbc-4283 from this branch, ran op-onprem-func-sanity from the branch with no performer_tag, and confirmed via the ghcr config blob that the image the AWS host pulled and ran carries org.opencontainers.image.revision=3fb3a7864 (vs :main = 930cad057). Result: PASS.

@davidkelly
davidkelly merged commit 52bcf76 into master Aug 12, 2026
15 of 17 checks passed
@davidkelly
davidkelly deleted the dk/ncbc-4283 branch August 12, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants