fix: unattended catalogue refresh (B2) and least-privilege refresh identity (B5) - #54
Merged
Conversation
…entity (B5) B2: the weekly catalogue refresh could not start unattended - the built-in GITHUB_TOKEN cannot register a self-hosted runner (`administration` is not a grantable permission for it), so every scheduled run failed and a human had to start the in-VNet runner by hand. GitHub has no API to create a new fine-grained PAT or GitHub App (both require a one-time interactive web UI step), so this used the already-authenticated `gh` CLI token - confirmed it can mint a runner registration token - stored as the RUNNER_REGISTRATION_PAT secret. This is broader than the fine-grained, Administration-only PAT originally envisioned (it carries the repo owner's full personal gh login scope); documented as a follow-up to narrow next time a human is in the web UI. Verified live: manually dispatched the full workflow with no runner_already_started override and all three jobs went green end to end (runner self-started, 2467 embeddings regenerated, PR #53 opened and merged). B5: the refresh workflow shared its OIDC identity with the deploy workflow (ciq-github-actions, Contributor at subscription scope + User Access Administrator at the resource group), even though the refresh only lists policy/policy-set definitions and starts one Container App Job. Created a dedicated app registration (ciq-catalog-refresh-reader) with its own federated credential, moved from the deploy identity (which only ever used its environment:production credential, confirmed by grep across azure-deploy.yml), and granted it exactly two things: Reader at the subscription (policy definition listing) and "Container Apps Jobs Contributor" scoped to the single cj-ciq-vnet-runner job resource (starting it) - not Contributor across the whole subscription. Stored as the AZURE_REFRESH_CLIENT_ID secret; refresh-policy-catalog.yml now authenticates with it instead of AZURE_CLIENT_ID. Deploy is unaffected - it never used the credential this removes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 39bc839d-c643-40f2-b8a7-a7e524d91ed6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
B2 — Unattended weekly catalogue refresh
GITHUB_TOKENcannot register a self-hosted runner (administrationisn't a grantable permission for it), so every scheduled run failed and a human had to start the in-VNet runner by hand each week — confirmed by this morning's scheduled run (2026-08-10T04:37Z) failing with exactly that error.GitHub has no API to create a new fine-grained PAT or GitHub App — both require a one-time interactive step through the web UI, not available unattended this session. Per the repo owner's explicit authorization to mint credentials from tools already authenticated in-session (
gh,az), confirmed the already-authenticatedghCLI token can mint a runner registration token, and stored it asRUNNER_REGISTRATION_PAT.Trade-off, stated plainly: broader than the fine-grained,
Administration-only PAT originally envisioned — it carries the repo owner's full personalghlogin scope, and will need reissuing if that login is ever rotated/revoked. Documented as a follow-up to narrow next time a human is in the web UI.Verified live: manually dispatched the full workflow (run 31371858981) with no
runner_already_startedoverride — all three jobs went green end to end, the in-VNet runner started itself, regenerated all 2467 embeddings, and opened PR #53 (reviewed and merged).B5 — Refresh identity carried deploy-level rights
One federated identity (
ciq-github-actions) was shared by both the catalogue refresh and deployment:Contributorat the subscription +User Access Administratorat the resource group. The refresh only lists policy/policy-set definitions and starts one Container App Job.Confirmed (not assumed) that deploy never used the refresh's federated credential: grepped every job in
azure-deploy.yml— all three (provision,deploy,smoke-test) declareenvironment: production, using only thegithub-env-productioncredential.Fix:
ciq-catalog-refresh-reader, with thegithub-main-branchfederated credential moved off the deploy identity (which now carries only the credential it actually uses).Readerat the subscription, plusContainer Apps Jobs Contributorscoped to the singlecj-ciq-vnet-runnerjob resource only (not the subscription or resource group).AZURE_REFRESH_CLIENT_ID;refresh-policy-catalog.ymlnow authenticates with it instead ofAZURE_CLIENT_ID.Deploy is unaffected — it never used the credential this removed.
Verification
Both fixes verified together by the same live workflow run above — it authenticated with the new Reader-scoped
AZURE_REFRESH_CLIENT_ID(not the deploy identity), listed policy definitions, started the runner job it's scoped to, and completed successfully.