diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1a4f679..5e6b7ff 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -223,3 +223,9 @@ jobs: echo "::endgroup::" done < <(find infra -type f \( -name '*.bicep' -o -name '*.bicepparam' \) -print0 | sort -z) exit "$status" + + # The Entra setup scripts sit next to the Bicep; shellcheck ships on the runner image. + - name: Shellcheck infra/ scripts + run: | + shellcheck --version + find infra -type f -name '*.sh' -print0 | xargs -0 --no-run-if-empty shellcheck diff --git a/CHANGELOG.md b/CHANGELOG.md index a10ddf6..8ba4130 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,3 +48,5 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Dependabot updates for GitHub Actions and Go modules. - Bicep for the metrics store (`infra/main.bicep`, `infra/main.bicepparam`): a serverless Cosmos DB account with key-based auth disabled, database `AIMetrics` and container `Events` partitioned on `/featureName`. - CI job that builds and lints every Bicep file under `infra/` with a version- and checksum-pinned Bicep CLI; linter levels live in `infra/bicepconfig.json`. +- Full metrics deployment in `infra/`: a container registry (no admin user), a user-assigned identity with `AcrPull` and the Cosmos DB Built-in Data Contributor role, a Container Apps environment with Log Analytics, and the Container App with `AUTH_MODE`, `ENTRA_TENANT_ID` and `ENTRA_AUDIENCE` set. Container Apps sign-in with Entra guards everything except `/events` and `/healthz`. Steps, outputs and teardown are in `infra/README.md`. +- `infra/setup-entra.sh` registers the metrics API (with the `Events.Write` app role) and the dashboard sign-in app (assignment required), and `infra/grant-events-write.sh` gives a pipeline's service connection `Events.Write`. CI runs shellcheck on both. diff --git a/README.md b/README.md index 3664425..8a32f81 100644 --- a/README.md +++ b/README.md @@ -196,8 +196,8 @@ Events may carry an optional `tenantId` (default `default`). Each tenant's event Missing or invalid credentials get `401`. [`metrics/scripts/send-event-curl.sh`](metrics/scripts/send-event-curl.sh) sends a test event with either credential. - -**Deployment (in progress).** [`infra/`](infra) currently holds Bicep for the Cosmos DB store (serverless, key auth disabled). Container Registry, Container Apps, the Entra setup script and step-by-step deploy docs will follow. + +**Deployment.** [`infra/README.md`](infra/README.md) deploys the service to Azure in one resource group: Cosmos DB with key auth disabled, a container registry, Container Apps running as a managed identity, and Entra sign-in in front of the dashboard. Two scripts handle Entra: [`infra/setup-entra.sh`](infra/setup-entra.sh) registers the API (with the `Events.Write` role) and the dashboard sign-in app, and [`infra/grant-events-write.sh`](infra/grant-events-write.sh) gives your pipeline's service connection the role. ## Repository layout @@ -206,7 +206,7 @@ Missing or invalid credentials get `401`. [`metrics/scripts/send-event-curl.sh`] | `ai-code-review/` | The review engine: PowerShell module, stage template, base prompt and Pester tests. | | `examples/` | An example PR pipeline and example rules files. | | `metrics/` | Optional metrics ingestion API and dashboard (Go, Cosmos DB). | -| `infra/` | Bicep for the metrics infrastructure. | +| `infra/` | Bicep and Entra setup scripts for the metrics infrastructure. | | `.github/` | GitHub Actions CI and Dependabot configuration. | | `.azuredevops/` | Azure Pipelines CI for the engine, for teams that host a copy in Azure Repos. | @@ -220,7 +220,7 @@ Every push to `main` and `development` and every pull request runs [GitHub Actio | Go | `gofmt`, `go vet`, `go test -race` and `go build` for `metrics/`. | | Docker | Builds the metrics image (no push), checks it does not run as root, smoke-tests `/healthz`, and checks an unauthenticated `POST /events` gets `401`. | | Secret scan | gitleaks over the full git history. | -| Bicep | Builds and lints every file under `infra/` with a pinned Bicep CLI. | +| Bicep | Builds and lints every file under `infra/` with a pinned Bicep CLI, and runs shellcheck on its scripts. | Actions are pinned by commit SHA, the workflow token is read-only, and Dependabot keeps actions, Go modules and the Docker base image current. diff --git a/infra/README.md b/infra/README.md new file mode 100644 index 0000000..1d39988 --- /dev/null +++ b/infra/README.md @@ -0,0 +1,123 @@ +# Metrics infrastructure + +Bicep and two `az` scripts that deploy the PullPicket metrics API and dashboard to Azure, locked down by default: + +- **Cosmos DB** (serverless, NoSQL): database `AIMetrics`, container `Events`, partition key `/tenantId` then `/featureName`. Key auth is disabled; only Entra identities with a data-plane role can read or write. +- **Container Registry** (Basic): holds the `pullpicket-metrics` image. No admin user, no anonymous pull. +- **Managed identity** (user-assigned): the app runs as it. It has `AcrPull` on the registry and the Cosmos DB Built-in Data Contributor role on the account. +- **Container Apps** (Consumption) with Log Analytics: the app on HTTPS, scaling to zero when idle. +- **Container Apps sign-in** with Entra in front of the dashboard and `/metrics/*`. Only users or groups you assign can sign in. `/events` and `/healthz` are excluded so pipelines can reach them; `/events` checks its own credential. +- **Two Entra app registrations**, made by [`setup-entra.sh`](setup-entra.sh) because Bicep cannot create them: the API (with the `Events.Write` app role pipelines need) and the dashboard sign-in app. + +| File | What it does | +| --- | --- | +| [`main.bicep`](main.bicep) | Everything in one resource group. Without `imageTag` it skips the Container App. | +| [`main.bicepparam`](main.bicepparam) | Example parameters. The image tag and Entra values come from environment variables. | +| [`modules/`](modules) | `cosmos`, `registry`, `identity` (identity and its role grants), `environment`, `containerapp` (app and sign-in). | +| [`setup-entra.sh`](setup-entra.sh) | Creates or updates the two app registrations and writes their IDs and the dashboard secret to `.env.entra`. | +| [`grant-events-write.sh`](grant-events-write.sh) | Gives a pipeline's service connection the `Events.Write` role. | + +## Before you start + +- Azure CLI 2.60 or later, signed in (`az login`), plus `jq` and `bash`. Azure Cloud Shell has all three. +- On the resource group: **Owner**, or **Contributor** plus **Role Based Access Control Administrator**, because the deployment creates role assignments. +- In Entra: rights to create app registrations and assign users to them (the default for members in most tenants, or **Application Administrator**). +- Docker is optional: `az acr build` builds the image in Azure. + +## Deploy + +Run these from the repository root. Replace the placeholders `` and ``. + +**1. Create the resource group and the base resources.** With no image tag the deployment creates Cosmos DB, the registry, the identity and the Container Apps environment, but not the app yet. It also prints the URL the app will get. + +```bash +az group create --name --location +az deployment group create --resource-group --parameters infra/main.bicepparam \ + --query properties.outputs --output json +``` + +Note the `registryName` and `metricsUrl` outputs. + +**2. Register the Entra apps.** Pass `metricsUrl` so the dashboard's sign-in redirect points at the app. By default you are assigned to the dashboard; add `--dashboard-group ` to let a group in instead. + +```bash +infra/setup-entra.sh --app-url +``` + +The script writes `.env.entra` (mode 600, ignored by git) with the API's client ID and application ID URI, the dashboard's client ID and the dashboard client secret. It is safe to rerun; it only adds a new secret when it creates the dashboard app or you pass `--rotate-secret`. + +**3. Build and push the image.** + +```bash +export PULLPICKET_IMAGE_TAG="$(git rev-parse --short HEAD)" +az acr build --registry --image "pullpicket-metrics:$PULLPICKET_IMAGE_TAG" metrics +``` + +If your subscription cannot run ACR Tasks, build locally and push instead: `az acr login --name `, then `docker build` and `docker push` to `.azurecr.io/pullpicket-metrics:$PULLPICKET_IMAGE_TAG`. + +**4. Deploy the app.** `main.bicepparam` reads the tag and the Entra values from the environment, so nothing secret goes on the command line or into a file in the repository. + +```bash +source .env.entra +az deployment group create --resource-group --parameters infra/main.bicepparam \ + --query properties.outputs.metricsUrl.value --output tsv +``` + +Then delete `.env.entra`, or move it somewhere safe; you only need it again to redeploy without rotating the secret. + +**5. Let the pipeline send metrics.** Find the application (client) ID of the Azure DevOps service connection the review stage uses (Project settings, Service connections, the connection, Manage App registration or Manage service principal), then: + +```bash +infra/grant-events-write.sh --principal +``` + +**6. Point the pipeline at the API.** Create a variable group, pass its name as `metricsVariableGroup` in the stage, and add: + +| Variable | Value | +| --- | --- | +| `aiMetricsApiUrl` | `metricsUrl` | +| `aiMetricsApiResource` | the API's application ID URI (`PULLPICKET_API_APP_ID_URI` in `.env.entra`, such as `api://pullpicket-metrics`) | + +**7. Check it.** Open `/` in a browser: you should be sent to the Microsoft sign-in page, then see the landing page. Without a credential, `POST /events` answers `401`: + +```bash +curl -sS -o /dev/null -w '%{http_code}\n' -X POST "/events" -H 'Content-Type: application/json' -d '{}' +``` + +After the next pull request review, the run appears on `/dashboard?feature=ai-code-review`. + +## Outputs + +| Output | Meaning | +| --- | --- | +| `metricsUrl` | Public URL of the API and dashboard. Known from step 1. | +| `containerAppName` | Container App name, `-metrics`. | +| `registryName`, `registryLoginServer` | Registry for `az acr build`; the image is `/pullpicket-metrics:`. | +| `cosmosEndpoint`, `cosmosDatabaseName`, `cosmosContainerName`, `cosmosAccountId` | The store, for running the API elsewhere or for queries. | + +## Options + +Set these in `main.bicepparam` or with `--parameters name=value`. + +| Parameter | Default | Meaning | +| --- | --- | --- | +| `namePrefix` | `pullpicket` | Prefix for resource names, 3 to 20 characters. | +| `authMode` | `entra` | `AUTH_MODE` for `POST /events`: `entra`, `key` or `entra,key`. | +| `metricsApiKey` | empty | Shared key for `key` or `entra,key`, at least 32 characters (`openssl rand -base64 32`), stored as a Container Apps secret. Pass it from an environment variable, not in the file. | +| `entraAllowedPrincipals` | empty | Object or client IDs allowed to write without the `Events.Write` role. | +| `entraTenantId` | this tenant | Tenant that issues the API's tokens and the dashboard sign-ins. | +| `minReplicas`, `maxReplicas` | `0`, `2` | Scale. With `0` the first request after an idle spell waits a few seconds while a replica starts. | +| `location` | resource group's region | Region for every resource. | + +To ship a new version, build with a new tag (step 3) and repeat step 4. + +## Teardown + +```bash +az group delete --name +source .env.entra # or look the IDs up with: az ad app list --display-name pullpicket-metrics +az ad app delete --id "$PULLPICKET_DASHBOARD_CLIENT_ID" +az ad app delete --id "$PULLPICKET_API_CLIENT_ID" +``` + +Deleting the resource group removes the stored events. The Cosmos account keeps 7 days of continuous backup until the account itself is deleted. Deleted app registrations stay restorable for 30 days under Entra ID, App registrations, Deleted applications. diff --git a/infra/grant-events-write.sh b/infra/grant-events-write.sh new file mode 100755 index 0000000..9768a83 --- /dev/null +++ b/infra/grant-events-write.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +# Lets a pipeline send metrics: assigns the metrics API's Events.Write app +# role to the service principal behind an Azure DevOps service connection +# (or any other sender's identity). Safe to rerun. +# +# Usage: +# infra/grant-events-write.sh --principal [options] +# +# Options: +# --principal ID the sender's identity: the service connection's +# application (client) id, a managed identity's client id, +# or a service principal object id (required) +# --api-client-id ID the API app registration's client id (default: +# $PULLPICKET_API_CLIENT_ID from .env.entra, else looked up +# by name) +# --name-prefix NAME prefix used by setup-entra.sh (default: pullpicket) +# +# The same grant by hand, with the object ids of both service principals and +# the role's id: +# az rest --method post \ +# --url https://graph.microsoft.com/v1.0/servicePrincipals//appRoleAssignments \ +# --body '{"principalId":"","resourceId":"","appRoleId":""}' +# +# Needs: az, signed in with rights to grant app roles on the API app (its +# owner, or Application Administrator). +set -euo pipefail + +usage() { sed -n '2,/^set -euo/p' "$0" | sed -e '$d' -e 's/^# \{0,1\}//'; } + +die() { echo "grant-events-write: $*" >&2; exit 1; } + +principal="" +api_client_id="${PULLPICKET_API_CLIENT_ID:-}" +name_prefix="pullpicket" + +while [ $# -gt 0 ]; do + case "$1" in + --principal) principal="${2:-}"; shift 2 ;; + --api-client-id) api_client_id="${2:-}"; shift 2 ;; + --name-prefix) name_prefix="${2:-}"; shift 2 ;; + -h|--help) usage; exit 0 ;; + *) die "unknown option: $1 (see --help)" ;; + esac +done + +[ -n "$principal" ] || die "--principal is required (see --help)." +command -v az >/dev/null || die "the Azure CLI (az) is required." + +graph="https://graph.microsoft.com/v1.0" + +if [ -z "$api_client_id" ]; then + api_client_id="$(az ad app list --display-name "${name_prefix}-metrics-api" --query '[].appId' --output tsv)" + [ -n "$api_client_id" ] || die "no app registration named ${name_prefix}-metrics-api: run infra/setup-entra.sh first." + [ "$(printf '%s\n' "$api_client_id" | grep -c .)" -eq 1 ] || die "more than one app registration is named ${name_prefix}-metrics-api; pass --api-client-id." +fi + +api_sp="$(az ad sp show --id "$api_client_id" --query id --output tsv)" || + die "the API app $api_client_id has no service principal: run infra/setup-entra.sh." +role_id="$(az ad sp show --id "$api_client_id" --query "appRoles[?value=='Events.Write'].id | [0]" --output tsv)" +[ -n "$role_id" ] || die "the API app $api_client_id has no Events.Write role: run infra/setup-entra.sh." + +sender_sp="$(az ad sp show --id "$principal" --query id --output tsv)" || + die "no service principal found for $principal. Use the service connection's application (client) id." +sender_name="$(az ad sp show --id "$sender_sp" --query displayName --output tsv)" + +existing="$(az rest --method get --url "$graph/servicePrincipals/$sender_sp/appRoleAssignments" \ + --query "length(value[?resourceId=='$api_sp' && appRoleId=='$role_id'])" --output tsv)" +if [ "$existing" != "0" ]; then + echo "$sender_name ($sender_sp) already has Events.Write." + exit 0 +fi + +az rest --method post --url "$graph/servicePrincipals/$sender_sp/appRoleAssignments" \ + --headers 'Content-Type=application/json' \ + --body "{\"principalId\":\"$sender_sp\",\"resourceId\":\"$api_sp\",\"appRoleId\":\"$role_id\"}" \ + --output none +echo "Granted Events.Write to $sender_name ($sender_sp)." +echo "Tokens issued from now on carry the role; a token fetched earlier does not, so rerun the pipeline." diff --git a/infra/main.bicep b/infra/main.bicep index dc58c74..962760a 100644 --- a/infra/main.bicep +++ b/infra/main.bicep @@ -1,7 +1,6 @@ // PullPicket metrics infrastructure, deployed to one resource group: // az deployment group create -g -f infra/main.bicep -p infra/main.bicepparam -// Today this creates the Cosmos DB store; the registry, Container App and -// identity wiring (I2, I3, I5 in the plan) are added as further modules. +// See infra/README.md for the full sequence (Entra setup, image build, app). targetScope = 'resourceGroup' @@ -24,6 +23,51 @@ param cosmosDatabaseName string = 'AIMetrics' @description('Container name for metrics events (hierarchical partition key /tenantId, /featureName).') param cosmosContainerName string = 'Events' +@description('Container registry name. Must be globally unique, letters and digits only; the default appends a hash of the resource group.') +@minLength(5) +@maxLength(50) +param registryName string = toLower('${replace(namePrefix, '-', '')}${uniqueString(resourceGroup().id)}') + +@description('Image tag in the registry to run, such as a git commit. Leave empty on the first deploy: everything but the Container App is created, so you can push the image and register the Entra apps first.') +param imageTag string = '' + +@description('How POST /events is authenticated (the app\'s AUTH_MODE).') +@allowed(['entra', 'key', 'entra,key']) +param authMode string = 'entra' + +@description('Tenant that issues the API\'s tokens (ENTRA_TENANT_ID). Defaults to this deployment\'s tenant.') +param entraTenantId string = tenant().tenantId + +@description('The API app registration\'s application ID URI (api://...), from setup-entra.sh.') +param apiAppIdUri string = '' + +@description('The API app registration\'s client (application) ID, from setup-entra.sh.') +param apiClientId string = '' + +@description('Optional comma-separated object ids or client ids allowed to write without the Events.Write role (ENTRA_ALLOWED_PRINCIPALS).') +param entraAllowedPrincipals string = '' + +@description('Shared key for AUTH_MODE key or entra,key (METRICS_API_KEY), at least 32 characters. Leave empty for entra.') +@secure() +param metricsApiKey string = '' + +@description('Client ID of the dashboard app registration (Container Apps sign-in), from setup-entra.sh. Required once imageTag is set.') +param dashboardClientId string = '' + +@description('Client secret of the dashboard app registration, from setup-entra.sh. Required once imageTag is set.') +@secure() +param dashboardClientSecret string = '' + +@description('Minimum replicas; 0 scales to zero when idle.') +@minValue(0) +@maxValue(10) +param minReplicas int = 0 + +@description('Maximum replicas.') +@minValue(1) +@maxValue(10) +param maxReplicas int = 2 + @description('Tags applied to every resource.') param tags object = { application: 'pullpicket' @@ -40,6 +84,64 @@ module cosmos 'modules/cosmos.bicep' = { } } +module registry 'modules/registry.bicep' = { + name: 'registry' + params: { + location: location + name: registryName + tags: tags + } +} + +module identity 'modules/identity.bicep' = { + name: 'identity' + params: { + location: location + name: '${namePrefix}-metrics-id' + registryName: registry.outputs.name + cosmosAccountName: cosmos.outputs.accountName + tags: tags + } +} + +module environment 'modules/environment.bicep' = { + name: 'environment' + params: { + location: location + name: '${namePrefix}-env' + logAnalyticsName: '${namePrefix}-logs' + tags: tags + } +} + +var appName = '${namePrefix}-metrics' + +module app 'modules/containerapp.bicep' = if (!empty(imageTag)) { + name: 'app' + params: { + location: location + name: appName + environmentId: environment.outputs.id + identityId: identity.outputs.id + identityClientId: identity.outputs.clientId + registryLoginServer: registry.outputs.loginServer + image: '${registry.outputs.loginServer}/pullpicket-metrics:${imageTag}' + cosmosEndpoint: cosmos.outputs.endpoint + cosmosDatabaseName: cosmos.outputs.databaseName + cosmosContainerName: cosmos.outputs.containerName + authMode: authMode + entraTenantId: entraTenantId + entraAudiences: filter([apiAppIdUri, apiClientId], audience => !empty(audience)) + entraAllowedPrincipals: entraAllowedPrincipals + metricsApiKey: metricsApiKey + dashboardClientId: dashboardClientId + dashboardClientSecret: dashboardClientSecret + minReplicas: minReplicas + maxReplicas: maxReplicas + tags: tags + } +} + @description('Resource ID of the Cosmos DB account.') output cosmosAccountId string = cosmos.outputs.accountId @@ -51,3 +153,15 @@ output cosmosDatabaseName string = cosmos.outputs.databaseName @description('COSMOS_CONTAINER for the metrics API.') output cosmosContainerName string = cosmos.outputs.containerName + +@description('Container registry name, for az acr build.') +output registryName string = registry.outputs.name + +@description('Container registry login server; the image is /pullpicket-metrics:.') +output registryLoginServer string = registry.outputs.loginServer + +@description('Container App name.') +output containerAppName string = appName + +@description('Public URL of the metrics API and dashboard. Known from the first deploy, before the app exists, so it can go into the Entra redirect URI.') +output metricsUrl string = 'https://${appName}.${environment.outputs.defaultDomain}' diff --git a/infra/main.bicepparam b/infra/main.bicepparam index 3d869b1..33ef6c6 100644 --- a/infra/main.bicepparam +++ b/infra/main.bicepparam @@ -1,8 +1,22 @@ -// Example parameters. Replace the placeholder values before deploying. +// Parameters for infra/main.bicep. Replace the placeholder values before deploying. +// The image tag and Entra values come from environment variables, so no ID or +// secret is written here: infra/setup-entra.sh writes them to .env.entra +// (source it), and you set PULLPICKET_IMAGE_TAG after pushing the image. +// With PULLPICKET_IMAGE_TAG unset, the deployment skips the Container App. using 'main.bicep' param namePrefix = 'pullpicket' +param imageTag = readEnvironmentVariable('PULLPICKET_IMAGE_TAG', '') + +param apiAppIdUri = readEnvironmentVariable('PULLPICKET_API_APP_ID_URI', '') + +param apiClientId = readEnvironmentVariable('PULLPICKET_API_CLIENT_ID', '') + +param dashboardClientId = readEnvironmentVariable('PULLPICKET_DASHBOARD_CLIENT_ID', '') + +param dashboardClientSecret = readEnvironmentVariable('PULLPICKET_DASHBOARD_CLIENT_SECRET', '') + param tags = { application: 'pullpicket' environment: 'dev' diff --git a/infra/modules/containerapp.bicep b/infra/modules/containerapp.bicep new file mode 100644 index 0000000..0922610 --- /dev/null +++ b/infra/modules/containerapp.bicep @@ -0,0 +1,228 @@ +// The metrics API and dashboard as a Container App. +// +// It runs as the user-assigned identity from identity.bicep: that identity +// pulls the image from ACR and signs in to Cosmos DB (key auth is disabled), +// found by the app through AZURE_CLIENT_ID. POST /events is guarded by the +// app itself (AUTH_MODE); everything else, including the dashboard and +// /metrics/*, sits behind Container Apps sign-in with Entra (signIn below). + +@description('Azure region for the app.') +param location string + +@description('Container App name: lowercase letters, digits and hyphens, at most 32 characters.') +@minLength(2) +@maxLength(32) +param name string + +@description('Resource ID of the Container Apps environment.') +param environmentId string + +@description('Resource ID of the user-assigned identity the app runs as.') +param identityId string + +@description('Client ID of that identity.') +param identityClientId string + +@description('Registry login server, such as .azurecr.io.') +param registryLoginServer string + +@description('Full image reference, such as .azurecr.io/pullpicket-metrics:.') +param image string + +@description('Cosmos DB document endpoint.') +param cosmosEndpoint string + +@description('Cosmos DB database name.') +param cosmosDatabaseName string + +@description('Cosmos DB container name.') +param cosmosContainerName string + +@description('AUTH_MODE for POST /events.') +@allowed(['entra', 'key', 'entra,key']) +param authMode string + +@description('ENTRA_TENANT_ID: the tenant that issues tokens.') +param entraTenantId string + +@description('ENTRA_AUDIENCE: accepted audiences, normally the API app\'s application ID URI and client id.') +param entraAudiences array + +@description('ENTRA_ALLOWED_PRINCIPALS: optional comma-separated object ids or client ids allowed to write without the Events.Write role.') +param entraAllowedPrincipals string + +@description('METRICS_API_KEY for AUTH_MODE key or entra,key; stored as a Container Apps secret.') +@secure() +param metricsApiKey string + +@description('Client ID of the dashboard app registration (Container Apps sign-in), from setup-entra.sh.') +@minLength(1) +param dashboardClientId string + +@description('Client secret of the dashboard app registration, from setup-entra.sh.') +@secure() +@minLength(1) +param dashboardClientSecret string + +@description('Minimum replicas. 0 scales to zero when idle (the first request after idle waits a few seconds).') +@minValue(0) +@maxValue(10) +param minReplicas int + +@description('Maximum replicas.') +@minValue(1) +@maxValue(10) +param maxReplicas int + +@description('Tags applied to every resource.') +param tags object + +var usesKey = contains(authMode, 'key') + +// Container Apps sign-in reads the dashboard app's client secret from this app secret. +var signInSecretName = 'microsoft-provider-authentication-secret' + +var secrets = concat( + [ + { + name: signInSecretName + value: dashboardClientSecret + } + ], + usesKey + ? [ + { + name: 'metrics-api-key' + value: metricsApiKey + } + ] + : [] +) + +var baseEnv = [ + { name: 'COSMOS_ENDPOINT', value: cosmosEndpoint } + { name: 'COSMOS_DATABASE', value: cosmosDatabaseName } + { name: 'COSMOS_CONTAINER', value: cosmosContainerName } + { name: 'AZURE_CLIENT_ID', value: identityClientId } + { name: 'AUTH_MODE', value: authMode } +] + +var entraEnv = contains(authMode, 'entra') + ? concat( + [ + { name: 'ENTRA_TENANT_ID', value: entraTenantId } + { name: 'ENTRA_AUDIENCE', value: join(entraAudiences, ',') } + ], + empty(entraAllowedPrincipals) ? [] : [{ name: 'ENTRA_ALLOWED_PRINCIPALS', value: entraAllowedPrincipals }] + ) + : [] + +var keyEnv = usesKey ? [{ name: 'METRICS_API_KEY', secretRef: 'metrics-api-key' }] : [] + +resource app 'Microsoft.App/containerApps@2025-01-01' = { + name: name + location: location + tags: tags + identity: { + type: 'UserAssigned' + userAssignedIdentities: { + '${identityId}': {} + } + } + properties: { + environmentId: environmentId + workloadProfileName: 'Consumption' + configuration: { + activeRevisionsMode: 'Single' + ingress: { + external: true + targetPort: 8080 + transport: 'auto' + allowInsecure: false + } + registries: [ + { + server: registryLoginServer + identity: identityId + } + ] + secrets: secrets + } + template: { + containers: [ + { + name: 'metrics' + image: image + resources: { + cpu: json('0.25') + memory: '0.5Gi' + } + env: concat(baseEnv, entraEnv, keyEnv) + probes: [ + { + type: 'Liveness' + httpGet: { + path: '/healthz' + port: 8080 + } + periodSeconds: 30 + } + { + type: 'Readiness' + httpGet: { + path: '/healthz' + port: 8080 + } + periodSeconds: 10 + } + ] + } + ] + scale: { + minReplicas: minReplicas + maxReplicas: maxReplicas + } + } + } +} + +// Container Apps sign-in (Easy Auth) in front of everything except the two +// paths pipelines call: POST /events checks its own credential (AUTH_MODE) +// and /healthz is the probe. Who may sign in is decided in Entra: the +// dashboard app requires assignment, so only assigned users or groups get in. +resource signIn 'Microsoft.App/containerApps/authConfigs@2025-01-01' = { + parent: app + name: 'current' + properties: { + platform: { + enabled: true + } + globalValidation: { + unauthenticatedClientAction: 'RedirectToLoginPage' + redirectToProvider: 'azureactivedirectory' + excludedPaths: [ + '/events' + '/healthz' + ] + } + httpSettings: { + requireHttps: true + } + identityProviders: { + azureActiveDirectory: { + enabled: true + registration: { + clientId: dashboardClientId + clientSecretSettingName: signInSecretName + openIdIssuer: '${environment().authentication.loginEndpoint}${entraTenantId}/v2.0' + } + } + } + } +} + +@description('Container App name.') +output name string = app.name + +@description('Public URL of the app.') +output url string = 'https://${app.properties.configuration.ingress.fqdn}' diff --git a/infra/modules/environment.bicep b/infra/modules/environment.bicep new file mode 100644 index 0000000..8d6f0e6 --- /dev/null +++ b/infra/modules/environment.bicep @@ -0,0 +1,59 @@ +// Container Apps environment (Consumption) with its logs in Log Analytics. +// It is deployed before the app so its default domain, and so the app's +// public URL, is known in time for the Entra redirect URI (setup-entra.sh). + +@description('Azure region for the environment and workspace.') +param location string + +@description('Container Apps environment name.') +param name string + +@description('Log Analytics workspace name.') +param logAnalyticsName string + +@description('Days to keep logs.') +@minValue(30) +@maxValue(730) +param logRetentionDays int = 30 + +@description('Tags applied to every resource.') +param tags object + +resource logs 'Microsoft.OperationalInsights/workspaces@2025-02-01' = { + name: logAnalyticsName + location: location + tags: tags + properties: { + sku: { + name: 'PerGB2018' + } + retentionInDays: logRetentionDays + } +} + +resource environment 'Microsoft.App/managedEnvironments@2025-01-01' = { + name: name + location: location + tags: tags + properties: { + appLogsConfiguration: { + destination: 'log-analytics' + logAnalyticsConfiguration: { + customerId: logs.properties.customerId + sharedKey: logs.listKeys().primarySharedKey + } + } + workloadProfiles: [ + { + name: 'Consumption' + workloadProfileType: 'Consumption' + } + ] + } +} + +@description('Resource ID of the environment.') +output id string = environment.id + +@description('Default domain; an app named is served at https://..') +output defaultDomain string = environment.properties.defaultDomain diff --git a/infra/modules/identity.bicep b/infra/modules/identity.bicep new file mode 100644 index 0000000..7f54592 --- /dev/null +++ b/infra/modules/identity.bicep @@ -0,0 +1,74 @@ +// The metrics app's identity and what it may do: pull from ACR (AcrPull) +// and read and write metrics events (Cosmos DB Built-in Data Contributor). +// +// A user-assigned identity rather than the app's system-assigned one: a +// system-assigned identity only exists once the app does, so its AcrPull +// grant would land after the app's first image pull and that pull would fail. +// Creating the identity and its grants first lets the app start on the first +// deploy. The app finds this identity through AZURE_CLIENT_ID. + +@description('Azure region for the identity.') +param location string + +@description('Managed identity name.') +param name string + +@description('Name of the container registry the app pulls from (AcrPull).') +param registryName string + +@description('Name of the Cosmos DB account the app reads and writes (Built-in Data Contributor).') +param cosmosAccountName string + +@description('Tags applied to every resource.') +param tags object + +// Built-in role: AcrPull. +var acrPullRoleId = '7f951dda-4ed3-4680-a7ca-43fe172d538d' + +// Cosmos DB data-plane role: Built-in Data Contributor (read and write items). +var cosmosDataContributorRoleId = '00000000-0000-0000-0000-000000000002' + +resource identity 'Microsoft.ManagedIdentity/userAssignedIdentities@2024-11-30' = { + name: name + location: location + tags: tags +} + +resource registry 'Microsoft.ContainerRegistry/registries@2025-04-01' existing = { + name: registryName +} + +resource acrPull 'Microsoft.Authorization/roleAssignments@2022-04-01' = { + name: guid(registry.id, identity.id, acrPullRoleId) + scope: registry + properties: { + roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', acrPullRoleId) + principalId: identity.properties.principalId + principalType: 'ServicePrincipal' + } +} + +resource cosmosAccount 'Microsoft.DocumentDB/databaseAccounts@2024-11-15' existing = { + name: cosmosAccountName +} + +// A data-plane (SQL) role assignment, not Azure RBAC: with key auth disabled +// this is the only way the app reaches the data. Scoped to the account. +resource cosmosDataContributor 'Microsoft.DocumentDB/databaseAccounts/sqlRoleAssignments@2024-11-15' = { + parent: cosmosAccount + name: guid(cosmosAccount.id, identity.id, cosmosDataContributorRoleId) + properties: { + roleDefinitionId: '${cosmosAccount.id}/sqlRoleDefinitions/${cosmosDataContributorRoleId}' + principalId: identity.properties.principalId + scope: cosmosAccount.id + } +} + +@description('Resource ID of the identity, to attach it to the app.') +output id string = identity.id + +@description('Client ID of the identity; the app gets it as AZURE_CLIENT_ID.') +output clientId string = identity.properties.clientId + +@description('Object (principal) ID of the identity, for data-plane role assignments.') +output principalId string = identity.properties.principalId diff --git a/infra/modules/registry.bicep b/infra/modules/registry.bicep new file mode 100644 index 0000000..ad34201 --- /dev/null +++ b/infra/modules/registry.bicep @@ -0,0 +1,37 @@ +// Azure Container Registry for the metrics image (D9: ACR only for 1.0). +// No admin user and no anonymous pull: the Container App pulls with its +// managed identity, which holds AcrPull on this registry (see identity.bicep). + +@description('Azure region for the registry.') +param location string + +@description('Registry name: 5-50 letters and digits, globally unique.') +@minLength(5) +@maxLength(50) +param name string + +@description('Tags applied to every resource.') +param tags object + +resource registry 'Microsoft.ContainerRegistry/registries@2025-04-01' = { + name: name + location: location + tags: tags + sku: { + name: 'Basic' + } + properties: { + adminUserEnabled: false + anonymousPullEnabled: false + publicNetworkAccess: 'Enabled' + } +} + +@description('Resource ID of the registry, for role assignments.') +output id string = registry.id + +@description('Registry name, for az acr build and az acr login.') +output name string = registry.name + +@description('Login server, such as .azurecr.io; image references start with it.') +output loginServer string = registry.properties.loginServer diff --git a/infra/setup-entra.sh b/infra/setup-entra.sh new file mode 100755 index 0000000..01a038d --- /dev/null +++ b/infra/setup-entra.sh @@ -0,0 +1,244 @@ +#!/usr/bin/env bash +# Creates or updates the two Microsoft Entra app registrations the metrics +# service needs (decision D8: an az CLI script, because Bicep cannot create +# app registrations): +# +# -metrics-api the API that senders get tokens for. It defines +# the Events.Write app role; POST /events accepts +# only tokens that carry it. +# -metrics-dashboard the sign-in app for Container Apps +# authentication in front of the dashboard. Only +# users or groups assigned to it can sign in. +# +# Safe to rerun: existing registrations are found by name and updated. +# +# Usage: +# infra/setup-entra.sh --app-url https://. [options] +# +# Options: +# --app-url URL the metricsUrl output of the first deployment (required) +# --name-prefix NAME prefix for the registrations' names (default: pullpicket) +# --api-app-id-uri URI application ID URI for the API (default: api://-metrics; +# falls back to api:// if the tenant refuses it) +# --dashboard-group ID object id of a group allowed to open the dashboard +# (default: the signed-in user) +# --rotate-secret add a new client secret for the dashboard app even if one +# is already recorded in the output file +# --out FILE where to write the results (default: .env.entra, mode 600) +# +# The output file holds export lines that infra/main.bicepparam reads, +# including the dashboard client secret. Keep it out of source control +# (.env.* is git-ignored) and delete it once the app is deployed. +# +# Needs: az (signed in to the target tenant, with rights to create app +# registrations) and jq. +set -euo pipefail + +usage() { sed -n '2,/^set -euo/p' "$0" | sed -e '$d' -e 's/^# \{0,1\}//'; } + +die() { echo "setup-entra: $*" >&2; exit 1; } + +app_url="" +name_prefix="pullpicket" +api_app_id_uri="" +dashboard_group="" +rotate_secret=false +out_file=".env.entra" + +while [ $# -gt 0 ]; do + case "$1" in + --app-url) app_url="${2:-}"; shift 2 ;; + --name-prefix) name_prefix="${2:-}"; shift 2 ;; + --api-app-id-uri) api_app_id_uri="${2:-}"; shift 2 ;; + --dashboard-group) dashboard_group="${2:-}"; shift 2 ;; + --rotate-secret) rotate_secret=true; shift ;; + --out) out_file="${2:-}"; shift 2 ;; + -h|--help) usage; exit 0 ;; + *) die "unknown option: $1 (see --help)" ;; + esac +done + +[ -n "$app_url" ] || die "--app-url is required: use the metricsUrl output of the first deployment." +app_url="${app_url%/}" +case "$app_url" in + https://*) ;; + *) die "--app-url must start with https://" ;; +esac +[ -n "$name_prefix" ] || die "--name-prefix must not be empty." +[ -n "$out_file" ] || die "--out must not be empty." +command -v az >/dev/null || die "the Azure CLI (az) is required." +command -v jq >/dev/null || die "jq is required." + +graph="https://graph.microsoft.com/v1.0" +api_name="${name_prefix}-metrics-api" +dashboard_name="${name_prefix}-metrics-dashboard" +api_app_id_uri="${api_app_id_uri:-api://${name_prefix}-metrics}" +# Container Apps sign-in uses the empty (default) app role when a user or +# group is assigned to an app that defines no roles of its own. +default_access_role="00000000-0000-0000-0000-000000000000" + +new_uuid() { + if command -v uuidgen >/dev/null; then + uuidgen | tr '[:upper:]' '[:lower:]' + elif [ -r /proc/sys/kernel/random/uuid ]; then + cat /proc/sys/kernel/random/uuid + else + die "cannot generate a UUID: install uuidgen." + fi +} + +# Prints the appId of the single app registration with this display name, +# or nothing. Fails if the name is ambiguous. +find_app() { + local ids count + ids="$(az ad app list --display-name "$1" --query '[].appId' --output tsv)" + count="$(printf '%s' "$ids" | grep -c . || true)" + [ "$count" -le 1 ] || die "$count app registrations are named '$1'; delete the extras or use --name-prefix." + printf '%s' "$ids" +} + +# Prints the object id of the app's service principal, creating it if needed. +ensure_sp() { + local id + id="$(az ad sp list --filter "appId eq '$1'" --query '[0].id' --output tsv)" + if [ -z "$id" ]; then + id="$(az ad sp create --id "$1" --query id --output tsv)" + fi + printf '%s' "$id" +} + +tenant_id="$(az account show --query tenantId --output tsv)" +[ -n "$tenant_id" ] || die "not signed in: run az login --tenant first." +echo "Tenant: $tenant_id" + +# --- API app registration ---------------------------------------------------- + +api_client_id="$(find_app "$api_name")" +if [ -z "$api_client_id" ]; then + echo "Creating app registration $api_name" + api_client_id="$(az ad app create --display-name "$api_name" --sign-in-audience AzureADMyOrg \ + --query appId --output tsv)" +else + echo "Found app registration $api_name ($api_client_id)" +fi + +# Events.Write: an application role, so only service principals (the +# pipeline's service connection) can hold it, never users. +roles="$(az ad app show --id "$api_client_id" --query appRoles --output json)" +if [ "$(jq '[.[] | select(.value == "Events.Write")] | length' <<<"$roles")" -eq 0 ]; then + echo "Adding the Events.Write app role" + roles_file="$(mktemp)" + trap 'rm -f "$roles_file"' EXIT + jq --arg id "$(new_uuid)" '. + [{ + allowedMemberTypes: ["Application"], + description: "Send review metrics events to POST /events.", + displayName: "Events.Write", + id: $id, + isEnabled: true, + value: "Events.Write" + }]' <<<"$roles" >"$roles_file" + az ad app update --id "$api_client_id" --app-roles "@$roles_file" +fi + +current_uri="$(az ad app show --id "$api_client_id" --query 'identifierUris[0]' --output tsv)" +if [ -n "$current_uri" ]; then + api_app_id_uri="$current_uri" +else + echo "Setting the application ID URI to $api_app_id_uri" + if ! az ad app update --id "$api_client_id" --identifier-uris "$api_app_id_uri" 2>/dev/null; then + # Some tenants only allow api:// or a verified domain. + api_app_id_uri="api://$api_client_id" + echo "The tenant refused it; using $api_app_id_uri instead" + az ad app update --id "$api_client_id" --identifier-uris "$api_app_id_uri" + fi +fi + +ensure_sp "$api_client_id" >/dev/null + +# --- Dashboard app registration ---------------------------------------------- + +redirect_uri="$app_url/.auth/login/aad/callback" +dashboard_client_id="$(find_app "$dashboard_name")" +dashboard_created=false +if [ -z "$dashboard_client_id" ]; then + echo "Creating app registration $dashboard_name" + dashboard_client_id="$(az ad app create --display-name "$dashboard_name" --sign-in-audience AzureADMyOrg \ + --web-redirect-uris "$redirect_uri" --enable-id-token-issuance true \ + --query appId --output tsv)" + dashboard_created=true +else + echo "Found app registration $dashboard_name ($dashboard_client_id); setting its redirect URI" + az ad app update --id "$dashboard_client_id" --web-redirect-uris "$redirect_uri" --enable-id-token-issuance true +fi + +dashboard_sp="$(ensure_sp "$dashboard_client_id")" +# Only assigned users and groups may sign in to the dashboard. +az ad sp update --id "$dashboard_sp" --set appRoleAssignmentRequired=true + +if [ -n "$dashboard_group" ]; then + principal="$dashboard_group" + principal_label="group $dashboard_group" +else + principal="$(az ad signed-in-user show --query id --output tsv 2>/dev/null || true)" + principal_label="you ($principal)" +fi +if [ -z "$principal" ]; then + echo "WARNING: signed in as a service principal, so nobody was given dashboard access." >&2 + echo " Rerun with --dashboard-group ." >&2 +else + assigned="$(az rest --method get --url "$graph/servicePrincipals/$dashboard_sp/appRoleAssignedTo" \ + --query "length(value[?principalId=='$principal'])" --output tsv)" + if [ "$assigned" = "0" ]; then + echo "Allowing $principal_label to open the dashboard" + az rest --method post --url "$graph/servicePrincipals/$dashboard_sp/appRoleAssignedTo" \ + --headers 'Content-Type=application/json' \ + --body "{\"principalId\":\"$principal\",\"resourceId\":\"$dashboard_sp\",\"appRoleId\":\"$default_access_role\"}" \ + --output none + else + echo "$principal_label can already open the dashboard" + fi +fi + +# Container Apps sign-in needs a client secret for the dashboard app. Keep the +# one already recorded in the output file unless asked to rotate it. +dashboard_secret="" +if [ "$rotate_secret" = false ] && [ "$dashboard_created" = false ] && [ -f "$out_file" ]; then + dashboard_secret="$(sed -n "s/^export PULLPICKET_DASHBOARD_CLIENT_SECRET='\(.*\)'$/\1/p" "$out_file")" +fi +if [ -z "$dashboard_secret" ]; then + echo "Adding a client secret to $dashboard_name (valid for 1 year)" + dashboard_secret="$(az ad app credential reset --id "$dashboard_client_id" --append \ + --display-name container-apps-auth --years 1 --query password --output tsv)" +fi + +# --- Results ------------------------------------------------------------------- + +( + umask 077 + cat >"$out_file" < +EOF