Phase 6: deploy the metrics infrastructure (I2–I7)

.github/workflows/ci.yml

@@ -223,3 +223,9 @@ jobs:
             echo "::endgroup::"
           done < <(find infra -type f \( -name '*.bicep' -o -name '*.bicepparam' \) -print0 | sort -z)
           exit "$status"
+
+      # The Entra setup scripts sit next to the Bicep; shellcheck ships on the runner image.
+      - name: Shellcheck infra/ scripts
+        run: |
+          shellcheck --version
+          find infra -type f -name '*.sh' -print0 | xargs -0 --no-run-if-empty shellcheck

CHANGELOG.md

@@ -48,3 +48,5 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Dependabot updates for GitHub Actions and Go modules.
 - Bicep for the metrics store (`infra/main.bicep`, `infra/main.bicepparam`): a serverless Cosmos DB account with key-based auth disabled, database `AIMetrics` and container `Events` partitioned on `/featureName`.
 - CI job that builds and lints every Bicep file under `infra/` with a version- and checksum-pinned Bicep CLI; linter levels live in `infra/bicepconfig.json`.
+- Full metrics deployment in `infra/`: a container registry (no admin user), a user-assigned identity with `AcrPull` and the Cosmos DB Built-in Data Contributor role, a Container Apps environment with Log Analytics, and the Container App with `AUTH_MODE`, `ENTRA_TENANT_ID` and `ENTRA_AUDIENCE` set. Container Apps sign-in with Entra guards everything except `/events` and `/healthz`. Steps, outputs and teardown are in `infra/README.md`.
+- `infra/setup-entra.sh` registers the metrics API (with the `Events.Write` app role) and the dashboard sign-in app (assignment required), and `infra/grant-events-write.sh` gives a pipeline's service connection `Events.Write`. CI runs shellcheck on both.

README.md

@@ -196,8 +196,8 @@ Events may carry an optional `tenantId` (default `default`). Each tenant's event
 
 Missing or invalid credentials get `401`. [`metrics/scripts/send-event-curl.sh`](metrics/scripts/send-event-curl.sh) sends a test event with either credential.
 
-<!-- TODO(I1-I7, X5): link infra/README.md and docs/metrics.md once the full deployment (ACR, Container Apps, Entra setup script) is in place. -->
-**Deployment (in progress).** [`infra/`](infra) currently holds Bicep for the Cosmos DB store (serverless, key auth disabled). Container Registry, Container Apps, the Entra setup script and step-by-step deploy docs will follow.
+<!-- TODO(X5): link docs/metrics.md (auth modes, using the dashboard, pricing file) once written. -->
+**Deployment.** [`infra/README.md`](infra/README.md) deploys the service to Azure in one resource group: Cosmos DB with key auth disabled, a container registry, Container Apps running as a managed identity, and Entra sign-in in front of the dashboard. Two scripts handle Entra: [`infra/setup-entra.sh`](infra/setup-entra.sh) registers the API (with the `Events.Write` role) and the dashboard sign-in app, and [`infra/grant-events-write.sh`](infra/grant-events-write.sh) gives your pipeline's service connection the role.
 
 ## Repository layout
 
@@ -206,7 +206,7 @@ Missing or invalid credentials get `401`. [`metrics/scripts/send-event-curl.sh`]
 | `ai-code-review/` | The review engine: PowerShell module, stage template, base prompt and Pester tests. |
 | `examples/` | An example PR pipeline and example rules files. |
 | `metrics/` | Optional metrics ingestion API and dashboard (Go, Cosmos DB). |
-| `infra/` | Bicep for the metrics infrastructure. |
+| `infra/` | Bicep and Entra setup scripts for the metrics infrastructure. |
 | `.github/` | GitHub Actions CI and Dependabot configuration. |
 | `.azuredevops/` | Azure Pipelines CI for the engine, for teams that host a copy in Azure Repos. |
 
@@ -220,7 +220,7 @@ Every push to `main` and `development` and every pull request runs [GitHub Actio
 | Go | `gofmt`, `go vet`, `go test -race` and `go build` for `metrics/`. |
 | Docker | Builds the metrics image (no push), checks it does not run as root, smoke-tests `/healthz`, and checks an unauthenticated `POST /events` gets `401`. |
 | Secret scan | gitleaks over the full git history. |
-| Bicep | Builds and lints every file under `infra/` with a pinned Bicep CLI. |
+| Bicep | Builds and lints every file under `infra/` with a pinned Bicep CLI, and runs shellcheck on its scripts. |
 
 Actions are pinned by commit SHA, the workflow token is read-only, and Dependabot keeps actions, Go modules and the Docker base image current.
 

infra/README.md

@@ -0,0 +1,123 @@
+# Metrics infrastructure
+
+Bicep and two `az` scripts that deploy the PullPicket metrics API and dashboard to Azure, locked down by default:
+
+- **Cosmos DB** (serverless, NoSQL): database `AIMetrics`, container `Events`, partition key `/tenantId` then `/featureName`. Key auth is disabled; only Entra identities with a data-plane role can read or write.
+- **Container Registry** (Basic): holds the `pullpicket-metrics` image. No admin user, no anonymous pull.
+- **Managed identity** (user-assigned): the app runs as it. It has `AcrPull` on the registry and the Cosmos DB Built-in Data Contributor role on the account.
+- **Container Apps** (Consumption) with Log Analytics: the app on HTTPS, scaling to zero when idle.
+- **Container Apps sign-in** with Entra in front of the dashboard and `/metrics/*`. Only users or groups you assign can sign in. `/events` and `/healthz` are excluded so pipelines can reach them; `/events` checks its own credential.
+- **Two Entra app registrations**, made by [`setup-entra.sh`](setup-entra.sh) because Bicep cannot create them: the API (with the `Events.Write` app role pipelines need) and the dashboard sign-in app.
+
+| File | What it does |
+| --- | --- |
+| [`main.bicep`](main.bicep) | Everything in one resource group. Without `imageTag` it skips the Container App. |
+| [`main.bicepparam`](main.bicepparam) | Example parameters. The image tag and Entra values come from environment variables. |
+| [`modules/`](modules) | `cosmos`, `registry`, `identity` (identity and its role grants), `environment`, `containerapp` (app and sign-in). |
+| [`setup-entra.sh`](setup-entra.sh) | Creates or updates the two app registrations and writes their IDs and the dashboard secret to `.env.entra`. |
+| [`grant-events-write.sh`](grant-events-write.sh) | Gives a pipeline's service connection the `Events.Write` role. |
+
+## Before you start
+
+- Azure CLI 2.60 or later, signed in (`az login`), plus `jq` and `bash`. Azure Cloud Shell has all three.
+- On the resource group: **Owner**, or **Contributor** plus **Role Based Access Control Administrator**, because the deployment creates role assignments.
+- In Entra: rights to create app registrations and assign users to them (the default for members in most tenants, or **Application Administrator**).
+- Docker is optional: `az acr build` builds the image in Azure.
+
+## Deploy
+
+Run these from the repository root. Replace the placeholders `<rg>` and `<region>`.
+
+**1. Create the resource group and the base resources.** With no image tag the deployment creates Cosmos DB, the registry, the identity and the Container Apps environment, but not the app yet. It also prints the URL the app will get.
+
+```bash
+az group create --name <rg> --location <region>
+az deployment group create --resource-group <rg> --parameters infra/main.bicepparam \
+  --query properties.outputs --output json
+```
+
+Note the `registryName` and `metricsUrl` outputs.
+
+**2. Register the Entra apps.** Pass `metricsUrl` so the dashboard's sign-in redirect points at the app. By default you are assigned to the dashboard; add `--dashboard-group <group object id>` to let a group in instead.
+
+```bash
+infra/setup-entra.sh --app-url <metricsUrl>
+```
+
+The script writes `.env.entra` (mode 600, ignored by git) with the API's client ID and application ID URI, the dashboard's client ID and the dashboard client secret. It is safe to rerun; it only adds a new secret when it creates the dashboard app or you pass `--rotate-secret`.
+
+**3. Build and push the image.**
+
+```bash
+export PULLPICKET_IMAGE_TAG="$(git rev-parse --short HEAD)"
+az acr build --registry <registryName> --image "pullpicket-metrics:$PULLPICKET_IMAGE_TAG" metrics
+```
+
+If your subscription cannot run ACR Tasks, build locally and push instead: `az acr login --name <registryName>`, then `docker build` and `docker push` to `<registryName>.azurecr.io/pullpicket-metrics:$PULLPICKET_IMAGE_TAG`.
+
+**4. Deploy the app.** `main.bicepparam` reads the tag and the Entra values from the environment, so nothing secret goes on the command line or into a file in the repository.
+
+```bash
+source .env.entra
+az deployment group create --resource-group <rg> --parameters infra/main.bicepparam \
+  --query properties.outputs.metricsUrl.value --output tsv
+```
+
+Then delete `.env.entra`, or move it somewhere safe; you only need it again to redeploy without rotating the secret.
+
+**5. Let the pipeline send metrics.** Find the application (client) ID of the Azure DevOps service connection the review stage uses (Project settings, Service connections, the connection, Manage App registration or Manage service principal), then:
+
+```bash
+infra/grant-events-write.sh --principal <service connection client id>
+```
+
+**6. Point the pipeline at the API.** Create a variable group, pass its name as `metricsVariableGroup` in the stage, and add:
+
+| Variable | Value |
+| --- | --- |
+| `aiMetricsApiUrl` | `metricsUrl` |
+| `aiMetricsApiResource` | the API's application ID URI (`PULLPICKET_API_APP_ID_URI` in `.env.entra`, such as `api://pullpicket-metrics`) |
+
+**7. Check it.** Open `<metricsUrl>/` in a browser: you should be sent to the Microsoft sign-in page, then see the landing page. Without a credential, `POST /events` answers `401`:
+
+```bash
+curl -sS -o /dev/null -w '%{http_code}\n' -X POST "<metricsUrl>/events" -H 'Content-Type: application/json' -d '{}'
+```
+
+After the next pull request review, the run appears on `<metricsUrl>/dashboard?feature=ai-code-review`.
+
+## Outputs
+
+| Output | Meaning |
+| --- | --- |
+| `metricsUrl` | Public URL of the API and dashboard. Known from step 1. |
+| `containerAppName` | Container App name, `<namePrefix>-metrics`. |
+| `registryName`, `registryLoginServer` | Registry for `az acr build`; the image is `<registryLoginServer>/pullpicket-metrics:<tag>`. |
+| `cosmosEndpoint`, `cosmosDatabaseName`, `cosmosContainerName`, `cosmosAccountId` | The store, for running the API elsewhere or for queries. |
+
+## Options
+
+Set these in `main.bicepparam` or with `--parameters name=value`.
+
+| Parameter | Default | Meaning |
+| --- | --- | --- |
+| `namePrefix` | `pullpicket` | Prefix for resource names, 3 to 20 characters. |
+| `authMode` | `entra` | `AUTH_MODE` for `POST /events`: `entra`, `key` or `entra,key`. |
+| `metricsApiKey` | empty | Shared key for `key` or `entra,key`, at least 32 characters (`openssl rand -base64 32`), stored as a Container Apps secret. Pass it from an environment variable, not in the file. |
+| `entraAllowedPrincipals` | empty | Object or client IDs allowed to write without the `Events.Write` role. |
+| `entraTenantId` | this tenant | Tenant that issues the API's tokens and the dashboard sign-ins. |
+| `minReplicas`, `maxReplicas` | `0`, `2` | Scale. With `0` the first request after an idle spell waits a few seconds while a replica starts. |
+| `location` | resource group's region | Region for every resource. |
+
+To ship a new version, build with a new tag (step 3) and repeat step 4.
+
+## Teardown
+
+```bash
+az group delete --name <rg>
+source .env.entra   # or look the IDs up with: az ad app list --display-name pullpicket-metrics
+az ad app delete --id "$PULLPICKET_DASHBOARD_CLIENT_ID"
+az ad app delete --id "$PULLPICKET_API_CLIENT_ID"
+```
+
+Deleting the resource group removes the stored events. The Cosmos account keeps 7 days of continuous backup until the account itself is deleted. Deleted app registrations stay restorable for 30 days under Entra ID, App registrations, Deleted applications.

infra/grant-events-write.sh

@@ -0,0 +1,78 @@
+#!/usr/bin/env bash
+# Lets a pipeline send metrics: assigns the metrics API's Events.Write app
+# role to the service principal behind an Azure DevOps service connection
+# (or any other sender's identity). Safe to rerun.
+#
+# Usage:
+#   infra/grant-events-write.sh --principal <app id or object id> [options]
+#
+# Options:
+#   --principal ID       the sender's identity: the service connection's
+#                        application (client) id, a managed identity's client id,
+#                        or a service principal object id (required)
+#   --api-client-id ID   the API app registration's client id (default:
+#                        $PULLPICKET_API_CLIENT_ID from .env.entra, else looked up
+#                        by name)
+#   --name-prefix NAME   prefix used by setup-entra.sh (default: pullpicket)
+#
+# The same grant by hand, with the object ids of both service principals and
+# the role's id:
+#   az rest --method post \
+#     --url https://graph.microsoft.com/v1.0/servicePrincipals/<sender sp id>/appRoleAssignments \
+#     --body '{"principalId":"<sender sp id>","resourceId":"<api sp id>","appRoleId":"<Events.Write id>"}'
+#
+# Needs: az, signed in with rights to grant app roles on the API app (its
+# owner, or Application Administrator).
+set -euo pipefail
+
+usage() { sed -n '2,/^set -euo/p' "$0" | sed -e '$d' -e 's/^# \{0,1\}//'; }
+
+die() { echo "grant-events-write: $*" >&2; exit 1; }
+
+principal=""
+api_client_id="${PULLPICKET_API_CLIENT_ID:-}"
+name_prefix="pullpicket"
+
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --principal) principal="${2:-}"; shift 2 ;;
+    --api-client-id) api_client_id="${2:-}"; shift 2 ;;
+    --name-prefix) name_prefix="${2:-}"; shift 2 ;;
+    -h|--help) usage; exit 0 ;;
+    *) die "unknown option: $1 (see --help)" ;;
+  esac
+done
+
+[ -n "$principal" ] || die "--principal is required (see --help)."
+command -v az >/dev/null || die "the Azure CLI (az) is required."
+
+graph="https://graph.microsoft.com/v1.0"
+
+if [ -z "$api_client_id" ]; then
+  api_client_id="$(az ad app list --display-name "${name_prefix}-metrics-api" --query '[].appId' --output tsv)"
+  [ -n "$api_client_id" ] || die "no app registration named ${name_prefix}-metrics-api: run infra/setup-entra.sh first."
+  [ "$(printf '%s\n' "$api_client_id" | grep -c .)" -eq 1 ] || die "more than one app registration is named ${name_prefix}-metrics-api; pass --api-client-id."
+fi
+
+api_sp="$(az ad sp show --id "$api_client_id" --query id --output tsv)" ||
+  die "the API app $api_client_id has no service principal: run infra/setup-entra.sh."
+role_id="$(az ad sp show --id "$api_client_id" --query "appRoles[?value=='Events.Write'].id | [0]" --output tsv)"
+[ -n "$role_id" ] || die "the API app $api_client_id has no Events.Write role: run infra/setup-entra.sh."
+
+sender_sp="$(az ad sp show --id "$principal" --query id --output tsv)" ||
+  die "no service principal found for $principal. Use the service connection's application (client) id."
+sender_name="$(az ad sp show --id "$sender_sp" --query displayName --output tsv)"
+
+existing="$(az rest --method get --url "$graph/servicePrincipals/$sender_sp/appRoleAssignments" \
+  --query "length(value[?resourceId=='$api_sp' && appRoleId=='$role_id'])" --output tsv)"
+if [ "$existing" != "0" ]; then
+  echo "$sender_name ($sender_sp) already has Events.Write."
+  exit 0
+fi
+
+az rest --method post --url "$graph/servicePrincipals/$sender_sp/appRoleAssignments" \
+  --headers 'Content-Type=application/json' \
+  --body "{\"principalId\":\"$sender_sp\",\"resourceId\":\"$api_sp\",\"appRoleId\":\"$role_id\"}" \
+  --output none
+echo "Granted Events.Write to $sender_name ($sender_sp)."
+echo "Tokens issued from now on carry the role; a token fetched earlier does not, so rerun the pipeline."