From c2bb93019131b1ea847232b0e08101087e6c8138 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 1 Nov 2026 19:52:10 +0000 Subject: [PATCH 1/2] Correct the 1.0.0 note on the dashboard's engine version The engine version in metrics events is the templates repository's git tag at HEAD, or its short commit, not ModuleVersion. The end-to-end test (R3) showed the commit hash, so the 1.0.0 intro now says that. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D8A4w121dBr3nYoutNLaV7 Signed-off-by: Claude --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 424eee9..e281e3e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [1.0.0] - 2026-11-01 -First release as PullPicket. The engine's `ModuleVersion` restarts at `1.0.0` (it was `7.0.0` under the old name), so the dashboard's engine version table lists `1.0.0` below older events from `7.0.0`. +First release as PullPicket. The engine's `ModuleVersion` restarts at `1.0.0` (it was `7.0.0` under the old name). The dashboard's engine version table is unaffected: it shows the tag the pipeline checks the templates repository out at (such as `v1.0.0`), or the short commit when it points at a branch. ### Breaking defaults at a glance From 1ca5366c690244307146c21867866b5142522e9f Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 1 Nov 2026 19:52:10 +0000 Subject: [PATCH 2/2] Document the setup snags found in the end-to-end test (R3) - Quick start: a new project's default repository is named after the project; classic Azure OpenAI needs foundryTokenResource; adding a service principal needs the org connected to Entra; the automatic connection's Contributor role is not needed. - infra/README: register the resource providers first; setup-entra.sh may fall back to api://. - Troubleshooting: "Given email address is invalid" when adding the identity, and "repository could not be retrieved". Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01D8A4w121dBr3nYoutNLaV7 Signed-off-by: Claude --- README.md | 8 +++++++- docs/troubleshooting.md | 3 +++ infra/README.md | 7 ++++++- 3 files changed, 16 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 9a58f48..0d74602 100644 --- a/README.md +++ b/README.md @@ -66,9 +66,11 @@ You need an Azure DevOps project with an Azure Repos repository, an Azure subscr Import PullPicket into your Azure DevOps organisation (Repos > Import repository) or keep it on GitHub and reach it through a GitHub service connection. Pin a release tag, such as `v1.0.0`, so template changes only reach your pipelines when you choose. +A new project already has a repository named after the project. Import into a new repository named `pullpicket`, or, if you import into that default one, use its name in step 6 (`name: /`). + ### 2. Deploy a model -Create an Azure OpenAI or Azure AI Foundry resource and deploy a chat model. Any chat model that supports structured output (`response_format` with a JSON schema) works; PullPicket does not prescribe one. Note the deployment's chat completions URL (it ends in `/chat/completions?api-version=...`). +Create an Azure OpenAI or Azure AI Foundry resource and deploy a chat model. Any chat model that supports structured output (`response_format` with a JSON schema) works; PullPicket does not prescribe one. Note the deployment's chat completions URL (it ends in `/chat/completions?api-version=...`). For a classic Azure OpenAI resource (`https://.openai.azure.com/`), also set `foundryTokenResource: 'https://cognitiveservices.azure.com'` in step 6; the default suits Azure AI Foundry. ### 3. Create a service connection and grant it access @@ -77,6 +79,10 @@ Create an **Azure Resource Manager** service connection (workload identity feder - the **Cognitive Services OpenAI User** role on the model resource, and - membership in your Azure DevOps organisation, with permission to **Contribute to pull requests** on the repositories it reviews. +To add a service principal as an organisation user, the organisation must be connected to the identity's Microsoft Entra tenant (Organization settings > Microsoft Entra). Otherwise Azure DevOps treats the name as an email address and rejects it. + +The review needs no Azure management role. When Azure DevOps creates the connection automatically it grants **Contributor** on the subscription or resource group you pick; you can remove that assignment. + ### 4. Create the variable group Create a variable group (default name `ai-code-review`) under Pipelines > Library: diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index b63060b..82a85ae 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -38,6 +38,8 @@ Reading history and posting are best-effort, so the step can still finish green To find the identity, open the service connection (Project settings > Service connections), choose **Manage Service Principal** or **Manage App registration**, and note its display name and application (client) id. +If adding the identity fails with `Given email address '' is invalid`, the organisation is not connected to a Microsoft Entra tenant, so Azure DevOps cannot look up service principals. Connect it under Organization settings > Microsoft Entra to the tenant that holds the service connection, sign out and back in, and pick the identity from the search suggestions. + ## The model call fails with 401 or 403 ``` @@ -95,6 +97,7 @@ To compare, run the same pipeline once on a Microsoft-hosted pool (`pool: vmImag | Log message | Cause | Fix | | --- | --- | --- | +| `The repository pullpicket in project could not be retrieved` | The `name:` of the `templates` repository resource does not match the imported repository. A new project's default repository is named after the project. | Set `name: /` to the repository you imported into. | | `promptMode is 'rules' but the base prompt file was not found` | The PullPicket repository was not checked out as `templates`. | Name the repository resource `templates` (see the quick start). | | `No model API key was provided and AuthMode is not entra` | `foundryAuth: key` without a `foundry-key` value. | Add `foundry-key` as a secret variable in the model variable group, or use `foundryAuth: entra`. | | `... was truncated (finish_reason: length)` | The model ran out of output tokens on a large batch. | Lower `reviewMaxFilesPerBatch` or `reviewMaxBatchTokens`, or lower `reasoningEffort`. | diff --git a/infra/README.md b/infra/README.md index 1d39988..6f2f933 100644 --- a/infra/README.md +++ b/infra/README.md @@ -20,6 +20,11 @@ Bicep and two `az` scripts that deploy the PullPicket metrics API and dashboard ## Before you start - Azure CLI 2.60 or later, signed in (`az login`), plus `jq` and `bash`. Azure Cloud Shell has all three. +- The resource providers registered on the subscription. A new subscription often has none of them yet, and the deployment then fails with "not registered": + + ```bash + for p in Microsoft.App Microsoft.OperationalInsights Microsoft.DocumentDB Microsoft.ContainerRegistry Microsoft.ManagedIdentity; do az provider register --namespace $p; done + ``` - On the resource group: **Owner**, or **Contributor** plus **Role Based Access Control Administrator**, because the deployment creates role assignments. - In Entra: rights to create app registrations and assign users to them (the default for members in most tenants, or **Application Administrator**). - Docker is optional: `az acr build` builds the image in Azure. @@ -76,7 +81,7 @@ infra/grant-events-write.sh --principal | Variable | Value | | --- | --- | | `aiMetricsApiUrl` | `metricsUrl` | -| `aiMetricsApiResource` | the API's application ID URI (`PULLPICKET_API_APP_ID_URI` in `.env.entra`, such as `api://pullpicket-metrics`) | +| `aiMetricsApiResource` | the API's application ID URI (`PULLPICKET_API_APP_ID_URI` in `.env.entra`). It is `api://pullpicket-metrics` unless the tenant only allows URIs on a verified domain; `setup-entra.sh` then uses `api://` and says so. | **7. Check it.** Open `/` in a browser: you should be sent to the Microsoft sign-in page, then see the landing page. Without a credential, `POST /events` answers `401`: