Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [1.0.0] - 2026-11-01

First release as PullPicket. The engine's `ModuleVersion` restarts at `1.0.0` (it was `7.0.0` under the old name), so the dashboard's engine version table lists `1.0.0` below older events from `7.0.0`.
First release as PullPicket. The engine's `ModuleVersion` restarts at `1.0.0` (it was `7.0.0` under the old name). The dashboard's engine version table is unaffected: it shows the tag the pipeline checks the templates repository out at (such as `v1.0.0`), or the short commit when it points at a branch.

### Breaking defaults at a glance

Expand Down
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,11 @@ You need an Azure DevOps project with an Azure Repos repository, an Azure subscr

Import PullPicket into your Azure DevOps organisation (Repos > Import repository) or keep it on GitHub and reach it through a GitHub service connection. Pin a release tag, such as `v1.0.0`, so template changes only reach your pipelines when you choose.

A new project already has a repository named after the project. Import into a new repository named `pullpicket`, or, if you import into that default one, use its name in step 6 (`name: <project>/<project>`).

### 2. Deploy a model

Create an Azure OpenAI or Azure AI Foundry resource and deploy a chat model. Any chat model that supports structured output (`response_format` with a JSON schema) works; PullPicket does not prescribe one. Note the deployment's chat completions URL (it ends in `/chat/completions?api-version=...`).
Create an Azure OpenAI or Azure AI Foundry resource and deploy a chat model. Any chat model that supports structured output (`response_format` with a JSON schema) works; PullPicket does not prescribe one. Note the deployment's chat completions URL (it ends in `/chat/completions?api-version=...`). For a classic Azure OpenAI resource (`https://<resource>.openai.azure.com/`), also set `foundryTokenResource: 'https://cognitiveservices.azure.com'` in step 6; the default suits Azure AI Foundry.

### 3. Create a service connection and grant it access

Expand All @@ -77,6 +79,10 @@ Create an **Azure Resource Manager** service connection (workload identity feder
- the **Cognitive Services OpenAI User** role on the model resource, and
- membership in your Azure DevOps organisation, with permission to **Contribute to pull requests** on the repositories it reviews.

To add a service principal as an organisation user, the organisation must be connected to the identity's Microsoft Entra tenant (Organization settings > Microsoft Entra). Otherwise Azure DevOps treats the name as an email address and rejects it.

The review needs no Azure management role. When Azure DevOps creates the connection automatically it grants **Contributor** on the subscription or resource group you pick; you can remove that assignment.

### 4. Create the variable group

Create a variable group (default name `ai-code-review`) under Pipelines > Library:
Expand Down
3 changes: 3 additions & 0 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ Reading history and posting are best-effort, so the step can still finish green

To find the identity, open the service connection (Project settings > Service connections), choose **Manage Service Principal** or **Manage App registration**, and note its display name and application (client) id.

If adding the identity fails with `Given email address '<name>' is invalid`, the organisation is not connected to a Microsoft Entra tenant, so Azure DevOps cannot look up service principals. Connect it under Organization settings > Microsoft Entra to the tenant that holds the service connection, sign out and back in, and pick the identity from the search suggestions.

## The model call fails with 401 or 403

```
Expand Down Expand Up @@ -95,6 +97,7 @@ To compare, run the same pipeline once on a Microsoft-hosted pool (`pool: vmImag

| Log message | Cause | Fix |
| --- | --- | --- |
| `The repository pullpicket in project <project> could not be retrieved` | The `name:` of the `templates` repository resource does not match the imported repository. A new project's default repository is named after the project. | Set `name: <project>/<repository>` to the repository you imported into. |
| `promptMode is 'rules' but the base prompt file was not found` | The PullPicket repository was not checked out as `templates`. | Name the repository resource `templates` (see the quick start). |
| `No model API key was provided and AuthMode is not entra` | `foundryAuth: key` without a `foundry-key` value. | Add `foundry-key` as a secret variable in the model variable group, or use `foundryAuth: entra`. |
| `... was truncated (finish_reason: length)` | The model ran out of output tokens on a large batch. | Lower `reviewMaxFilesPerBatch` or `reviewMaxBatchTokens`, or lower `reasoningEffort`. |
Expand Down
7 changes: 6 additions & 1 deletion infra/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@ Bicep and two `az` scripts that deploy the PullPicket metrics API and dashboard
## Before you start

- Azure CLI 2.60 or later, signed in (`az login`), plus `jq` and `bash`. Azure Cloud Shell has all three.
- The resource providers registered on the subscription. A new subscription often has none of them yet, and the deployment then fails with "not registered":

```bash
for p in Microsoft.App Microsoft.OperationalInsights Microsoft.DocumentDB Microsoft.ContainerRegistry Microsoft.ManagedIdentity; do az provider register --namespace $p; done
```
- On the resource group: **Owner**, or **Contributor** plus **Role Based Access Control Administrator**, because the deployment creates role assignments.
- In Entra: rights to create app registrations and assign users to them (the default for members in most tenants, or **Application Administrator**).
- Docker is optional: `az acr build` builds the image in Azure.
Expand Down Expand Up @@ -76,7 +81,7 @@ infra/grant-events-write.sh --principal <service connection client id>
| Variable | Value |
| --- | --- |
| `aiMetricsApiUrl` | `metricsUrl` |
| `aiMetricsApiResource` | the API's application ID URI (`PULLPICKET_API_APP_ID_URI` in `.env.entra`, such as `api://pullpicket-metrics`) |
| `aiMetricsApiResource` | the API's application ID URI (`PULLPICKET_API_APP_ID_URI` in `.env.entra`). It is `api://pullpicket-metrics` unless the tenant only allows URIs on a verified domain; `setup-entra.sh` then uses `api://<API client id>` and says so. |

**7. Check it.** Open `<metricsUrl>/` in a browser: you should be sent to the Microsoft sign-in page, then see the landing page. Without a credential, `POST /events` answers `401`:

Expand Down
Loading