From 0622b9854aca46016bceca9f3704c7db811009f8 Mon Sep 17 00:00:00 2001 From: Paulo Lacerda Date: Sat, 10 Oct 2026 13:13:47 -0300 Subject: [PATCH 1/2] docs: record verified custom hosted private acceptance Close T098 with actual version-7 private runtime, exact capability grants and valid-token authorization rejection. Keep fresh S7 and conversational acceptance explicitly incomplete. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .../quickstart.md | 62 ++++++++++++++++++- specs/002-agent-landing-zone-rebrand/tasks.md | 8 +-- 2 files changed, 63 insertions(+), 7 deletions(-) diff --git a/specs/002-agent-landing-zone-rebrand/quickstart.md b/specs/002-agent-landing-zone-rebrand/quickstart.md index e48cb48a..3acd2015 100644 --- a/specs/002-agent-landing-zone-rebrand/quickstart.md +++ b/specs/002-agent-landing-zone-rebrand/quickstart.md @@ -30,6 +30,61 @@ environment per scenario and delete it afterwards with `azd down --purge`. ## Acceptance by release +### Integrated-source and private-hosted evidence (2026-10-10) + +The acceptance fixes are integrated into `develop`: platform #774 at +`dd4c8978db94ff56d4dbbdb30c8c9168625b75a3`, +Azure/agent-app-ui#144 at `ec18a7330982b188d1c1e6da9157230a20865242`, +and Azure/AI-Landing-Zones#149 at +`1f7c1d707abe99468c6c42f4a26c39f0443b76a8`. No repository rules were changed. +The first UI merge-push check still selected the old protected evaluator from +`github.event.before`; it is not evidence that the newly adopted gate passed. +A separate reference PR against the new base is being validated. + +T098's custom hosted isolation and RBAC criteria are now verified: + +- Repeated runtime checks confirmed hosted version 7 is active, uses the + previously recorded immutable image digest, resolves inside its private + spoke, and returns the exact project greeting through the Responses endpoint. + The Container App sample also returned its exact live platform contract. + These deployments retain corrective source + `0ab4665db29fbfac6bc5a41a372c0b64df32b73e`. +- All 13 private endpoints have successful provisioning and approved + connections. Foundry, registry, App Configuration and the canonical vault + have public network access disabled. Foundry local authentication and the + registry admin account are disabled; the canonical vault uses RBAC. + App Configuration local authentication remains enabled as declared by the + foundation template; this audit does not claim otherwise. +- The actual version-7 runtime principal has exactly five grants, including + inherited assignments: App Configuration Data Reader on its store, AcrPull + on its registry, Key Vault Secrets User on the canonical vault, and + Cognitive Services User plus Cognitive Services OpenAI User on its Foundry + account. No additional grant or broader scope was found. +- A dedicated unassigned principal obtained a valid Foundry resource token. + Its request to the actual private custom Responses endpoint was rejected + with structured HTTP 403 identifying an authorization failure. This is + not an expired-token or anonymous-request test. + +Local receipts are retained under `.azure/` as +`s7-custom-runtime-check-20261010.json`, +`custom-hosted-control-audit.json`, and +`custom-hosted-rbac-negative-20261010.json`. They contain no credential values. +The integrated contract suite passed 118 tests and 74 subtests. + +The private trio's non-placeholder UI, Entra provider and login redirect, +ingestion health, and anonymous panel rejection were rechecked successfully. +The evaluation OAuth credential was renewed through encrypted transport into +Key Vault with a matching one-day expiry. None of these checks proves user +login, OBO, grounded answers, citations or document-level authorization. + +T035, T085 and T093 remain incomplete. A legitimate delegated-token request +returned `AADSTS65001` (`consent_required`, `InteractionRequired`). +The supported VPN profile import was invoked, but automatic Windows DNS still +resolved the frontend publicly; application of the private DNS rules is not +verified. Fresh stock one-command sample deployments and the complete +classic/hosted conversational acceptance must not be inferred from the +successful private custom deployment, control-plane or health checks. + ### Latest corrective-source evidence (2026-10-09) Both private custom-sample root `azd deploy` executions passed at exact source @@ -56,9 +111,10 @@ Independent runtime probes then verified: These results do **not** establish source provenance solely from the deployed digest, a fresh stock one-command `azd up`, delegated-user OBO, grounded answers, -citations or positive/negative document authorization. T035, T085, T098 and T093 -remain open until their complete scenario criteria are exercised; successful -custom samples must not be substituted for the bundled conversational stack. +citations or positive/negative document authorization. At that checkpoint, +T035, T085, T098 and T093 remained open. The later T098 audit above supersedes +its isolation/RBAC status; successful custom samples must not be substituted +for the bundled conversational stack. | Release | Scenarios that must pass | | --- | --- | diff --git a/specs/002-agent-landing-zone-rebrand/tasks.md b/specs/002-agent-landing-zone-rebrand/tasks.md index bc8c5070..fd15d4fc 100644 --- a/specs/002-agent-landing-zone-rebrand/tasks.md +++ b/specs/002-agent-landing-zone-rebrand/tasks.md @@ -91,7 +91,7 @@ hosted orchestrator modes, and sees only the Agent Landing Zone name. - [X] T033 [US1] Pin the preview component tags from T021 in manifest.json `components[]` - [X] T034 [US1] Shrink the temporary allow-list to the entries still needed for the preview (old names kept by R14 dual-read in components, `infra/` submodule) in contracts/naming-map.md - [ ] T035 [US1] Validate S1 steps 1, 2, and 4 end to end in a fresh validation environment, in classic mode and hosted mode; record evidence (no private env names) in specs/002-agent-landing-zone-rebrand/quickstart.md acceptance table - - Status: blocked: MCAPS policy (Key Vault public access 403, regional quota) prevents a clean full S1 run in the maintainer subscription; S2 path validated + - Status: partial (2026-10-10): private UI/ingestion and authentication-boundary probes passed with renewed Key Vault credential. Legitimate delegated-token acquisition returns AADSTS65001 (consent_required / InteractionRequired); user OBO, grounded answers, citations and document authorization remain unverified. Classic recovery requires a separate VNet-integrated Container Apps Environment. No public-access or authentication bypass is permitted. **Checkpoint**: US1 passes S1 at the preview. SC-008 surfaces (README top, UI, `azd` template, App Configuration) are clean. @@ -213,8 +213,8 @@ modes). Fallback per R16 if FR-015a–h does not land by 2026-10-09. - Status: deferred: central docs repo (external) - [ ] T085 [US5] Validate S5 (invalid and changed definitions rejected before any Azure change) and S7 (sample deploys with one `azd up` in both modes) and record evidence in specs/002-agent-landing-zone-rebrand/quickstart.md - Status: partial: S5 schema/binding negatives passed locally and on the private runner without modifying the bound environment. Both sample definitions validate; focused schema/roles/binding/composition/hooks suite passed 43 tests and 28 subtests. S7 still requires fresh one-command deploys in both modes; current default new Standard embedding allocation requests 100 with only 50 regional headroom. Existing allocations cannot be credited to a new account. -- [ ] T098 [US5] Deploy the hosted mode with `NETWORK_ISOLATION=true` and verify private endpoints and RBAC for the hosted-agent component (FR-015c); record evidence in specs/002-agent-landing-zone-rebrand/quickstart.md - - Status: partial: bundled hosted component deployed on corrective source with network isolation; managed greeting passed, private endpoints are Approved/Succeeded, public access remains disabled, exact least-privilege runtime grants verified, and a valid token from an unassigned principal was rejected by the actual private hosted endpoint with structured HTTP 403. Custom hosted sample parity (FR-015c) remains unverified; these checks do not prove user OBO or document authorization. +- [X] T098 [US5] Deploy the hosted mode with `NETWORK_ISOLATION=true` and verify private endpoints and RBAC for the hosted-agent component (FR-015c); record evidence in specs/002-agent-landing-zone-rebrand/quickstart.md + - Status: verified (2026-10-10): actual custom hosted version 7 is active on corrective source 0ab4665 with private DNS and exact project greeting. All 13 endpoints are Succeeded/Approved; public access is disabled on runtime dependencies. Current runtime principal has exactly the five declared profile grants at canonical resource scopes, including inherited assignments. A valid token from an unassigned principal was rejected by the actual private custom Responses endpoint with structured HTTP 403. See quickstart.md for authentication-control details and receipts. This closes isolation/RBAC, not T035/T085/T093 or user OBO/document authorization. - [X] T099 [P] [US5] Compare the role assignments generated by the profiles (`base`, `model-user`, `retrieval-reader`, `conversation-store`, `blob-delegator`, `ingestion-writer`) with the trio's current role assignments and record a parity checklist (FR-015e, SC-006) in tests/test_app_definition_schema.py and specs/002-agent-landing-zone-rebrand/quickstart.md - [X] T086 [US5] If FR-015a–h misses 2026-10-09, publish the R16 manual procedure instead and mark FR-015 as fallback in specs/002-agent-landing-zone-rebrand/spec.md and the central docs page - Status: N/A. FR-015 (app definition) shipped in v4.0.0, so the fallback was not needed. @@ -237,7 +237,7 @@ modes). Fallback per R16 if FR-015a–h does not land by 2026-10-09. - [X] T091 [P] Add the `## [v4.0.0] - YYYY-MM-DD` entry (breaking rename, new deployments only, preview redeploy, component table from manifest.json) in CHANGELOG.md - [X] T092 Run the full test suite (`pytest tests`) and both hook variants for every changed script - [ ] T093 Run quickstart.md S1–S7 on a fresh validation environment and confirm SC-002, SC-005, SC-006 (no functional regression), and SC-009 - - Status: GA: run before T094 + - Status: partial (2026-10-10): integrated contracts passed 118 tests and 74 subtests; T098 passed independently. Fresh one-command S7 and classic/hosted S1 user/grounding/document-authorization outcomes are still required. Full S1-S7 acceptance is not complete. - [X] T094 After Paulo's approval, tag and publish `v4.0.0` with title `v4.0.0`, release notes with `## Changed`, `## Component versions` (from manifest.json `components[]` and `infra.source`), `## Validation`, the FR-001 transition statement ("GPT-RAG is now Agent Landing Zone"), the FR-004 note, a list of any items still pending on 2026-10-09 (FR-020), and no `gptrag-\d{10}` tokens - Status: gated: needs Paulo's approval - [X] T095 Confirm SC-007 dates (preview by 2026-10-02, GA by 2026-10-09) and close Azure/GPT-RAG#695 with a link to the release From 91d3a7083a48891a5fac50f914870c683dc7e54d Mon Sep 17 00:00:00 2001 From: Paulo Lacerda Date: Sat, 10 Oct 2026 13:17:57 -0300 Subject: [PATCH 2/2] docs: record UI reference gate acceptance Record merged reference PR and successful CI against the new protected develop base. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- specs/002-agent-landing-zone-rebrand/quickstart.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/specs/002-agent-landing-zone-rebrand/quickstart.md b/specs/002-agent-landing-zone-rebrand/quickstart.md index 3acd2015..b0eb3b8c 100644 --- a/specs/002-agent-landing-zone-rebrand/quickstart.md +++ b/specs/002-agent-landing-zone-rebrand/quickstart.md @@ -39,7 +39,12 @@ and Azure/AI-Landing-Zones#149 at `1f7c1d707abe99468c6c42f4a26c39f0443b76a8`. No repository rules were changed. The first UI merge-push check still selected the old protected evaluator from `github.event.before`; it is not evidence that the newly adopted gate passed. -A separate reference PR against the new base is being validated. +The reference Azure/agent-app-ui#145 subsequently passed all nine checks +against the new protected `develop` base +`ec18a7330982b188d1c1e6da9157230a20865242` in CI run `38066592411` +at head `d41aa8da85754a19321f5d5004831754f6d59683` and merged at +`e93b3d3e0cf36d8e92e8f9d0c6d1c911aa0f715f`. This supplies positive +new-base gate evidence; the earlier merge-push check does not. T098's custom hosted isolation and RBAC criteria are now verified: