Skip to content

docs: record verified private hosted acceptance evidence #775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Paulo Lacerda (placerda) merged 2 commits into from
Oct 10, 2026
Merged

docs: record verified private hosted acceptance evidence #775

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0622b98
docs: record verified custom hosted private acceptance
placerda Oct 10, 2026
91d3a70
docs: record UI reference gate acceptance
placerda Oct 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 64 additions & 3 deletions specs/002-agent-landing-zone-rebrand/quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,66 @@ environment per scenario and delete it afterwards with `azd down --purge`.

## Acceptance by release

### Integrated-source and private-hosted evidence (2026-10-10)

The acceptance fixes are integrated into `develop`: platform #774 at
`dd4c8978db94ff56d4dbbdb30c8c9168625b75a3`,
Azure/agent-app-ui#144 at `ec18a7330982b188d1c1e6da9157230a20865242`,
and Azure/AI-Landing-Zones#149 at
`1f7c1d707abe99468c6c42f4a26c39f0443b76a8`. No repository rules were changed.
The first UI merge-push check still selected the old protected evaluator from
`github.event.before`; it is not evidence that the newly adopted gate passed.
The reference Azure/agent-app-ui#145 subsequently passed all nine checks
against the new protected `develop` base
`ec18a7330982b188d1c1e6da9157230a20865242` in CI run `38066592411`
at head `d41aa8da85754a19321f5d5004831754f6d59683` and merged at
`e93b3d3e0cf36d8e92e8f9d0c6d1c911aa0f715f`. This supplies positive
new-base gate evidence; the earlier merge-push check does not.

T098's custom hosted isolation and RBAC criteria are now verified:

- Repeated runtime checks confirmed hosted version 7 is active, uses the
previously recorded immutable image digest, resolves inside its private
spoke, and returns the exact project greeting through the Responses endpoint.
The Container App sample also returned its exact live platform contract.
These deployments retain corrective source
`0ab4665db29fbfac6bc5a41a372c0b64df32b73e`.
- All 13 private endpoints have successful provisioning and approved
connections. Foundry, registry, App Configuration and the canonical vault
have public network access disabled. Foundry local authentication and the
registry admin account are disabled; the canonical vault uses RBAC.
App Configuration local authentication remains enabled as declared by the
foundation template; this audit does not claim otherwise.
- The actual version-7 runtime principal has exactly five grants, including
inherited assignments: App Configuration Data Reader on its store, AcrPull
on its registry, Key Vault Secrets User on the canonical vault, and
Cognitive Services User plus Cognitive Services OpenAI User on its Foundry
account. No additional grant or broader scope was found.
- A dedicated unassigned principal obtained a valid Foundry resource token.
Its request to the actual private custom Responses endpoint was rejected
with structured HTTP 403 identifying an authorization failure. This is
not an expired-token or anonymous-request test.

Local receipts are retained under `.azure/` as
`s7-custom-runtime-check-20261010.json`,
`custom-hosted-control-audit.json`, and
`custom-hosted-rbac-negative-20261010.json`. They contain no credential values.
The integrated contract suite passed 118 tests and 74 subtests.

The private trio's non-placeholder UI, Entra provider and login redirect,
ingestion health, and anonymous panel rejection were rechecked successfully.
The evaluation OAuth credential was renewed through encrypted transport into
Key Vault with a matching one-day expiry. None of these checks proves user
login, OBO, grounded answers, citations or document-level authorization.

T035, T085 and T093 remain incomplete. A legitimate delegated-token request
returned `AADSTS65001` (`consent_required`, `InteractionRequired`).
The supported VPN profile import was invoked, but automatic Windows DNS still
resolved the frontend publicly; application of the private DNS rules is not
verified. Fresh stock one-command sample deployments and the complete
classic/hosted conversational acceptance must not be inferred from the
successful private custom deployment, control-plane or health checks.

### Latest corrective-source evidence (2026-10-09)

Both private custom-sample root `azd deploy` executions passed at exact source
Expand All @@ -56,9 +116,10 @@ Independent runtime probes then verified:

These results do **not** establish source provenance solely from the deployed
digest, a fresh stock one-command `azd up`, delegated-user OBO, grounded answers,
citations or positive/negative document authorization. T035, T085, T098 and T093
remain open until their complete scenario criteria are exercised; successful
custom samples must not be substituted for the bundled conversational stack.
citations or positive/negative document authorization. At that checkpoint,
T035, T085, T098 and T093 remained open. The later T098 audit above supersedes
its isolation/RBAC status; successful custom samples must not be substituted
for the bundled conversational stack.

| Release | Scenarios that must pass |
| --- | --- |
Expand Down
8 changes: 4 additions & 4 deletions specs/002-agent-landing-zone-rebrand/tasks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ hosted orchestrator modes, and sees only the Agent Landing Zone name.
- [X] T033 [US1] Pin the preview component tags from T021 in manifest.json `components[]`
- [X] T034 [US1] Shrink the temporary allow-list to the entries still needed for the preview (old names kept by R14 dual-read in components, `infra/` submodule) in contracts/naming-map.md
- [ ] T035 [US1] Validate S1 steps 1, 2, and 4 end to end in a fresh validation environment, in classic mode and hosted mode; record evidence (no private env names) in specs/002-agent-landing-zone-rebrand/quickstart.md acceptance table
- Status: blocked: MCAPS policy (Key Vault public access 403, regional quota) prevents a clean full S1 run in the maintainer subscription; S2 path validated
- Status: partial (2026-10-10): private UI/ingestion and authentication-boundary probes passed with renewed Key Vault credential. Legitimate delegated-token acquisition returns AADSTS65001 (consent_required / InteractionRequired); user OBO, grounded answers, citations and document authorization remain unverified. Classic recovery requires a separate VNet-integrated Container Apps Environment. No public-access or authentication bypass is permitted.

**Checkpoint**: US1 passes S1 at the preview. SC-008 surfaces (README top, UI, `azd` template, App Configuration) are clean.

Expand Down Expand Up @@ -213,8 +213,8 @@ modes). Fallback per R16 if FR-015a–h does not land by 2026-10-09.
- Status: deferred: central docs repo (external)
- [ ] T085 [US5] Validate S5 (invalid and changed definitions rejected before any Azure change) and S7 (sample deploys with one `azd up` in both modes) and record evidence in specs/002-agent-landing-zone-rebrand/quickstart.md
- Status: partial: S5 schema/binding negatives passed locally and on the private runner without modifying the bound environment. Both sample definitions validate; focused schema/roles/binding/composition/hooks suite passed 43 tests and 28 subtests. S7 still requires fresh one-command deploys in both modes; current default new Standard embedding allocation requests 100 with only 50 regional headroom. Existing allocations cannot be credited to a new account.
- [ ] T098 [US5] Deploy the hosted mode with `NETWORK_ISOLATION=true` and verify private endpoints and RBAC for the hosted-agent component (FR-015c); record evidence in specs/002-agent-landing-zone-rebrand/quickstart.md
- Status: partial: bundled hosted component deployed on corrective source with network isolation; managed greeting passed, private endpoints are Approved/Succeeded, public access remains disabled, exact least-privilege runtime grants verified, and a valid token from an unassigned principal was rejected by the actual private hosted endpoint with structured HTTP 403. Custom hosted sample parity (FR-015c) remains unverified; these checks do not prove user OBO or document authorization.
- [X] T098 [US5] Deploy the hosted mode with `NETWORK_ISOLATION=true` and verify private endpoints and RBAC for the hosted-agent component (FR-015c); record evidence in specs/002-agent-landing-zone-rebrand/quickstart.md
- Status: verified (2026-10-10): actual custom hosted version 7 is active on corrective source 0ab4665 with private DNS and exact project greeting. All 13 endpoints are Succeeded/Approved; public access is disabled on runtime dependencies. Current runtime principal has exactly the five declared profile grants at canonical resource scopes, including inherited assignments. A valid token from an unassigned principal was rejected by the actual private custom Responses endpoint with structured HTTP 403. See quickstart.md for authentication-control details and receipts. This closes isolation/RBAC, not T035/T085/T093 or user OBO/document authorization.
- [X] T099 [P] [US5] Compare the role assignments generated by the profiles (`base`, `model-user`, `retrieval-reader`, `conversation-store`, `blob-delegator`, `ingestion-writer`) with the trio's current role assignments and record a parity checklist (FR-015e, SC-006) in tests/test_app_definition_schema.py and specs/002-agent-landing-zone-rebrand/quickstart.md
- [X] T086 [US5] If FR-015a–h misses 2026-10-09, publish the R16 manual procedure instead and mark FR-015 as fallback in specs/002-agent-landing-zone-rebrand/spec.md and the central docs page
- Status: N/A. FR-015 (app definition) shipped in v4.0.0, so the fallback was not needed.
Expand All @@ -237,7 +237,7 @@ modes). Fallback per R16 if FR-015a–h does not land by 2026-10-09.
- [X] T091 [P] Add the `## [v4.0.0] - YYYY-MM-DD` entry (breaking rename, new deployments only, preview redeploy, component table from manifest.json) in CHANGELOG.md
- [X] T092 Run the full test suite (`pytest tests`) and both hook variants for every changed script
- [ ] T093 Run quickstart.md S1–S7 on a fresh validation environment and confirm SC-002, SC-005, SC-006 (no functional regression), and SC-009
- Status: GA: run before T094
- Status: partial (2026-10-10): integrated contracts passed 118 tests and 74 subtests; T098 passed independently. Fresh one-command S7 and classic/hosted S1 user/grounding/document-authorization outcomes are still required. Full S1-S7 acceptance is not complete.
- [X] T094 After Paulo's approval, tag and publish `v4.0.0` with title `v4.0.0`, release notes with `## Changed`, `## Component versions` (from manifest.json `components[]` and `infra.source`), `## Validation`, the FR-001 transition statement ("GPT-RAG is now Agent Landing Zone"), the FR-004 note, a list of any items still pending on 2026-10-09 (FR-020), and no `gptrag-\d{10}` tokens
- Status: gated: needs Paulo's approval
- [X] T095 Confirm SC-007 dates (preview by 2026-10-02, GA by 2026-10-09) and close Azure/GPT-RAG#695 with a link to the release
Expand Down