docs(security): entitlement + API-key scopes designs, + concurrent secrets/env design (#8)

GitHub org→repo entitlement + API-key scopes designs (design-only, 8/8 cross-checks hold); the concurrent secrets/environments design + .sisyphus plan; and a manual zai-cli-smoke.sh hardening. See docs/design/security-designs/decisions-for-kevin.md.

Author: kivo360

.sisyphus/plans/secrets-and-environments-scoping-plan.md

@@ -0,0 +1,166 @@
+# Secrets & Environments Scoping — Design & Plan
+
+> Status: **Proposed** (no code changed yet). Authored 2026-06-02.
+> Authoritative target model: `docs/design/background-agent-platform/04-secrets-byok-litellm/secrets-management.md`,
+> `06-environments-and-docker/*`, `12-data-models/conceptual-schema.md`.
+> Executable task tracker: `.sisyphus/plans/secrets-and-environments-scoping-plan.md`.
+> Deep schema reference: `./target-architecture.md`.
+
+This document is the front door for closing the gaps in **environment** and
+**secrets** management across the full scope hierarchy — **org → team → user →
+repo → environment → run**. It also covers **activating Teams in Better Auth**,
+which is a prerequisite for half of the missing secret scopes.
+
+---
+
+## 1. Why this exists
+
+The platform today has the *coarse* half of the secret hierarchy (org + repo)
+with strong per-tenant crypto, but is missing the *fine* half (team, user,
+environment, run), any value-versioning / rotation history, any per-secret
+authorization, and the entire **environment** subsystem that an
+"environment-scoped secret" would even attach to. RBAC roles are defined but
+not wired in. Teams are disabled.
+
+The design pack (`docs/design/background-agent-platform/`) already specifies the
+target. This plan makes the codebase match it, in a safe, phased, zero-downtime
+order that **coordinates with the in-flight `multi-tenant-backend-confidence-plan`**
+(another agent is actively editing the same files — see §6).
+
+---
+
+## 2. Current state (verified against code)
+
+| Area | What exists | Evidence |
+|---|---|---|
+| Tenancy root | `organization` + `member` (single `role` text) + `invitation` | `src/db/auth-schema.ts:137-191` |
+| Teams | **Disabled** | `src/lib/auth.ts:40` → `teams: { enabled: false }` |
+| RBAC roles | Defined (`ac`/`owner`/`admin`/`member`) but **never wired** into the org plugin | `src/lib/permissions.ts:1-42`; absent from `auth.ts:39-58` |
+| Context resolution | `{ userId, organizationId }` only; fail-closed on bad active org; auto-creates personal workspace | `src/lib/resolve-context.ts:5-65` |
+| Secrets storage | One table `tenant_secret`: org-level (`excludedRepos` blocklist) + repo-scoped (override) | `src/db/app-schema.ts:51-92` |
+| Secret resolution | `getEffectiveForRepo` = `(org − exclusions) + repo`, repo wins | `src/lib/secret-repo.ts:344-425` |
+| Secret crypto | Per-org AES-GCM-256, HKDF-derived key, IV+tag separate, `ENCRYPTION_KEY_PREVIOUS` rotation | `src/lib/crypto.ts`, `app-schema.ts:60-63` |
+| Secret authz | `requireAuth` only — **any** org member / valid org key can read-metadata, write, delete every org secret; no role gate | `src/index.ts:819-821` |
+| Secret audit | create/update/delete now wired | `src/index.ts:898` |
+| Environments | **None** — no entity, no autodiscovery; "environment" = repo/branch/model on the DO `session` row | grep empty |
+| BYOK delivery | Raw provider keys injected into sandbox env; no gateway/virtual key | `src/do/session-do.ts:1496,1521` |
+| Migrations | `wrangler d1 migrations apply` over `drizzle/*.sql`; **`_journal.json` is out of sync** (lists 0000-0003,0007; files have 0004-0008) | `drizzle/meta/_journal.json`, `drizzle/` |
+
+---
+
+## 3. The scope matrix (the core gap)
+
+Design pack secret scopes: `org, team, repo, environment, run, user`
+(`secrets-management.md:34-41`).
+
+| Scope | Designed | Built | Blocked on |
+|---|:---:|:---:|---|
+| **org** | ✓ | ✓ | — |
+| **repo** | ✓ | ✓ | — (`excludedRepos` + override) |
+| **team** | ✓ | ✗ | Teams not enabled → no team entity |
+| **user** | ✓ | ✗ | Just unmodeled (`createdByUserId` is provenance, not a scope) |
+| **environment** | ✓ | ✗ | No `environment_profiles` entity to bind to |
+| **run** | ✓ | ✗ | No durable run / run-token object |
+
+**4 of 6 scopes missing, and they are the four most precise ones.** The
+missing scopes are blocked by missing *entities*, not just storage columns — so
+this is an architecture change, not a column add.
+
+---
+
+## 4. Target architecture (summary)
+
+Full schemas, precedence rules, and RBAC model in **`./target-architecture.md`**.
+Headlines:
+
+1. **Teams (Better Auth)** — flip `teams.enabled`, regenerate `auth-schema.ts`
+   (`team`, `teamMember` tables + `invitation.teamId`), wire the dead
+   `permissions.ts` roles into the org plugin, extend `AppContext` with an
+   optional `teamId`.
+2. **Generalized secret scope tuple** — evolve `tenant_secret` from two bespoke
+   columns into `(scope_type, scope_id)` covering all six scopes, with one
+   documented precedence chain replacing the two-layer merge. Absorbs the
+   existing `repoOwner/repoName/excludedRepos` model losslessly.
+3. **Secret governance** — add `secret_versions` (value rotation history, distinct
+   from the crypto `key_version`) and `secret_access_policies` (per-secret RBAC);
+   add `secret` + `environment` as RBAC resources; split `read_metadata` vs
+   `use_runtime`.
+4. **Environment entity** — add `environment_profiles` (+`_versions`, +`_issues`)
+   per the pack, giving env-scoped secrets something to bind to and unblocking the
+   `production secrets: false` style key restriction. Autodiscovery
+   (Detective/Builder/Verifier) is a documented follow-on.
+5. **Run scope + BYOK gateway** — run-scoped, ephemeral secrets and a LiteLLM
+   gateway / virtual-key layer are **cross-dependencies** on the run-token work
+   already tracked in the architecture review's slices; this plan defines the
+   schema seam and defers the gateway itself.
+
+---
+
+## 5. Phased roadmap (execution order)
+
+Ordered so every phase leaves the app working and demoable, security/identity
+foundations first, the risky shared-table migration last (to clear the active
+agent's in-flight work).
+
+| Phase | Title | Depends on | Collision risk |
+|---|---|---|---|
+| **0** | Migration hygiene + coordination | — | reconcile `_journal.json`; agree migration ownership |
+| **1** | **Activate Teams in Better Auth** | 0 | low (auth-schema + auth.ts) |
+| **2** | Wire RBAC + `secret`/`environment` resources; gate routes by role | 1 | low |
+| **3** | Environment entity (`environment_profiles` + API + SDK + page) | 0 | none (new tables) |
+| **4** | Generalized secret scope model (team/user/env scopes, versions, policies, precedence, injection) | 1,2,3 + **active plan landed** | **HIGH** — shares `tenant_secret`/`secret-repo.ts`/`session-do.ts` |
+| **5** | Run-scoped secrets + BYOK gateway seam | 4 + run-token work | coordinate |
+| **6** | Observability/audit completeness + UI surfaces | 1-5 | low |
+
+Phase 4 is the keystone and is intentionally sequenced **after** the
+`multi-tenant-backend-confidence-plan` lands its repo-scoped secret work, to
+avoid two agents migrating the same table.
+
+---
+
+## 6. Coordination with the active agent (READ FIRST)
+
+`boulder.json` shows agent `atlas` executing
+`.sisyphus/plans/multi-tenant-backend-confidence-plan.md`, which is **actively
+modifying** `src/db/app-schema.ts` (`tenant_secret`), `src/lib/secret-repo.ts`,
+`src/do/session-do.ts`, and adding D1 migrations.
+
+Hard rules to avoid corruption:
+- **Phases 0-3 are safe to start now** — they touch `auth-schema.ts`,
+  `auth.ts`, `permissions.ts`, `resolve-context.ts`, and *new* env tables, none
+  of which the active plan owns. (Phase 1 touches `resolve-context.ts`, which the
+  active plan reads but does not migrate — diff, don't clobber.)
+- **Phase 4 MUST NOT start** until the active plan's repo-scoped secret tasks
+  (T1-T3) are merged. Migrating `tenant_secret` concurrently from two agents will
+  produce conflicting migrations and a broken journal.
+- **All migrations** must be reconciled through Phase 0 first; any new migration
+  number must be claimed in `.sisyphus/notepads/` to prevent number collisions.
+
+---
+
+## 7. Decisions to resolve
+
+These are flagged in the plan with recommended defaults; none block authoring.
+
+1. **Secret precedence on key collision** — proposed default (broad → wins):
+   `org → team → repo → environment → user → run`, last wins. (See
+   `target-architecture.md §4`.) Confirm before Phase 4.
+2. **`tenant_secret` migration shape** — scope-tuple (recommended, clean,
+   extensible) vs additive nullable columns (lower-risk, messier). Recommend
+   scope-tuple, executed as an additive-then-cutover migration.
+3. **Default team on org create** — auto-create a "Default" team per org in the
+   `databaseHooks.user.create.after` path, or leave orgs teamless until first
+   team is made? Recommend auto-create for a frictionless default.
+4. **Vault boundary** — stay D1-ciphertext (documented simplification) vs
+   external vault. Recommend stay D1 now; revisit with the BYOK gateway (Phase 5).
+5. **`AppContext` shape** — add `teamId?: string` and thread an `activeTeamId`
+   through sessions, or resolve team per-request from membership? Recommend
+   `activeTeamId` on the Better Auth session, mirrored into `AppContext`.
+
+---
+
+## 8. Files in this design set
+
+- `README.md` — this file (state, gap matrix, roadmap, coordination, decisions).
+- `target-architecture.md` — schemas, precedence semantics, RBAC model, env entity, contracts.
+- `.sisyphus/plans/secrets-and-environments-scoping-plan.md` — executable, task-tracked plan.