[Frontier] [DESK16-01] Watch and take over a working computer safely #410

Parent

#392 — #392; complete S16 target #392.

Model tier

Frontier: this slice requires judgment about protected authority, lifecycle/recovery or a new cross-client/service contract; exact new APIs and migrations are not yet settled.

Context to read

AGENTS.md; CONTEXT.md; docs/agents/workflow.md; docs/specs/S16-rakazo-experience.md; docs/specs/S15-conversational-autonomy.md; ADR 0064 and ADRs 0074–0077; docs/design/S16-rakazo-experience.md; docs/design/S16-mobbin-research.md. Reuse the completed design research; do not repeat broad Mobbin discovery.

Starting implementation/contract surfaces: src/s15/browser/types.ts; src/s15/vm/; src/s15/web/browser.ts; src/s15/web/computers.ts; src/s16/client/. Read their current definitions and relevant tests before selecting changes. Paths with braces denote existing module families, not settled new signatures.

Deliverable

Watch the actual working screen, acquire exclusive owner control, complete protected browser input and hand back with fresh input generations; ordinary VM desktop and protected website surface remain distinct.

Baseline and readiness

Audited stack head 17b814220371cd1e2126be2ec9bd9753537f69e6; main remains the pre-S16 baseline. #398/#399/#403/#404 are open drafts, not merged/live acceptance. Existing S15 services are reusable baseline, and #400/#401 supply draft text/binary custody. No new behavior is claimed here. Branches/PRs and issue ownership were inspected on 2026-11-05; this new ticket has no implementation claim. Recheck at dispatch. Preserve the primary-owned jit/402-s16-file-memory branch.

Needs triage: contract/engineering choices remain; no ready-for-agent label. Resolve named contract decisions and verify prerequisite revisions before unattended implementation. Independent audit/design work may proceed now; final acceptance needs these deliveries.

Implementation steps

1. Recheck ownership, branches/PRs, prerequisites and exact source baseline. Compare existing implementation, pinned Rakazo reuse and a simpler alternative; record the chosen tradeoff and versioned contract before coding.
2. Select frame/input transport and capability protocol with bounded queues and authenticated owner ingress.
3. Implement takeover barrier waiting for agent-input settlement before owner input; fence prior handles and frames.
4. Keep sensitive login on protected browser transport, not root-controlled guest desktop; recheck session on hand-back.
5. Bind authored watch/control/disconnect states and accessibility controls to actual frame and lease facts.
6. Integrate the authored experience using real service facts, English/Thai dictionary keys in their designated files, accessible loading/error/empty/reconnect states and current capabilities. For non-UI evidence work, verify those states instead of inventing new UI. Record migrations, compatibility and exact build evidence.

Acceptance criteria

• Concurrent agent input and takeover never share a writable generation.
• Old frames/input after reconnect and hand-back are rejected.
• Disconnect during login remains owner-controlled or fenced.
• Screen content carries private lineage; no credentials leak through captures/history.
• Deliverable works end to end through its actual owning service/client; source presence, transport success and model prose alone are insufficient.
• No S14 default, S15 opt-in, installed policy, historical private scope or owner-host configuration changes without its separate explicit transition.
• New contracts, unresolved limits and exact revision/evidence are recorded; applicable upstream licenses, immutable provenance and modified-file notices ship.

Tests and evidence

1. Concurrent agent input and takeover never share a writable generation. Use scripted models, protocol fixtures, fake clocks and barriers for deterministic cases; assert actual resulting state/bytes/effects.
2. Old frames/input after reconnect and hand-back are rejected. Use scripted models, protocol fixtures, fake clocks and barriers for deterministic cases; assert actual resulting state/bytes/effects.
3. Disconnect during login remains owner-controlled or fenced. Use scripted models, protocol fixtures, fake clocks and barriers for deterministic cases; assert actual resulting state/bytes/effects.
4. Screen content carries private lineage; no credentials leak through captures/history. Use scripted models, protocol fixtures, fake clocks and barriers for deterministic cases; assert actual resulting state/bytes/effects.
5. Run relevant existing module/composed regressions plus the new scenario suite; for rendered behavior, exercise the real bundle in English/Thai, keyboard and mobile widths. Record exact test paths/commands in the PR after implementation selects them.
6. Keep real host/account/device checks opt-in and evidence-labelled; no absent credential blocks independent deterministic work. Public evidence contains hashes/outcomes only; installation-specific details belong in ignored .local records.

Requirements

S16-AC05, S16-AC07; FR-02, FR-10, FR-14, FR-15, FR-18, FR-21, FR-35, FR-36, FR-38, FR-40, NFR-01, NFR-06, NFR-07, NFR-11, NFR-12, NFR-14. Retain S15 authority decisions and FR/NFR identifiers.

Blocked by

• [Frontier] [BROW16-01A] Prove a modern browser mediation contract #408: prerequisite owning contract/delivery; currently open, not merged acceptance.
• [Frontier] [API16-01] Run a real tool-backed conversation through a reusable Jit client contract #395: prerequisite owning contract/delivery; currently open, not merged acceptance.

Contracts used/delivered

Use current owning POLICY, DATA15, TEAM, MEM, WORK, CHAT and S16 client/file contracts where relevant, plus Schway's supported public host surface. Specific inputs and proposed delivery: src/s15/browser/types.ts; src/s15/vm/; src/s15/web/browser.ts; src/s15/web/computers.ts; src/s16/client/. The Deliverable and ordered steps define the missing contract outcome; freeze version, schemas, migration/recovery and conformance fixtures in the implementation ADR/PR. Proposed operations are not already supported APIs. A demonstrated missing engine seam requires a bounded public-only reproduction and a linked Schway contract ticket, never an internal import or duplicate runtime.

Out of scope

Other tickets' outcomes, hosted multi-user SaaS, marketplace operation, upstream branding, silent billing, replacing Schway, general historical data conversion, and unapproved live deployment. Live credentials, real-device acceptance and owner-host cutover are separately owned by LIVE16-01/CUT16-01; deterministic implementation remains in scope.

Do not

Do not use source/model/tool output as authority; expose credentials, sensitive-site sessions or protected state to working execution (trusted-site sessions are the S16-D09 exception); remove guards before proof; auto-retry unknown effects; weaken or bypass the S16-D07 human-required list or let anything but the in-chat owner decision approve it; treat current stack limitations as scope cuts; add model attribution; use implement-parallel; alter #402 ownership; self-merge or deploy. No placeholder controls or fabricated success.

Definition of done

Run bun install --frozen-lockfile, bun run typecheck, bun test --timeout 30000 for affected deterministic suites, and git diff --check; record exact selected suite commands and results. Add real-browser tests for rendered paths and package checks for shipped artifacts. Pure feasibility documentation runs link/coverage checks and git diff --check; a prototype must run its conformance cases. PR body includes Closes #410 and Refs #392, contract versions, tested head and honest pending evidence. No self-merge; parent #392 remains open until full accepted outcomes.

Owner revision 2026-11-06

Grok-style autonomy (S16-D06–D11, ADR 0084) and Telegram removal (ADR 0085) apply. Only the S16-D07 human-required list waits for the owner, approved by a one-press in-chat card; GitHub push/PR/review replies/merge after required checks are the product default; trusted sites may keep sessions in the working VM browser while sensitive sites stay in the protected browser. Jit has no messaging channel; notifications use Web Push (#441). Policy/approval changes belong to #442.

Protected-input takeover is required for sensitive sites; on trusted sites the owner may log in directly in the VM desktop/browser.