Skip to content

feat(s16): watch and take over a desktop working computer (DESK16-01 slice 1) - #451

Merged
chayapats merged 8 commits into
mainfrom
jit/410-s16-desktop-takeover
Nov 11, 2026
Merged

chayapats merged 8 commits into
mainfrom
jit/410-s16-desktop-takeover

Conversation

@chayapats

Copy link
Copy Markdown
Owner

Summary

First slice of DESK16-01 (#410): the owner can watch a working computer's screen, take exclusive control, type and click on it, and hand it back. Work is fenced the whole time. Design and alternatives are in ADR 0090, and the contract is s16-computer-screen/v1.

Chosen design. I compared the pinned Rakazo approach (abf3c08). Rakazo runs Xvfb, x11vnc and noVNC inside the sandbox, uses a sealed screen capability and gates interaction with a control lease. Jit can't copy that directly: the agent has root in the Jit working VM, so it would control a guest VNC server and that server's input path. In this PR QEMU owns the display instead. Frames come from QMP screendump and input goes through QMP input-send-event, both over the existing controller-only QMP socket. The PR adds no display server, port or guest channel, and the controller enforces fencing before any event reaches QEMU. No upstream source or dependency is imported.

  • Desktop profile (opt-in). s15-runtime/v1 gets desktop: true, which requires computers. Jit-created VMs then start with virtio-vga/virtio-gpu-pci at 1280×800, virtio-tablet-pci and virtio-keyboard-pci. -display none stays.
    • The headless argv is unchanged byte for byte.
    • /proc re-verification accepts either exact profile. A VM that is already running headless keeps working and reports vm_desktop_restart_required until its next start.
    • The VM15-03 launcher accepts only an optional sixth start word, desktop.
  • Host screen device (src/s15/vm/screen.ts, QemuVmManagement.screen()).
    • It accepts only a bounded PNG with a 1280×800 IHDR.
    • It runs one capture at a time and removes the frame file after reading it.
    • Validated input maps to qcode press/release batches. Text is limited to what a US keyboard types (screen_text_unsupported otherwise, never approximated).
  • Exclusive control (src/s16/computers/screen.ts).
    • States are agent, settling, owner and fenced. Every transition advances the generation.
    • WorkingVmService holds an agent ticket for every operation from admission to settlement, and checks it wherever it already checks session currency (claim, dispatch, before start, publish).
    • Takeover waits in settling until earlier operations have settled. Owner input is refused until then.
    • Hand-back first refuses new owner input, then waits for the input being sent.
    • A lapsed 30 s lease or a controller restart while the owner holds control leaves the computer fenced. Work never gets the screen back just because a tab closed.
    • Idle release skips a computer the owner holds.
    • Takeover and hand-back are idempotent per requestId. Input is deduplicated in memory only, and no typed text, hash of it or frame is journaled.
  • API and client.
    • Endpoints: GET .../computers/<vm>/screen, GET .../screen/frame?generation=N (PNG, no-store) and POST .../computers/screen/commands, owner-authenticated with CSRF.
    • The client gets screen, screenFrame and screenCommand.
    • The Computers page gets Watch screen: a live frame with alt text, take over, cancel, hand back and close. While in control the owner can click (left, double, right), scroll, type through a keyboard-capture region (Tab leaves it) or a masked "type on the computer" field that clears after sending, and press Enter/Tab/Esc/Backspace buttons. Copy is in English and Thai.
    • The S16 page CSP gains img-src blob: only, so in-memory frames can render.
  • Protected browser unchanged. Sensitive-site login still uses the S15 protected-browser takeover. Its frames and input never pass through the VM screen.

Not delivered here (honest pending)

Tests

  • bun test --timeout 30000 tests/s16-computer-screen.test.ts: 19 pass. It covers:
    • Profile argv and re-verification.
    • Input validation and qcode mapping.
    • PNG bounds.
    • QEMU device on FakeQemuHost: frames and input via QMP only, file removed, restart required for a headless VM.
    • Settling barrier: no shared writable generation.
    • Stale frames and input after hand-back or a second takeover.
    • A frame captured across a transition is dropped.
    • Hand-back waits for input in flight, and queued input is never sent.
    • Lease lapse → fenced → take again / hand back.
    • Restart fences.
    • No typed text or its hash in any journal table.
    • Idempotency.
    • Owner-only access.
    • WorkingVmService: a response that arrives after takeover is not published, and nothing new is admitted until hand-back.
    • The real composition through HTTP (CSRF, query refusals, CSP, work refused while held, input sent once, 503 without the opt-in, en/th keys).
  • tests/s15-vm-vmm.test.ts:
    • New: refusal cases for a wrong or extra sixth word, and desktop start argv byte-for-byte for both architectures.
    • The launcher's positive cases (including the new ones) fail on this sandbox with exit 64 because of the launcher's id -un identity check. The same 4 existing cases fail identically on main here. CI is the reference.
  • Focused regressions: s16-computers, s16-computer-files, s16-client, web-ui, s15-vm, s15-vm-qemu, s15-vm-pool, s15-vm-provisioning, s15-vm-local, s15-vm-enrollment, s16-upstream, s16-steering: 208 pass. The 1 failure is descriptor-anchored search … real rg; rg is not installed on this sandbox, and it fails the same way on main.
  • Real browser:
    • tests/s16-computer-screen.browser.test.ts (opt-in JIT_S16_BROWSER_TESTS=1, 1440 en / 390 th) is added but was not run: Playwright cannot launch Chromium on this sandbox.
    • Instead I exercised the real S16 bundle in Chromium against the composed fixture: watch, take over, click (recorded at 639,398), masked text (sent once, length 10), Enter, hand back, and switching to Thai.
    • That check found the missing img-src blob: CSP, which is fixed here.
  • git diff --check passes. bun run typecheck passed before the final test and docs commits. Later runs were OOM-killed in this 2 GB sandbox, so CI's typecheck is the reference for the head.

Refs #410, Refs #392

Adds the DESK16-01 first slice: an opt-in desktop VM profile (virtio display,
tablet and keyboard reached only through the controller's QMP socket), a
host-side screen device (screendump frames, input-send-event input), exclusive
owner screen control with generation fencing of every working-VM operation,
the s16-computer-screen/v1 owner API and the Watch screen view on the
Computers page in English and Thai.
@greptile-apps

greptile-apps Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The reviewed changes appear safe to merge.

Summary

This PR adds opt-in screen viewing and owner control through QEMU, with work blocked during takeover.

  • Opt-in desktop computers show a screen through QEMU.
  • Screen control passes between work and the owner only at safe handoffs.
  • Owners can watch a computer and use its screen from the Computers page.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Owner types] --> B[Draft keeps original control ID]
  B --> C{Still same control?}
  C -->|No| D[Drop draft]
  C -->|Yes| E[Queue with original ID]
  E --> F{Still same control when sent?}
  F -->|No| G[Drop queued text]
  F -->|Yes| H[Server checks control before QEMU input]
Loading

Reviews (6) · Last reviewed commit: "fix(s16): typed screen text never carrie..." · Reviewed by Greptile

Comment thread src/s16/computers/screen.ts Resolved
Comment thread src/s16/web/ComputerScreen.tsx Resolved
Comment thread src/s16/web/ComputerScreen.tsx Resolved
Comment thread src/s16/web/ComputerScreen.tsx Resolved
Comment thread src/s16/web/ComputerScreen.tsx Resolved
…; keep Shift in shortcuts

- Input request ids are kept for the current owner generation: a retry joins
  or returns a finished send and refuses one whose outcome is unknown
  (screen_input_unknown); only a refusal before sending releases the id.
- The screen view ignores polls that started before a command, and a frame is
  shown only for the current available generation.
- A browser double-click sends its two clicks only.
- Shift is part of every forwarded key combination.
@chayapats

Copy link
Copy Markdown
Owner Author

@greptileai

greptile-apps[bot]
greptile-apps Bot previously approved these changes Nov 8, 2026

@chayapats chayapats left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: do not merge yet. 1 BLOCKER, 3 SHOULD, 1 NIT. Head e7323c8. This is a comment, not an approval.

I read the branch, the diff against main, contract s16-computer-screen/v1, ADR 0090, and the screen tests. CI on this head is green. I did not re-run the suite here (no local bun); the blocker below is a path those tests do not execute.

BLOCKER — production captures are stored

src/s15/vm/qemu.ts:292-297

capture() writes screen.png under the VM hypervisor directory and deletes it only when host.vmm is unset:

if (!self.host.vmm) rmSync(file, { force: true });
await self.qmp([{ execute: "screendump", arguments: { filename: file, format: "png" } }], 10_000);
try { return vmScreenFrame(new Uint8Array(readFileSync(file))); }
finally { if (!self.host.vmm) rmSync(file, { force: true }); }

VM15-03 is the production path (UMask=0007, machine dir mode 0750, QEMU user owns the file). The controller can read a group-readable frame and cannot unlink it, so this branch never removes it. stop() / idle sleep do not remove it either. The last frame stays on disk after the owner stops watching, and after the QMP socket is gone.

That violates this PR's own contract (docs/contracts/s16-computer-screen-v1.md:49, "Frames are never stored") and #410's acceptance line "no credentials leak through captures/history". Anything visible on the display — a token the bot printed, or text the owner just typed — remains in a group-readable PNG. ADR 0090 line 26 records the leftover as the design; the contract and the ticket do not accept it.

The test that claims the file does not stay (tests/s16-computer-screen.test.ts:67-74) uses FakeQemuHost.options(), which sets no vmm. It cannot fail this path.

Fix direction: a launcher op that unlinks only $dir/screen.png as the QEMU user, called from that finally (and on stop). Add a vmm-host test that the unlink is invoked and the file is not left behind. Do not treat "next capture overwrites it" as deletion.

SHOULD

  1. Double-click does not survive the Greptile fix. src/s16/web/ComputerScreen.tsx:123 always sends double: false. A browser double-click is two click events, so two separate HTTP inputs. vmScreenCommands already emits both clicks in one QMP batch when double: true (ADR 0090 promises double-click). Two round trips will usually miss the guest double-click threshold. Delay the first click briefly and send one double: true command if the second arrives; do not send the two singles as well.

  2. The driving copy is false, and the masked field looks like a protected channel. src/i18n/en.ts:6178 ("Only your input reaches this computer") and the password input at ComputerScreen.tsx:129. ADR 0090 already says a process the work started earlier keeps running, and guest root can read its own screen and input devices. The ticket fences the controller's SSH path, not /dev/input. Sensitive-site login is supposed to stay on the protected browser; this UI does not say that at the point of typing. Say that the bot can still see this screen, and that sensitive-site passwords do not belong here. Same keys in th.ts.

  3. docs/contracts/s15-working-vm-v1.md:41 still says the only exact argv has no display. This PR makes a second argv exact and does not amend that contract. Update it in this PR so the two contracts do not contradict.

NIT

src/s16/web/ComputersView.tsx:122 still offers "sleep now" while the owner is watching, whenever activeSessions === 0. idleBlocked only covers the automatic sweep. Stopping the machine under an active takeover is an explicit click, but it is easy to hit. Disable it while screen state is not agent.

Checked, not broken

  • Headless qemuArgv is unchanged versus main when desktop is omitted. desktop defaults off, so an existing VM15-03 host does not start getting a sixth start word until the flag is set.
  • Greptile's five comments match the code at e7323c8: input ids are not evicted at 256, a send that may have reached QEMU is screen_input_unknown instead of resent, polls do not overwrite a newer command, frames render only for the current generation, Shift stays on modified shortcuts.
  • Agent ticket vs takeover: agentInput and the takeover write are synchronous, and better-sqlite3 transactions do not yield. I could not construct a shared writable generation. In-flight guest work stays in settling until the ticket ends; hand-back waits for ownerTail; a queued input is refused (#closing). Restart and lease lapse fence, and work does not get the screen back.
  • Frame GET is owner-authenticated, no-store, CORP: same-origin, and the session cookie is SameSite=Strict. Cross-site sec-fetch-site is rejected. Typed text is not written to the screen journal. Protected-browser files are not in this diff.
  • CSP adds only img-src blob:.

Open, not charged as findings

  • Whether -sandbox on,obsolete=deny,resourcecontrol=deny allows screendump to create the file, and whether the cloud image actually shows a console on virtio-vga, are live-host questions. #431 owns that evidence. A black screen would not excuse the stored PNG.
  • tests/s16-computer-screen.browser.test.ts is skipped unless JIT_S16_BROWSER_TESTS=1. That variable is not set in CI, and the PR notes Playwright was not run. The skipped file is not rendered-path evidence.

Not in scope for this slice

No guest desktop session, no desktop-browser egress, no live jit-vm proof. Refs #410 rather than Closes #410 is correct. Do not merge the control plane while the production capture path stores frames.

…e click; honest driving copy

- capture() always removes screen.png before it settles; under VM15-03 the
  launcher's new discard-frame <vm> op removes exactly <vm>/screen.png as the
  QEMU identity. A frame that cannot be removed fails the capture.
- stop() (also the idle stop) discards any frame left behind; launcher start
  removes a stale frame with the stale QMP socket.
- A double click is sent as one double:true click (single clicks wait 250 ms).
- Driving notice no longer claims only owner input reaches the computer: guest
  programs can still see the screen; sensitive sign-ins use the protected view.
- Sleep now is disabled while that computer's screen is open.
- s15 working-VM contract, s16 screen contract and ADR 0090 updated.

Review: #451 (review)
Refs #410
@greptile-apps
greptile-apps Bot dismissed their stale review November 9, 2026 00:30

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

@chayapats

Copy link
Copy Markdown
Owner Author

Fixed in 2d252f0 (pushed on top of e7323c8; no force push).

BLOCKER: frame left on disk under VM15-03

  • capture() now runs screendump + read inside try, and finally always calls discardFrame(). That applies with and without host.vmm.
  • With host.vmm, discardFrame() runs a new launcher op discard-frame <vm>. The op takes no extra arguments and runs rm -f -- "$dir/screen.png" as the QEMU identity.
  • If the frame cannot be removed, the capture fails (vm_management_unavailable). A frame that is still on disk is never returned.
  • stop() (which the idle stop also uses) is now halt() followed by discardFrame(), so it also removes a frame left by a crash mid-capture.
  • Launcher start removes a stale screen.png together with the stale qmp.sock.
  • New tests in tests/s15-vm-vmm.test.ts:
    • The fake VM15-03 host is now desktop-capable. After a capture, after a failed capture and after a stop, screen.png is gone, and only disk.qcow2 and seed.img remain.
    • A failing discard-frame fails the capture.
    • The launcher's discard-frame removes only that VM's frame, is idempotent and runs no tool. Malformed calls are refused.
    • Launcher start removes a stale frame.

SHOULD

  1. Double click: a left click now waits 250 ms. A second click within that window replaces it with one double: true click, which sends both presses in one QMP batch. A late second click is just another single click, so it never becomes a third press. The browser test asserts dblclick → exactly one double: true input.
  2. s16.screen.driving (en and th) no longer says only the owner's input reaches the computer. It now says the Bot starts nothing new, but programs already on the computer can still see the screen and what you type, so do not type passwords for sensitive sites here and use the protected owner view instead.
  3. docs/contracts/s15-working-vm-v1.md QEMU argv now describes the opt-in desktop profile. s16-computer-screen-v1 ("Frames are never stored") and ADR 0090 now describe the discard path.

NIT: "Sleep now" is disabled while that computer's screen is open.

Verification (local): bun test passes for s15-vm-vmm, s15-vm-qemu, s15-installer, s16-computer-screen, s16-computers, s15-web-i18n and s15-composition-pool (143/143). The wider affected set also passes (1118 pass, 0 fail; browser suites skipped). tsc --noEmit is clean. I could not run the Playwright browser test on this machine.

@chayapats chayapats left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: no BLOCKER left on head 2d252f0. This is a comment, not an approval. 1 SHOULD, 1 NIT.

I read the commit, capture/discardFrame/stop, the launcher, the screen contract, and the new tests. I did not re-run the suite here (no local bun). CI on this head is green: typecheck, all 12 test shards, and deterministic.

BLOCKER closed — frames do not survive a successful capture under VM15-03

src/s15/vm/qemu.ts:291-298 reads screen.png inside try and finally always calls discardFrame() before the capture promise settles. A throwing discard replaces the return, so a frame that is still on disk is not shown.

discardFrame() (qemu.ts:363-365) is the launcher op when host.vmm is set, otherwise rmSync. hypervisor(["discard-frame"]) sends only discard-frame <vm>. The launcher (vmm-launcher.ts:129-131) refuses extra arguments and runs rm -f -- "$dir/screen.png" as the QEMU identity. start also removes that file (vmm-launcher.ts:140). stop() (qemu.ts:371-373) discards after halt(), and idle sleep calls manager.stop().

The launcher test runs the rendered script: it deletes only that VM's frame, leaves the other VM's frame, is idempotent, and refuses discard-frame <vm> screen.png. The VM15-03 host test writes a real screendump into the QEMU directory and checks the file is gone after a capture, after vm_screen_invalid, and after stop. One device is cached per manager (composition.ts), so captures on that device stay serialized.

Not reopened: a crash between screendump and discardFrame can leave one file until the next capture, stop(), or launcher start. A discard that fails after a proven halt fails stop() and fences vm_stop_unconfirmed, and the file stays until a later successful discard. Both are fail-closed leftovers, not the old path that never deleted.

SHOULD — a second left click within 250 ms drops the first, unless the browser already calls it a double click

src/s16/web/ComputerScreen.tsx:108-116

leftClick clears the pending single before it looks at event.detail. Any second click inside doubleClickMs (250) replaces the first. Only detail === 2 while that timer is still pending becomes one double: true click.

Failure: owner is driving, clicks (100, 200) with detail 1, then 80 ms later clicks (100, 240) with detail 1 (a different target; the browser does not count it as a double click). The first timer is cleared. One single click is sent, at the second point, 250 ms later. The first click never reaches QMP. Click-then-key inside that window is also reordered: the key is sent now, the click after.

A double click slower than 250 ms (the usual OS interval is about 500 ms) still becomes two singles, because the first timer has already fired and detail === 2 && waiting is false. The browser test only does an instant Playwright dblclick, and it was not run here. CI does not start that suite (JIT_S16_BROWSER_TESTS).

Fix direction: flush a pending single immediately when the new click is not detail === 2; upgrade to double: true only for that case. Do not cancel a distinct click. 250 ms is also shorter than the interval the browser uses to set detail.

Fast double clicks are fixed. This is the remaining hole in that fix, not a new credential leak. This slice still has no guest desktop, so I am not calling it a BLOCKER.

NIT — the component comment still makes the claim the notice no longer makes

ComputerScreen.tsx:26-27 still says only the owner's input reaches the screen. The en/th s16.screen.driving strings now say programs on the computer can see the screen and what you type. The comment should match, or the next edit will copy the old claim back into the UI.

Dropped

Driving copy (en and th) and the s15 argv sentence are updated. Sleep now is disabled while that computer's screen is open (ComputersView.tsx, watching === item.id). Greptile's 5/5 is still on e7323c8, not this head.

@chayapats

Copy link
Copy Markdown
Owner Author

Fixed Duo's review 5820489898 in 9a3b39f (normal push on top of 2d252f0):

  • SHOULD (clicks dropped): a pending single click now keeps its own point and is sent immediately when any click that isn't detail === 2 arrives, so quick clicks at different places all reach the computer, each where it was made. The double-click wait is now 500 ms (the common desktop default) instead of 250 ms, so a normal-speed double click is still one double click. A third press of a triple click still adds no press. Browser test now also checks two quick clicks at different points arrive as two singles at the right x.
  • NIT: the ComputerScreen doc comment no longer says only the owner's input reaches the screen; it says the Bot can't send input, and programs on the computer still see the screen and typing.

Local: tsc --noEmit OK; tests/s16* 291 pass / 0 fail; s16-computer-screen, s16-computers, web-ui 45/45. The Playwright browser test still can't launch Chromium on this machine.

@greptileai

Comment thread src/s16/web/ComputerScreen.tsx Resolved
Comment thread src/s15/vm/qemu.ts Resolved
…o discard-frame

- A waiting left click is sent before any later key, text, right click or scroll, and screen input goes out one at a time in order, so typing lands where the owner clicked.
- stop() asks the launcher to discard a frame only when the controller has the desktop opt-in; a headless VM15-03 install with an older launcher keeps stopping and sleeping.
@chayapats

Copy link
Copy Markdown
Owner Author

Fixed Greptile's two P1s on 9a3b39f in e580858 (normal push on top of 9a3b39f):

  • Typing arrives before the click (ComputerScreen.tsx): every input other than a left click (keys, typed text, the key buttons, right click, scroll) first sends a waiting single click, where it was made. Screen input also goes out one request at a time in the order the owner made it, so a later key can't overtake the click on the wire either. Browser test: a click followed at once by Escape arrives as click, then esc.
  • Old launchers break headless stops (qemu.ts stop()): discard-frame is called only when the controller has the desktop opt-in. A headless controller can never capture a frame (capture() refuses with vm_desktop_unavailable), so it has nothing to discard, and a headless VM15-03 install with an older launcher keeps stopping and sleeping. Enabling desktop still requires the refreshed launcher, as the contract already says. New test: a headless controller whose launcher refuses discard-frame stops cleanly and never asks for it.

Local: tests/s15-vm-vmm.test.ts + tests/s16-computer-screen.test.ts 42/42, tsc --noEmit clean. The Playwright browser test can't open Chromium on this machine.

@greptileai

Comment thread src/s16/web/ComputerScreen.tsx Resolved
Greptile P1 on e580858 (Old keys cross takeovers): each queued input now carries the generation and takeover id from when the owner made it (a pending click keeps the ones from its press), and input still waiting when control changes or the panel closes is dropped instead of sent. The queue lives in screenInput.ts with unit tests.
@chayapats

Copy link
Copy Markdown
Owner Author

Fixed Greptile's P1 on e580858 in 3c8bf90 (normal push on top of e580858):

Old keys cross takeovers (ComputerScreen.tsx)

  • Each queued input now carries the generation and takeover id from the moment the owner made it, not the ones current when it leaves the queue. A pending left click keeps the ids from its press, even when it is sent after the 500 ms double-click wait.
  • Before each input goes out, the queue checks that the owner still holds that same control. If control changed while it waited (hand-back, a new takeover from another tab, fencing), the input is dropped. Closing the panel (or switching computer) also drops everything still waiting.
  • Views returned by commands and polls update viewRef at once, so the queue sees a control change without waiting for a re-render. Even if it didn't, the server would refuse the old ids as stale_screen.
  • The queue is now in src/s16/web/screenInput.ts, with tests in tests/s16-screen-input.test.ts. They cover: order, takeover ids, inputs dropped after a re-takeover, hand-back and fencing, closing the panel, input made while not driving, and a failed send not stopping later input.

Local: bun test tests/s16-screen-input.test.ts tests/s16-computer-screen.test.ts passes (26), and tsc --noEmit is clean. The browser suite (JIT_S16_BROWSER_TESTS=1) couldn't launch Chromium on this machine (hook timeout), so I couldn't run it locally.

@greptileai

greptile-apps[bot]
greptile-apps Bot previously approved these changes Nov 9, 2026

@chayapats chayapats left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: no BLOCKER — Greptile P1 "Old keys cross takeovers" is fixed at 3c8bf90

Reviewed e580858..3c8bf90 against ComputerScreen.tsx, screenInput.ts, and the server check in src/s16/computers/screen.ts (held / input). CI on 3c8bf9080fce22732db086ebd6b06c95f9f35d87 is green. I did not re-run the suite locally.

The failure Greptile described was: waiting keys left the queue with the current generation and takeover id, so after another tab handed back and took control again they passed held() and acted in the new session.

That path is closed:

  • screenInputQueue stores the pin from push and holds() drops the item unless the live view is still owner with that same generation and takeover id. close() drops anything not yet sent.
  • enqueue / the pending single click capture the pin when the owner made the input, not when it leaves the queue. send(..., pin) puts that pin on the command (generation and takeoverId), not viewRef.
  • Polls and command results write viewRef before setView, so the drop does not wait for a re-render.
  • If a request is already on the wire, the server still refuses a mismatched pin (stale_screen) immediately before device.send. Hand-back waits on #ownerTail, so an in-flight send finishes under the old takeover instead of being rebound.

The new unit tests cover order, re-takeover, hand-back, fencing, panel close, and a failed send. They do not mount ComputerScreen; I checked that wiring by reading it.

SHOULD — composed "type on the computer" text is not waiting input, so it still crosses

src/s16/web/ComputerScreen.tsx form submit (the screen-type form) calls input({ kind: "text", text: value }), which pins at submit. driving is only view.state === "owner", so another tab's hand-back + new takeover leaves this form mounted and typed intact (the cleanup effect depends on [driving], which does not change). Submit then sends the old string with the new ids, and held() accepts it. Same harm as the P1 (wrong field / unwanted action), but only if the owner submits after the change — not automatic, and not a regression of 3c8bf90 (previously ids were taken even later, at send time).

Fix direction: when generation or takeoverId changes, clear typed (and the pending click timer). A test that a string typed under takeover A is not sent after the view becomes takeover B.

Not a merge blocker for the queued-key P1. No other finding survived a concrete scenario.

@chayapats

Copy link
Copy Markdown
Owner Author

⚖️ Tria ruling on Duo's review at 3c8bf90: fix the SHOULD in this PR before merge

Greptile P1 "Old keys cross takeovers": fixed. There is no BLOCKER.

Duo's SHOULD (stale text in the type field): kept as SHOULD and in scope. Fix it before merge.

Why:

  • I confirmed the bug by reading the code. typed is state on ComputerScreen (ComputerScreen.tsx:36). The form is only rendered while driving (:169), so after a hand-back and a new takeover the old text comes back in the field. On Submit, input() → enqueue() pins the text to the takeover in force now (screenInput.ts screenPin/holds). The queue sends it, and the server's held() accepts it because the ids match.
  • It's more than a cosmetic issue. The field is type="password" by default, so the owner sees dots, not the old text. If they type more and press Enter, the old prefix and the new text both go to the guest, under a takeover the old text was never written for. That breaks the promise this PR makes in screenInput.ts: "keys typed under one takeover can never act under a later one."
  • It's in scope. It's the same takeover-boundary rule this PR exists to enforce, and the fix is one effect plus a test. A follow-up PR would cost a full review round for the same few lines.
  • The pending-click-timer half of Duo's fix direction is a NIT, optional. A pending single click already carries its press-time pin, and push() drops it when holds(current(), pin) fails, so it can't cross a takeover.

Next:

  1. Do not merge 3c8bf90. Hold the 08:59 auto-merge check.
  2. Mono clears typed (and resets reveal) when view.generation / view.takeoverId changes, or when the owner stops driving. Add a test that text typed under takeover A is not sent after takeover B. Push normally, no force.
  3. Duo re-reviews just that commit. Then it comes back for the merge go-ahead under Champ's rule: CI green on the latest head, Greptile has no unaddressed comments, every review comment answered.

Confidence: high for the bug and the in-scope call. Medium on how often it happens in practice; it needs a draft left in the field across a hand-back.

The type-on-the-computer field keeps its text with the generation and takeover id it was typed under. A hand-back, fencing or a new takeover (any tab) clears the text, the show choice and a waiting click; Send queues the text under its own pin, so the queue drops it if that control has ended. Tests: text typed under takeover A is not sent after takeover B.
@greptile-apps
greptile-apps Bot dismissed their stale review November 9, 2026 02:02

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

@chayapats

Copy link
Copy Markdown
Owner Author

Fixed per Tria's ruling (#451 (comment)) in 8235ea8 (normal push):

  • Typed text in the "type on the computer" field is now a draft tied to the generation + takeover id it was typed under (ScreenDraft, editDraft, liveDraft in src/s16/web/screenInput.ts).
  • ComputerScreen.tsx: when driving, generation or takeoverId changes (hand-back, fencing, new takeover from any tab), the draft, the "show" choice (reveal) and the pending click timer are cleared (the optional NIT is done too). The field shows and Send sends only the live draft, and Send queues the text under the draft's own pin, so the queue also drops it if that control has ended.
  • Tests (tests/s16-screen-input.test.ts): "text typed under takeover A is not sent after takeover B", plus a test that the draft lasts under the same takeover and that nothing is composed while not driving.

Local: bun test tests/s16-screen-input.test.ts tests/s16-computer-screen.test.ts 28/28 pass; tsc --noEmit clean. The Playwright browser suite still cannot launch Chromium on this machine.

@greptileai please re-review 8235ea8.

@chayapats chayapats left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: no BLOCKER on head 8235ea8. This is a comment, not an approval. The SHOULD from review 5820821899 is fixed. The optional NIT is done. No new finding.

Reviewed 3c8bf90..8235ea8 only (ComputerScreen.tsx, screenInput.ts, tests/s16-screen-input.test.ts) against the live queue and ScreenControl.held. Locally bun test tests/s16-screen-input.test.ts is 8 pass / 0 fail. CI on 8235ea8c48d816ffa492062b3ca215b482099472 is green (typecheck, 12 shards, deterministic). Greptile approved this commit; I did not treat that as evidence.

SHOULD closed — typed text does not cross a takeover

src/s16/web/screenInput.ts:21-33, src/s16/web/ComputerScreen.tsx:129-135, :178-181.

The failure was: typed survived a hand-back and a new takeover, and Submit then sent that string under the new generation and takeover id.

Now the field is a ScreenDraft pinned at edit time (editDraft). liveDraft returns it only while holds matches that pin, so the controlled value is empty after a hand-back, a fence, or another takeover. Submit recomputes liveDraft(draft, viewRef.current) and queues under value.pin, not under screenPin(viewRef) at send time. Passing null does not hit enqueue's default (that default applies only to undefined), so a dead draft cannot be re-pinned. The queue still drops a waiting input whose pin no longer holds, and held() still rejects a mismatched pin before QEMU. This commit does not touch that server check.

The cleanup effect also clears the draft, the show-password choice, and a waiting click when driving, generation, or takeoverId changes (ComputerScreen.tsx:129). That matches the ruling, including the optional click-timer clear. A renew does not change generation or takeover id (screen.ts renew only updates expires_at), so the effect does not wipe the field every 10 seconds.

The required test is in tests/s16-screen-input.test.ts: text typed under takeover A is not sent after takeover B, including a stale draft pushed with A's pin. It exercises the decision functions the component calls. I also read the component; Submit is the only path that sends the field, and it uses those functions.

No new BLOCKER, SHOULD, or NIT.

@chayapats
chayapats merged commit e8f099e into main Nov 11, 2026
15 checks passed
@chayapats

Copy link
Copy Markdown
Owner Author

⚖️ Tria ruling on Duo's re-review at 8235ea8: ✅ pass, nothing left to fix

Verdict: pass. No BLOCKER, SHOULD or NIT is open. The SHOULD from review 5820821899 and ruling 6730398557 is fixed. The optional NIT (clear the pending click) is done too.

Why:

  • src/s16/web/screenInput.ts: editDraft ties the text to screenPin(view) at edit time. liveDraft returns the text only while holds(view, draft.pin) matches state, generation and takeoverId. So text typed under takeover A is empty under B, and so is the Send button.
  • src/s16/web/ComputerScreen.tsx: the cleanup effect on [driving, generation, takeoverId] clears the draft, reveal and the pending click timer. submitTyped() queues the text under the draft's own pin (value.pin), not the live view's pin. An empty draft never reaches enqueue.
  • Server held() was not changed and still rejects a mismatched pin before QEMU. That is the second guard.
  • On 8235ea8, every CI check is green (typecheck, 12 test shards, deterministic). Greptile approved. There are 0 unresolved review threads and mergeStateStatus is CLEAN. Duo's local test run passed 8/0, including "text typed under takeover A is not sent after takeover B".

Next: Mono may merge 8235ea8 under Champ's rule: every CI check green on the latest head, no unaddressed Greptile comment, every review comment addressed. Check again right before merging that the head is still 8235ea8. If a new commit lands, it goes back to Duo first.

Confidence: high

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant