Bake-off 3: Measure behavior, resources, and maintenance cost

[danielmrdev]

Parent

Part of #49.

What to build

Measure the completed Floem and Tauri candidates through the same Linux protocol and produce an auditable comparison package. A maintainer can see which hard gates pass, which native behaviors work, and what each candidate costs to build and maintain before any stack identity is revealed in the visual review.

Acceptance criteria

• Both candidates are measured on the same machine, display scale, window geometry, fixtures, warm-start protocol, settle times, and isolated data directories.
• Warm startup, idle CPU, complete process-tree RSS/PSS, thread/process counts, compressed native package size, 1/10 MiB interaction behavior, typing/scroll responsiveness, and repeated-cycle memory behavior are recorded.
• Results apply the hard limits: ≤1 second warm startup, ≤1% idle CPU, ≤150 MiB idle PSS, ≤250 MiB 10 MiB interaction PSS, and ≤25 MiB compressed native package.
• Integrated-titlebar drag, double-click, window controls, focus, inactive, scale, keyboard, hit-target, and accessibility behavior are recorded as pass or fail.
• Selection, clipboard, undo/redo, IME, wrapping, scrolling, shortcuts, context menu, Spellcheck without autocorrect, App scale, and 1/10 MiB Document checks are recorded for both candidates.
• The Tauri package uses the system WebKitGTK dependency; unsupported AppImage size is not substituted for the native-package gate.
• Implementation time, framework-specific code, dependencies, patches, forks, workarounds, edit-preview cycle, platform defects, and permanent regression burden are compared.
• Paper/Ink screenshots and interaction recordings use identical content and dimensions and are anonymized as candidate A and B.
• Raw evidence and limitations are retained; no failed hard gate is averaged into an overall score.

Blocked by

• Bake-off 2A: Reproduce the target in Floem #51
• Bake-off 2B: Reproduce the target in Tauri and CodeMirror #52

Workspace

• Mode: integration
• Branch: feat/49-ui-stack-bakeoff
• Worktree: /home/daniel/Projects/lettera/.worktrees/feat-49-ui-stack-bakeoff
• Integrates into: feat/49-ui-stack-bakeoff
• Remote policy: local only — never push
• Completion: validate and commit directly in the integration worktree after both prototype lanes have merged and been cleaned up

[danielmrdev]

Bake-off 3 measurement results — Floem vs Tauri prototypes

Measured 2026-08-12 on the same machine/session as the Floem baseline: Arch Linux, Hyprland 0.56 (Wayland), i7-1255U (12 logical CPUs), 3840×2160 @ scale 1.6, isolated XDG dirs. Protocol per docs/specs/ui-stack-bakeoff.md and the Floem baseline. Prototypes use simulated content (no production persistence); the large-Document measurement ends after edit and stabilization per the bake-off spec.

Hard gates

#	Gate	Limit	Floem	Tauri (release)	Result
1	Warm window-map startup (median of 7 runs)	≤ 1 s	281 ms (265–335)	866 ms (843–915)	Both pass
2	Idle CPU	≤ 1% of one logical CPU	0.60%	0.1%	Both pass
3	Idle process-tree PSS	≤ 150 MiB	85.4 MiB (1 proc)	211.6 MiB (host 60.6 + WebKitNetworkProcess 23.0 + WebKitWebProcess 133.0)	Floem pass · Tauri FAIL
4	PSS after 10 MiB Document + edit	≤ 250 MiB	108.3 MiB	282.6 MiB (webproc 188.8)	Floem pass · Tauri FAIL
5	Compressed native package	≤ 25 MiB	6.3 MiB (gzip -9 stripped, baseline proxy)	2.0 MiB binary + 113 KiB dist	Both pass
6	Typing / scroll responsiveness	no perceptible latency	242.6 ms / 20 chars; 36.7 ms PageDown dispatch	177.3 ms / 20 chars; 26.0 ms PageDown	Both pass (external dispatch timing, same proxy as baseline)
7	Memory over repeated edit+idle cycles	no sustained growth	63.6 → 64.0 MiB (+0.4, 5 cycles)	220.5 → 215.3 MiB (−5.2, 5 cycles)	Both pass
8	Integrated titlebar complete behavior + accessibility	complete contract	drag ✓ (native) · double-click ✗ · minimize n/o · maximize ✓ · close ✓ · accessible names ✗ · hit targets 40×36	drag ✓ (data-tauri-drag-region) · double-click ✗ · minimize n/o · maximize ✓ · close ✓ · accessible names ✓ (aria-labels) · hit targets 40×36	Both incomplete

n/o = not observable: Hyprland 0.56 has no minimize dispatcher; minimize() returns Ok without window-state change (compositor limitation, not a prototype defect).

Titlebar visual integration (pixel-verified)

Fresh captures of both floating windows, active state:

Aspect	Floem	Tauri
Background	#F5F5F5 white — titlebar_view() sets no background(); inherits the Floem window default	#F3EEE1 paper cream — inherits var(--bg) from the app root
Window controls	Left (glyphs x 0–94) — Position::Absolute inset_right(8) has no effect; controls stay in the flex flow	Right (x 1165–1254) — CSS position: absolute; right: 8px
Title	not centered (in-flow with controls)	centered
Minimize glyph	1-px line, nearly invisible	visible

Floem prototype fails the integrated-themed-titlebar visual contract; Tauri meets it. This is prototype-implementation evidence (the Floem bar is fixable in code); the bake-off scores the candidates as built within their budgets.

1 MiB / 10 MiB exercise

	Floem	Tauri
1 MiB load PSS	96.2 MiB (edit verified by OCR)	259.2 MiB (host 88.1 + net 26.2 + webproc 151.1)
10 MiB load + edit PSS	108.3 MiB (edit "BENCHMARKZ" verified by OCR)	282.6 MiB (edit verified by OCR)

Notable: Tauri already exceeds the 250 MiB gate-4 limit at 1 MiB; the 1→10 MiB delta is only ~23 MiB because WebKitWebProcess runtime (~151–189 MiB) dominates over document size.

Interaction and native behavior (recorded)

• Typing: both accept input (verified by OCR: "BENCHMARKZLarge Document", "HELLO").
• Scroll: PageDown works in both (gate 6 timings).
• Undo/redo: Floem Ctrl+Z/Ctrl+Y verified via typed text disappearing/returning. Tauri CodeMirror provides mod-z/mod-y by default (defaultKeymap); manual OCR verification was inconclusive due to focus/geometry noise.
• Selection/clipboard: Floem not conclusively verifiable via OCR; Tauri CodeMirror ships mod-c/mod-v by default.
• Keyboard reachability (a11y finding): Tauri without autofocus cannot reach the CodeMirror editor via Tab (focus stops at titlebar buttons); Floem's editor has initial focus.
• IME, context menu, Spellcheck suggestions: not verifiable in this environment (no IME composition events possible via wtype; right-click needs a working pointer synthesizer — ydotool moves the cursor but its button events are not delivered to windows in this session; xdotool/wf-recorder not installed).
• App scale: out of prototype scope (no App scale control in either prototype).

Raw evidence / limitations

• Scripts: /tmp/measure_startup.py, /tmp/run_startup.py, /tmp/measure_idle.py, /tmp/measure_gate4.py, /tmp/measure_gates67.py; screenshots in .scratch/ui-stack-bakeoff/screens/.
• Single samples except warm startup (7 runs each); external key dispatch is not end-to-end frame latency (same proxy as the Floem baseline).
• Hyprland 0.56: minimize not observable; tiled windows can't be dragged (drag verified by code path: drag_window_area / data-tauri-drag-region).

Maintenance cost comparison (initial)

• Floem prototype: single ~1,150-line Rust file; one binary; build cargo build --release ~10 s incremental; 14 compiler warnings in the prototype; no webview processes; no per-platform webview risk.
• Tauri prototype: Rust host (~5-line main + lib.rs) + SolidJS/TS frontend (App, Titlebar, WritingSurface/CodeMirror, DocumentSelector, Settings, tokens) + Vite + CodeMirror 6 + pnpm 11 (needs allowBuilds: { esbuild: true } in pnpm-workspace.yaml); frontend assets ship separately (not embedded by plain cargo build --release); WebKitGTK system dependency; 3-process runtime.

Verdict per spec

"A candidate fails if any applicable hard gate fails." Tauri fails gates 3 and 4 (memory) in both debug and release — the WebKitWebProcess alone costs ~133 MiB idle and ~189–212 MiB with a 10 MiB Document, a framework cost no build profile changes. Tauri cannot win the bake-off: it must pass every hard gate and be clearly better, and it is not acceptable on the resource budget. Floem passes all applicable hard gates; its gate 8 (titlebar) is as incomplete as Tauri's (no double-click) and additionally fails the visual integration contract of the current prototype.

Remaining before #54: Paper/Ink A/B screenshot set with identical dimensions/content (candidates anonymized), maintenance audit final pass, and the blind visual review itself.