HANDOFF: Terminal-native typst preview for Kitty-protocol terminals

A live typst preview rendered directly into a Neovim window inside the host terminal (Ghostty, Kitty, WezTerm, foot), with keystroke-to-pixels latency within striking distance of chomosuke/typst-preview.nvim (browser path).

No browser, no external window, no file watcher.

Goal

Deliver a Neovim plugin whose preview UX feels indistinguishable from typst-preview.nvim while running entirely inside the terminal. Target P50 keystroke-to-pixels: 40-60 ms under Ghostty on a modern laptop.

Why it doesn't already exist

Not a physics problem — an engineering gap.

The browser path is path-of-least-resistance: typst already emits SVG, browsers already render SVG with patches via DOM diffing, WebSocket framing is trivial. A terminal path needs a custom rasterizer pipeline and a Kitty-graphics-protocol encoder where today there is nothing.

The terminal protocols (Kitty graphics protocol, Sixel) are also fragmented: covering "every terminal" is hard, so building "browser" stayed the default. This project chooses to ignore non-Kitty-protocol terminals.

Latency budget

Stage	Browser path	Terminal path
Keystroke → buffer text → IPC	~1 ms	~1 ms
typst incremental compile delta	1-10 ms	1-10 ms
Serialize delta (SVG vs PNG/zlib/base64)	1-2 ms	3-10 ms
Transport to renderer	1-2 ms (WS)	<1 ms (PTY)
Renderer decode + present	5-15 ms (browser)	5-20 ms (Ghostty: base64→zlib→GPU upload→blit)
Vsync quantization	0-16 ms	0-16 ms
Total	~10-50 ms	~15-60 ms

The terminal path's extra ~15-25 ms comes from rasterization + image-encoding. Region-of-interest rendering closes most of it.

Architecture

Three components, one Unix-socket protocol.

┌──────────────┐    AF_UNIX     ┌───────────────────────┐    PTY (TIOCSTI/stdout)
│ nvim plugin  │ ←------------→ │  preview-server (Rust)│ ←------------------------- terminal
│ Lua, ~150 LOC│   JSON-lines   │  ~800 LOC             │   Kitty graphics escapes
└──────────────┘                └───────────────────────┘
        ↑                                   ↑
        │                                   │
   buffer changes                       typst-ide
   cursor moves                       tiny-skia raster
   window resizes                     kitty-image encode

Component 1: `typst-term-preview` (Rust binary)

Responsibilities

Hold a long-lived typst::Document and incremental compile state.
Apply source edits (range-based, character-precise) received over the socket.
Re-layout incrementally; query dirty-region info from typst-ide.
Rasterize affected page region via tiny-skia at the requested DPI.
Emit Kitty graphics protocol escape sequences via stdout.
Accept positioning hints (column/row in cells) from the plugin.

Crates

typst — compiler core.
typst-ide — incremental rebuild + dirty-region reporting.
tiny-skia — rasterizer (already a typst dep).
kitty-image (https://crates.io/crates/kitty-image) — Kitty graphics protocol encoder. (If unsuitable, hand-roll: format documented at https://sw.kovidgoyal.net/kitty/graphics-protocol/.)
serde_json / tokio — IPC.

Protocol (JSON lines over AF_UNIX)

// Plugin → server
{"type":"open","path":"/abs/path.typ","content":"<full text>","root":"/abs/root"}
{"type":"edit","start":{"line":12,"col":4},"end":{"line":12,"col":5},"text":"x"}
{"type":"viewport","cols":80,"rows":48,"cell_px":{"w":9,"h":18},"placement":{"x":120,"y":0}}
{"type":"close"}

// Server → plugin
{"type":"frame","page":2,"emitted_image_id":7,"compile_ms":4}
{"type":"diag","severity":"error","range":{...},"message":"..."}
{"type":"error","message":"<server-side fatal>"}

The frame event is purely advisory — the actual pixels reach the terminal directly via the server's stdout, not through the plugin. The plugin's PTY is inherited by the spawned server process, so escape sequences land in the host terminal naturally. The image is positioned using Kitty graphics protocol's x_offset / y_offset controls relative to a stable cursor anchor that the plugin parks in the preview window.

Component 2: Neovim Lua plugin

Responsibilities

On BufRead *.typ / FileType typst: spawn the server (singleton per Neovim instance), open the AF_UNIX socket, send open.
On TextChangedI / TextChanged: compute the minimal range delta against a snapshot, send edit.
On WinResized and at startup: compute the preview window's pixel rectangle (cell size × dimensions) and send viewport.
Maintain a preview window: a vertical split holding an empty [Preview] buffer with buftype=nofile. Position the cursor at the top of that window; the server's image is placed at that screen cell.
Handle server lifecycle: respawn on crash, kill on :q!.

Files

plugin/typst-term-preview.lua    -- bootstrap, autocmds
lua/typst_term_preview/init.lua  -- public API
lua/typst_term_preview/server.lua -- AF_UNIX client
lua/typst_term_preview/window.lua -- preview window placement
lua/typst_term_preview/delta.lua  -- text-change → edit op

Component 3: Kitty graphics protocol layer

This is encapsulated in the Rust server, but worth calling out:

Use image IDs to keep one "preview image" alive in the terminal across frames. Replace it with a=T,i=<id>,q=2-style commands instead of re-uploading from scratch every frame, when payload size permits.
For multi-page docs, allocate one image ID per page and pre-upload all pages on open; subsequent frames only re-upload changed pages.
Use f=100 (PNG) for first transmission, f=24/f=32 (raw RGB/RGBA) for delta uploads when the delta region is small (skips zlib).
Position images with C=1 placement and cursor anchoring; coordinates are in terminal cells, NOT pixels. The plugin computes cell positions and sends them; the server emits the escape with those values.

Implementation phases

Aim for working-end-to-end before optimization. Each phase produces a runnable demo.

Phase 0: Spike (1 day)

Static .typ file → typst::compile once → tiny-skia raster of page 1 → write PNG → emit Kitty graphics protocol escape via a small Rust binary to print the image in the terminal.
No Neovim integration yet. Just: typst-term-preview demo.typ shows the page in the terminal.
Exit criterion: an image appears in Ghostty when the binary runs.

Phase 1: Incremental compile loop (2-3 days)

Wire typst-ide for incremental edits. Accept edits on stdin (raw, no socket yet), re-render, re-emit.
Use the same image ID for every frame so the displayed image swaps in place.
Exit criterion: typing cargo run --bin typst-term-preview demo.typ <<< '... text deltas ...' updates the on-screen image without flicker.
Latency target: P50 < 80 ms (whole-page raster, no ROI yet).

Phase 2: AF_UNIX IPC + minimum-viable plugin (2 days)

Move stdin to AF_UNIX. Lua plugin that spawns server, opens socket, sends one edit per TextChangedI.
Static placement: preview window is a :vsplit with the image rendered at its top-left corner.
Exit criterion: typing in a .typ buffer in Neovim updates the preview in the adjacent split. End-to-end without browser.

Phase 3: Region-of-interest rasterization (3-5 days)

Query typst-ide for the dirty layout regions after each edit. Rasterize only those regions. Upload as Kitty-protocol delta frames over the background image ID.
Exit criterion: P50 keystroke-to-pixels < 50 ms on a 5-page document when editing a single paragraph.

Phase 4: Polish (2-3 days)

Multi-page navigation: <leader>pn / <leader>pp to scroll pages.
Window-resize handling: recompute viewport, re-render at new DPI.
Server crash recovery: plugin auto-respawns.
Error surfacing: typst diagnostics piped into vim.diagnostic.
:checkhealth typst_term_preview for terminal-protocol detection.

Total: ~10-15 engineer-days for first working version.

Risks and open questions

Risk: Kitty graphics protocol idiosyncrasies across implementations

Ghostty, Kitty, WezTerm, and foot all implement the protocol, but with divergent edge cases — image deletion semantics, placement origin, virtual cell behavior. Test matrix needs all four early. Bias toward the intersection of supported commands.

Mitigation: a small compat.rs module that gates feature use by detected terminal (via $TERM + Kitty's terminal-identification query).

Risk: typst-ide's incremental API stability

typst-ide is 0.x and reshuffles between minor versions. Dependency pinning required; expect to track typst releases manually.

Risk: Cursor anchoring vs. Neovim's window redraw

Neovim repaints the preview-window cells around the image, which can erase or scroll the image depending on terminal protocol semantics. Using Kitty's "virtual placement" with U=1 (unicode placeholder) and a buffer of Z U+10EEEE cells in the preview buffer is the canonical fix. The plugin must populate the preview buffer with placeholder characters.

Open: full-page PNG vs raw RGBA delta

Phase 3 must benchmark both. PNG is smaller (cheaper IPC) but slower to encode. Raw RGBA is heavier but lets the encode step disappear.

Open: multi-buffer / multi-document

V1 should be single-buffer. Multi-buffer adds session-management complexity (which document is the preview tracking?) without a clear UX answer.

Open: search / scroll affordances

A typst preview is not just an image — users want to click-to-jump, scroll, zoom. V1 punts on this; phase 4 adds keyboard scroll, but click-jump requires reverse-mapping from preview pixel to source position (typst-ide exposes this).

Non-goals

Sixel support. Kitty graphics protocol only.
Non-typst document types (markdown, AsciiDoc, etc.).
Terminals without Kitty graphics protocol support (tmux without the passthrough patch, generic xterm, GNU screen).
Running the server over SSH / on a remote machine. Local Unix socket only for v1.
Fancy UI chrome (page numbers, mini-map). Image only.

Comparison with existing solutions

	typst-preview.nvim	tinymist preview	This project
Renderer	Browser (SVG)	Browser (HTML/SVG)	Terminal (Kitty graphics)
Latency P50	~30 ms	~50 ms	target ~40-60 ms
Window count	Browser + Neovim	Browser + Neovim	Neovim only
Terminal coverage	Any (browser)	Any (browser)	Kitty-protocol only
Streaming updates	WebSocket	WebSocket	Kitty graphics protocol image-replace

Reference material

Kitty graphics protocol spec: https://sw.kovidgoyal.net/kitty/graphics-protocol/
Ghostty's image support docs: ghostty repo docs/features/graphics.md
typst-ide source: https://github.com/typst/typst/tree/main/crates/typst-ide
typst-preview.nvim (reference architecture for the browser path): https://github.com/chomosuke/typst-preview.nvim
kitty-image crate: https://crates.io/crates/kitty-image
Unicode placeholder rendering trick (Neovim integration): Kitty docs section "Unicode placeholders".

Quick start (when picking this up cold)

Read the Kitty graphics protocol spec end-to-end. It is the load-bearing piece of unfamiliar knowledge.
Skim chomosuke/typst-preview.nvim's repo to understand the existing IPC shape — this project mirrors it, swapping browser for Kitty.
Run phase 0 in an afternoon to verify the terminal-image path works on the target hardware.
Decide phase 1's incremental hookup strategy by reading typst-ide's query() + lib::compile() sources for current version.
Implement bottom-up. Plugin glue last.

If P50 latency in phase 1 is over 150 ms whole-page, abandon — something about the rasterizer or protocol path is slower than expected and the project's premise needs revisiting. Otherwise proceed to phase 2.

FilesExpand file tree

HANDOFF.md

Latest commit

History