Render the transcript natively: tables, code, HTML-lite, math, progressive streaming

[drumih]

Summary

Closes #86. The Mac transcript now renders what the model actually emits, instead of falling back to raw markdown whenever an answer contained a table and showing LaTeX as literal text.

Measured before this change, on three real Gemma 4 decodes: one markdown table anywhere sent the whole answer to raw text (two of three comparison-style answers contained one), code blocks had no visible container in dark mode, math was raw and the markdown pass mangled it (\\ collapsed inside cases/aligned, braces unescaped, \[ delimiters eaten), and the reader watched raw markdown until a restyle at the end.

• Tables render through NSTextTable from the intents Foundation's parser already produces: borders, semibold header row, per-column alignment, bold and <br> inside cells, native selection. The whole-message table fallback is gone.
• Code blocks get a single-cell container with proper intra-block spacing; inline code gets a fill that is visible in both appearances.
• Inline HTML replaces the old HTML regex gate: <br> becomes a line separator, <sub>/<sup> a baseline offset, <b>/<i>/<code>/<kbd> traits; an unrecognised tag keeps its text instead of vanishing (so Vec<String> survives). Block HTML still falls back. - [x] items render checkbox glyphs.
• Math: spans are lifted out before the markdown pass behind indexed private-use sentinels, with code regions masked first and \$ treated as a literal dollar inside a span (Gemma writes $\$0.60$). A normalizer rewrites only commands the pinned SwiftMath revision rejects — a test forbids approximating notation that already typesets. Equations rasterize once to an alpha mask and tint at draw time from the run's colour, so the same attachment is correct in light and dark, in headings and in quotes, with no appearance re-render. A failed equation stays visible as its raw source; a placement that cannot be verified falls back to the whole untouched message.
• Streaming styles the answer block by block: completed blocks render once and look exactly like the final render, while only the block still arriving re-renders each tick with its unterminated syntax closed for display (fence, **, backtick, trailing <tag/[link](). Unclosed inline $ is never auto-closed — currency ambiguity. Finalize is still one whole-answer render, pinned equal to the streamed concatenation.
• SwiftMath (MIT, pure Swift, CoreText) is linked to the Mac presentation target only, pinned by revision 1d2c908: tag 1.7.3 typesets 433 of a 607-string coverage sweep against that revision's 479, and 517 after normalization. The pinned sweep and its 90 recorded expected failures are checked in as the contract for any future pin move.

Two fixes ride along, each pinned by a test that fails on the old code: the streaming append path recomputed grapheme counts over the whole response every tick (tail/head tick-cost ratio 9.2 on a 10 KB answer, now 1.16 via UTF-8 offsets with a rebuild fallback at non-Character boundaries), and the HTML gate regex matched across newlines so Prove $a<b$ and $b>c$. forced raw fallback.

TURBO_FIELDFARE_DISABLE_MATH=1 and TURBO_FIELDFARE_PROGRESSIVE_RENDER=0 restore the previous behaviour.

Validation

• swift build, swift build --build-tests, swift build -c release --product TurboFieldfareMac — clean.
• swift test --filter TurboFieldfareMacPresentationTests — 243 tests in 20 suites pass: span detector, normalizer (including the checked-in 647-case coverage sweep), typesetter geometry and tint, renderer, block splitter, tail auto-close, controller, progressive-render identity, plus a frame harness that renders an 18-fixture corpus (three real Gemma decodes and synthetic corner cases: currency, shell dollars in fences and prose, CRLF, CJK, unclosed fence and unclosed $$, nested lists, task lists, table with <br>, heading and quote math, a link with $ in the URL, a 10 KB code block, a 50-equation answer) to PNGs in both appearances.
• Streaming cost gate: tail/head decile tick ratio 1.16 on the 10 KB code block and 1.28 on the 50-equation answer, against a limit of 3. Finalize 4.6 ms warm, one terminal render per response, bounded render cache.
• Real-model run of the release build (Gemma 4 26B, this branch): streamed at 22.9–29.0 tok/s with completed blocks styled live; finalize showed typeset display equations, a native table and literal $20/$30; light and dark both correct; Copy returned the raw markdown with LaTeX intact; cancel mid-equation left the partial $$… readable; clean exit.
• A multi-agent review of this work confirmed 18 defects, all fixed here with regressions verified to fail on the pre-fix code (normalizer stack overflow on deep nesting, a fence following prose without a blank line, a lone mid-message $$ un-styling the rest of an answer, CRLF defeating the detector, an O(k·n) dollar scan measured at 357.8 s on a 180 KB line and now 25.4 ms, nested-list markers, selection copy losing equations, mid-stream rebuild gluing completed blocks, and others).
• periphery scan --retain-public from this checkout: no unused declarations in the added code; the reported items on these files are Periphery not seeing Hashable/Equatable synthesis and Swift Testing's macro-generated entry points, plus pre-existing findings elsewhere in the tree.

Memory and performance

Not applicable to the model path: no runtime, importer, streaming-weights, Metal, tokenizer, server-bytes or cache-identity changes, and decode speed is unchanged (22.9–29.0 tok/s in the runs above). Rendering cost is bounded per block and measured above; equation bitmaps are equation-sized and held in a bounded cache.

Remaining limitations

• Math coverage is 517 of 607 LLM-shaped strings after normalization; the 90 recorded expected failures (for example CJK inside math, \tag, some environments) render as their raw source rather than typeset, never blank.

• A table nested inside a blockquote or list item deliberately falls back to raw text so the container context is not silently dropped; native nested-table rendering is future work.

• Finalize of a very long answer (about 59 KB) costs roughly 60 ms in a debug build; reusing the cached block renders at finalize is a planned follow-up.

• VoiceOver readout of equations was not exercised in this pass, though attachments carry their LaTeX as an accessibility description.

• CLI, server and iOS are unchanged by design: raw LaTeX is the correct representation on the wire and in a terminal.

• The change does not load a complete checkpoint, shard, or large model
  tensor into Swift heap memory.

• Logs and artifacts contain no credentials, private paths, or model
  weights.