Skip to content

Draw: typed ink, Excel InkML and Draw with Trackpad (Force Touch) - #1187

Open
rustycawk wants to merge 11 commits into
storytold:mainfrom
rustycawk:draw-ink-trackpad
Open

rustycawk wants to merge 11 commits into
storytold:mainfrom
rustycawk:draw-ink-trackpad

Conversation

@rustycawk

@rustycawk rustycawk commented Oct 13, 2026 •

Copy link
Copy Markdown

Brings the Draw tab closer to Excel's. Ink gets a typed model with pressure, Excel's InkML file format, editable pens and erasers, and Draw with Trackpad with Force Touch pressure on macOS.

What changes for users

  • Pens: an editable pen gallery (pen, pencil, highlighter) where each pen has a colour, a width and an optional effect (rainbow, galaxy, gold…). Pens can be added and removed.
  • Erasers: a stroke eraser, a segment eraser that cuts at crossings, and small and medium erasers. Each eraser drag is one undo step.
  • Lasso: a freehand loop selects ink, which can then be moved or deleted.
  • Stroke width: width follows pressure when the input reports it, and drawing speed otherwise.
  • Fast strokes keep their shape, like Excel:
    • a stroke takes every pointer move and keeps following the pointer past the cells (over the headers or out of the window); the grid clips what lies outside;
    • smoothing only settles jitter (at most 1.5 px) and uses a centripetal spline through the samples, so a fast zigzag keeps sharp, full-height turns instead of curling into loops;
    • on macOS, while a Draw tool is on, mouse drags aren't coalesced, and a hard press on the trackpad mid-stroke doesn't force-click.
  • Draw with Trackpad: a floating canvas over the sheet. One finger draws, two fingers move the canvas, and any key exits.
    • On macOS, a light touch draws the thinnest line; clicking the pad down and pressing harder thickens it. It uses NSPressureBehavior.primaryGeneric, so there is no force click or Look Up.
    • Backends: macOS (AppKit touches), Linux (evdev) and Windows (precision-touchpad HID). The device is probed off the main thread and opened only when the mode starts.
  • Files: XLSX ink is saved as Excel InkML behind mc:AlternateContent, with a DrawingML fallback for readers without InkML. Plain DrawingML polylines from Preserve editable ink strokes across XLSX saves #712 still open as editable ink.
  • Print and PDF: highlighter and pencil ink print translucent, and effect strokes print in their colours.

Commits

One commit per layer, in dependency order: model → PDF → XLSX → engine → UI → app (trackpad backends) → docs. Each commit passes cargo check --all-targets for the crates it touches. The whole branch passes cargo xtask ci (fmt, clippy, test, assets, layers, wasm, parity, keytips).

Notes for review

  • macOS UI freeze when the trackpad mode starts (fixed). Draining touch events with nextEventMatchingMask in the default run-loop mode inside winit's redraw handler re-entered winit's and Core Animation's observers. nextDrawable then hit its 1 s timeout. Touch events are now drained in a private run-loop mode.
  • Known limitation: Force Touch pressure lags a little. winit dispatches pressure events between frames, so a per-frame poll mostly sees NSApp.currentEvent. Seeing every event would need addLocalMonitorForEventsMatchingMask:handler:, which is an unsafe fn in objc2, and the workspace forbids unsafe. A new finger identity always starts a new stroke, so a missed lift doesn't join two strokes.
  • Not yet tested on physical Linux and Windows touchpads. Those backends have synthetic-input tests only.
  • Shared crate: the app-independent parts (ink model, InkML, trackpad capture, Draw widgets) are proposed for craft-libs in RFC: a shared Draw stack for the Craft apps (craft-ink, craft-trackpad, Draw components in craft-ui) craft-libs#99.

Testing

  • Unit and UI tests:
    • ink model, InkML import and export, and the DrawingML fallback, all with hostile-input bounds;
    • segment eraser and lasso;
    • smoothing keeps a fast zigzag's turns, and a stroke follows the pointer past the cells (both tests fail without the fix);
    • trackpad frames, including Force Touch press and a press in place;
    • native AppKit mouse events never reach the touch or pressure accessors.
  • Manual testing on a MacBook with a Force Touch trackpad:
    • the mode starts without a UI freeze;
    • touch strokes are thinner than clicked ones, and pressing harder thickens the line;
    • a fast swipe off the edge no longer joins strokes.
    • fast zigzags drawn by clicking and dragging on the trackpad now match Excel's (turns run the full height, no loops).

Ink keeps its points, per-point pressure, pen (pen, pencil, highlighter), width and colour
effect in Shape::ink instead of x,y text; old text strokes upgrade on load. Outline and
triangle geometry, opacity and effect colours live with the model.
fill_alpha registers one ExtGState per opacity, so highlighters and pencils print
translucent.
Ink saves as InkML parts behind mc:AlternateContent, with a DrawingML path (and a
urn:gridcraft:ink:v1 extension keeping pen, pressure and effect) for readers without
InkML. Reading is bounded (points, traces, nodes, a workbook-wide point budget) and
never follows external parts. Plain DrawingML polylines from other files still open as
editable ink (storytold#551), now in the typed model; ink left of column A or above row 1 keeps
its place across saves.
Editable pen gallery (pen, pencil, highlighter; colour, width, effects), stroke and
segment erasers (segments cut at crossings, one undo step per drag), freehand lasso
select/move/delete, smoothed strokes with pressure or inverse-speed width, and ink in
printed PDFs.
The Draw ribbon (pen gallery and settings, colours and effects, eraser modes, lasso,
trackpad switch), ink painted as the saved stroke (translucent and effect strokes), and
the floating trackpad canvas fed by TrackpadFrame: one finger draws, two move the canvas,
any key exits. Force Touch press maps to stroke force (touch thinnest, click and press
thicker). Labels in every catalog, including Serbian.
macOS AppKit indirect touches, Linux evdev, Windows precision-touchpad HID; the device is
probed off the main thread and opened only when the mode starts. On macOS touch events
drain in a private run-loop mode (draining the default mode inside winit's handler froze
the UI for 1-2 s), the pad uses NSPressureBehavior.primaryGeneric for continuous stage-1
pressure without a force click, and a new finger identity starts a new stroke.
The two [1,2,1] passes now move a point at most 1.5 px, and the curve is a centripetal
Catmull-Rom spline through every sample (points every ~3 px on long segments), so a fast
zigzag keeps its full-height turns instead of curling into loops.
A stroke reads every PointerMoved of the frame (times spread across it) and keeps following
the pointer over the headers or out of the window, like Excel; the grid clips what lies
outside. On macOS, while a Draw tool is on, mouse drags aren't coalesced and the pad uses
generic pressure, so a hard press mid-stroke doesn't force-click.
The precision-touchpad descriptor reader spells out what it reads: usages by name and page
(CONTACT_COUNT is Digitizer 0x54), item types and tags as in HID 1.11 §6.2.2, named bounds,
and a FingerState instead of a four-slot array. Tests build their descriptors item by item.
No behaviour change.
@alosslessdev

Copy link
Copy Markdown

Superseded Draw-tab PRs

We are closing the small Excel-parity Draw PRs in favour of this work. The issues they targeted should be tracked here:

This PR replaces the fixed-pen Draw tab with the editable pen gallery and shares stroke_shape() between the live preview and draw.stroke, so the preview/commit mismatch those PRs patched no longer exists in the same form. If this PR lands, those issues are covered by it.

Still separate and not covered here:

Round-trip evidence for this PR's InkML work

Excel 16.0 (Build 20430) writes a Draw-tab stroke as xl/ink/ink1.xml (InkML with X/Y in 1/1000 cm and first/second differences) referenced by mc:AlternateContent/xdr:contentPart, plus a PNG fallback. On current main, GridCraft imports only the fallback (gridcraft-cli info: images: 1, shapes: 0), so the stroke is not editable ink. crates/xlsx/src/inkml.rs is the fix; it would be valuable to land that part even if the trackpad/pen-gallery work is split.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants