Skip to content

feat(board): add Apple Pencil handwriting - #237

Draft
maxdu0016-gif wants to merge 2 commits into
vcmf:mainfrom
maxdu0016-gif:ipad-pencil-handwriting-pr
Draft

feat(board): add Apple Pencil handwriting#237
maxdu0016-gif wants to merge 2 commits into
vcmf:mainfrom
maxdu0016-gif:ipad-pencil-handwriting-pr

Conversation

@maxdu0016-gif

Copy link
Copy Markdown

Summary

Adds an initial Apple Pencil handwriting implementation to Dim0 as a discussion-ready prototype.

  • pressure-aware freehand ink nodes
  • coalesced and raw Pointer Events sampling
  • sparse-sample interpolation and smoothed rendering
  • whole-stroke eraser with hardware eraser detection
  • basic palm rejection and touch/pen separation
  • persistence, sync, conversion, and backend round-trip support
  • geometry and serialization tests

Architecture status

This is intentionally opened as a draft in its current Dim0-layer form so the implementation can be reviewed inline. Based on the maintainer discussion, the pen/eraser tool, ink geometry, built-in ink node, and preview path should ultimately move into canvas-harness as first-class engine features. Dim0 should then retain only toolbar/product defaults and backend conversion.

Validation

  • typecheck passed
  • lint passed
  • targeted ink and node-type tests passed
  • production build passed

Known limitation

Safari/PWA does not expose Apple Pencil double-tap. Native support would require an iPadOS bridge using UIPencilInteraction.

@winlp4ever

Copy link
Copy Markdown
Contributor

Thanks for opening this, and for the "Architecture status" note in the description. That's exactly the right framing, so let's lock the plan in here.

Reviewed the current commit end to end: _storedColors + style on the live insert is correct, the camelCase/snake_case seam is handled both ways, and typecheck/lint/tests/build all pass. Solid prototype.

Confirming the direction: the pen/eraser tool, ink geometry, and the built-in ink node move into canvas-harness as first-class engine features, and this PR slims down to the integration layer. Here's the file-level split so we're on the same page.

Move into canvas-harness (engine)

  • ink/ink-geometry.ts → the pure geometry: buildInkOutline, traceSmoothInkOutline, interpolateInkSamples, distanceToSegment, hitTestInkLocal/World, drawInkNode. (createInkNode / readInkProperty stay behind as dim0 glue, since they build the Note shape.)
  • ink/ink-input-layer.tsx → becomes a useInkTool hook wired into <Canvas> alongside useArrowTool, reusing the engine's pointer capture, store/palm-rejection.ts, and interaction-draft state. This removes the overlay and the canvasTool="pan" workaround.
  • ink/ink-pointer.ts → drop it. Palm rejection and eraser detection (buttons & 32) already exist in the engine; no need for a second copy.
  • node-types/ink/def.ts → a built-in ink node type in the engine (same shape as rect/ellipse/text).
  • ink-geometry.test.ts → travels with the geometry.

Stays here (dim0 integration)

  • harness-canvas.tsx → simplifies: pass tool="ink" + an inkDefaults prop, drop the overlay.
  • chrome/toolbar.tsx, store/board-app-store.ts → pen/eraser buttons, color/width popover, tool state.
  • convert/{node-to-note,note-to-node,node-type}.ts, types/note.ts, types/style.ts, newsfeed/types/properties.ts → Note ↔ Node mapping, NoteProperties.inkData, createDefaultStyle ink case, autofit set.
  • use-style-memory.ts exclusion + the Eraser icon.
  • All backend files (InkProperty pydantic, NodeType.INK, wire map, round-trip test) stay, since the backend never touches the engine.

Sequencing

  1. Land the canvas-harness PR first: built-in ink node + useInkTool + an inkDefaults prop (color/size factory, mirroring arrowDefaults), plus a creating-ink interaction mode drawing the live stroke on the interactive canvas.
  2. Cut a canvas-harness release.
  3. Rebase this PR down to the integration layer above. Keep it draft until the engine bits are in.

Two things to carry into the port (not fix here)

  • The input layer binds both pointermove and pointerrawupdate to the same handler, so pen samples get processed twice. Pick one stream in the engine version.
  • createInkNode persists both points and outline, and outline is derivable from points + size. Worth storing only points and rebuilding the outline on load, especially since all graph content lands in Qdrant.

Happy to pair on the canvas-harness side, or I can stub out useInkTool + the built-in node and hand it back to you for the geometry, whichever you prefer. Really like this feature, just want it one layer down so every canvas-harness user gets it.

@maxdu0016-gif

Copy link
Copy Markdown
Author

The canvas-harness engine PR is now open: winlp4ever/canvas-harness#39

It adds the built-in ink node, useInkTool, inkDefaults, live interaction-canvas previews, whole-stroke erasing, and reuses the existing palm-rejection helpers. I also incorporated both review follow-ups: only points + size are persisted (the outline is derived/cached), and pointermove + coalesced events is the single sampling stream, with no pointerrawupdate binding.

I’ll keep this Dim0 PR in draft until the engine PR lands and a canvas-harness release is available, then rebase it down to the integration layer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants