Skip to content

feat(scan): let a phone scan from the camera or the photo library - #578

Merged
alchaplinsky merged 4 commits into
mainfrom
feat/scan-from-camera
Oct 11, 2026
Merged

alchaplinsky merged 4 commits into
mainfrom
feat/scan-from-camera

Conversation

@alchaplinsky

Copy link
Copy Markdown
Member

Summary

On a phone, "Scan a card or document…" in the Add picker used to open the photo library only, so a card had to be photographed first and then picked. It now first asks where the photo comes from, in a bottom sheet shaped like the attachment source sheet of #577:

  • Take Photo opens the camera directly through a hidden <input type="file" accept="image/*" capture="environment">. The photo is a File with no path on disk, so it is sent to Rust as the raw IPC body and scanned by a new scan_image_bytes command.
  • Photo Library reuses the Rust-run pick_file image dialog (iOS shows the Photos picker for it), keeping the existing path grant.

Choosing either closes the sheet and the picker and hands over to the editor the scan fills. Backing out of the camera or the picker leaves the sheet open. Desktop is unchanged.

Changes

  • src/components/Main/Scan/SourceSheet.tsx: the sheet, mounted from Scan (after AddSecret in Main) behind a ui.scanSource flag so it stacks over the picker. It is not rendered inside the picker because the bottom-sheet card's translate would box in a fixed sheet.
  • src/components/Main/Scan/capture.ts: reads the captured file's bytes and runs the shared scan; run.ts now takes a path or bytes.
  • ActionRow moved to src/components/elements/ActionRow.tsx with the same content as feat(attachments): attach from the photo library on a phone #577, so the two merge as one move.
  • Rust: scan::scan_bytes and an Ocr::recognize_data with a Vision implementation (VNImageRequestHandler(data:)); bytes and recognized lines are zeroized after parsing. Other platforms report the bytes path as unsupported.
  • src-tauri/gen/apple/rowel_iOS/Info.plist: NSCameraUsageDescription, required for the camera option.
  • Strings: "Photo Library" and its translations copied from feat(attachments): attach from the photo library on a phone #577; new "Take Photo" and the sheet title in all ten locales. The old "Scan a photo…" mobile label is replaced by the desktop's "Scan a card or document…".
  • docs/threat-model.md: why scan_image_bytes needs no grant (the bytes are data the webview already holds; nothing on disk is read).

Testing

  • bun run test (1080 tests, incl. new src/test/scanMobile.test.tsx covering the sheet, both sources, cancel and hand-over), bun run lint, bun run typecheck.
  • cargo test scan, cargo check, cargo fmt --check.
  • A dry-run merge with feat(attachments): attach from the photo library on a phone #577 auto-merges everything except one adjacent-line conflict in icons.tsx.
  • Not verified on a device: the camera sheet needs a check on an iPhone build.

Decisions

  • adopted · implementation · [scan, compact-shell] — On the phone the scan row first asks the source in an in-app bottom sheet (Take Photo / Photo Library) shaped like the attachment sheet of feat(attachments): attach from the photo library on a phone #577. — Consistency with the attachment flow: one sheet shape and the system's names for sources.
  • adopted · implementation · [scan] — A camera capture is scanned from its bytes via scan_image_bytes with no path grant; a library photo keeps the granted pick_file dialog. — A capture has no path on disk; bytes the webview already holds let it read nothing new.
  • rejected · implementation · [scan, compact-shell] — Use WKWebView's own file-input sheet (Take Photo / Photo Library / Choose File) for the source choice. — Not consistent with the attachment sheet, and it adds a Files option that cannot be removed.
  • fact · implementation · [compact-shell] — A fixed-position dialog rendered inside a BottomSheet card is boxed in by the card's translate; stacked sheets are mounted from Main instead. — CSS translate establishes a containing block for fixed descendants.

On a phone the scan row first asks where the photo comes from, in a
bottom sheet shaped like the attachment source sheet (Take Photo /
Photo Library). Take Photo opens the camera through a capture input and
scans the photo from its bytes (scan_image_bytes, a raw IPC body with no
path to grant); Photo Library keeps the granted pick_file image dialog.
Desktop is unchanged. Adds NSCameraUsageDescription to the iOS plist.
@greptile-apps

greptile-apps Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the previous pending-photo issue is fixed.

Summary

This PR adds a camera-or-library choice for phone scans and sends camera photos to Rust as bytes.

  • Phone scans offer the camera and photo library as sources.
  • Camera photos now fill card and identity editors from their bytes.

Reviews (4) · Last reviewed commit: "fix(scan): drop a captured photo whose s..." · Reviewed by Greptile

Comment thread src/components/Main/Scan/capture.ts
Comment thread docs/threat-model.md Outdated
… as best effort

A camera photo whose bytes cannot be loaded now lands as a failed scan in
the status overlay rather than vanishing after the sheets have closed.
The threat model and the command's docs now say what is scrubbed: the
Rust working copy and the recognized lines, not the IPC body, Vision's
NSData or the webview's buffers.
Comment thread src/components/Main/Scan/capture.ts
The read of the capture's bytes is an await with nothing of the session
in hand. A lock while it runs (and an unlock of another workspace after)
used to let the finished read scan the photo into the new session's
editor. The app store now counts dropped sessions; the capture stamps
one before the read and discards the photo if it has moved.
@alchaplinsky
alchaplinsky merged commit c1b0511 into main Oct 11, 2026
14 checks passed
@alchaplinsky
alchaplinsky deleted the feat/scan-from-camera branch October 11, 2026 00:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant