Skip to content

fix(ios): Spacebar trackpad gesture support for Skia TextBox - #23909

Open
ajpinedam wants to merge 5 commits into
unoplatform:masterfrom
ajpinedam:dev/anpi/fix.ios.textbox.spacebar.trackpad
Open

fix(ios): Spacebar trackpad gesture support for Skia TextBox#23909
ajpinedam wants to merge 5 commits into
unoplatform:masterfrom
ajpinedam:dev/anpi/fix.ios.textbox.spacebar.trackpad

Conversation

@ajpinedam

Copy link
Copy Markdown
Contributor

GitHub Issue: closes #23871

PR Type: 🐞 Bugfix

What changed? 🚀

On Skia iOS, the software keyboard's spacebar trackpad mode (long-press space, then drag) did not move the TextBox caret: UIKit mutates the hidden proxy view's selection through internal controllers, which bypass the setSelectedTextRange: override that the existing native→managed selection sync relies on (#22533 / #22638), so the managed TextBox was never notified.

Three fixes, validated on a physical device (iPhone 14 Pro, iOS 26.5) with a Skia-renderer app:

  1. Sync trackpad selection to Skia TextBox — re-adds the textFieldDidChangeSelection: / textViewDidChangeSelection: delegate callbacks (originally introduced in fix: ios skia textbox arrow text navigation #22533, removed in fix: iOS input selection #22638 because they fired unguarded mid-keystroke) routed through a new InvisibleTextBoxViewExtension.OnNativeSelectionChanged() guarded against managed-initiated text/selection changes, IME composition, and in-flight native edits (native text ≠ managed text ⇒ ProcessNativeTextInput/SetPendingSelection owns the selection). The delegate callback observes every native selection change regardless of which internal UIKit path produced it.

  2. Track TextBox size on invisible proxy viewXamlRoot.InvalidateOverlays() lost its per-frame callers in an earlier rendering refactor, so the proxy's frame was only set at focus time. A TextBox growing while focused left the proxy wrapping its text at a stale width, making vertical trackpad movement cross the wrong lines. The extension now subscribes to TextBox.SizeChanged during entry and re-runs InvalidateLayout().

  3. Match invisible proxy wrap metrics to TextBox — trackpad horizontal movement is constrained to the proxy's layout lines, so wrap-point divergence from the rendered text creates invisible mid-line "walls". The off-screen proxy is now sized to the content element's text area (instead of the full control bounds) and adopts the TextBox's font size, keeping its line breaks aligned with the Skia layout.

Known limitation: the proxy lays out with the iOS system font at the matched size; apps rendering with a custom font can still see rare wrap divergence around long words at line boundaries. Projecting the Skia layout geometry through closestPositionToPoint:/caretRectForPosition: (as already done for firstRectForRange:) would remove that residual gap and is left as a follow-up.

PR Checklist ✅

  • 🧪 Added Runtime tests, UI tests, or a manual test sample (for bug fixes / features, if applicable) — manual-validation-only: the gesture requires the iOS software keyboard's trackpad mode on a device; validated manually (single-line and multiline, horizontal and vertical movement, typing/autocorrect/undo regression pass)
  • 📚 Docs have been added/updated following the documentation template (for bug fixes / features)
  • 🖼️ Validated PR Screenshots Compare Test Run results.
  • ❗ Contains NO breaking changes
  • 👀 Reviewed 2 other open pull requests (optional but appreciated!)

Copilot AI review requested due to automatic review settings July 29, 2026 18:06
@github-actions github-actions Bot added platform/ios 🍎 Categorizes an issue or PR as relevant to the iOS platform area/skia ✏️ Categorizes an issue or PR as relevant to Skia labels Jul 29, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes iOS (AppleUIKit) Skia TextBox caret/selection synchronization when UIKit updates the invisible native proxy’s selection out-of-band (notably via the software keyboard spacebar “trackpad mode”), and improves proxy layout fidelity while editing.

Changes:

  • Re-introduces native delegate selection-change callbacks and routes them through a guarded InvisibleTextBoxViewExtension.OnNativeSelectionChanged() to sync native selection back to the managed TextBox.
  • Keeps the invisible proxy view’s frame in sync with the focused TextBox by subscribing to TextBox.SizeChanged and invalidating layout.
  • Aligns proxy wrapping more closely with Skia layout by matching font size and sizing the proxy to the text-area rather than full control bounds.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/Uno.UI.Runtime.Skia.AppleUIKit/UI/Xaml/Controls/TextBox/SinglelineInvisibleTextBoxView.cs Exposes managed-driven text/selection mutation flags for selection-change guarding.
src/Uno.UI.Runtime.Skia.AppleUIKit/UI/Xaml/Controls/TextBox/SinglelineInvisibleTextBoxDelegate.cs Adds DidChangeSelection callback to forward native selection changes to the extension.
src/Uno.UI.Runtime.Skia.AppleUIKit/UI/Xaml/Controls/TextBox/MultilineInvisibleTextBoxView.cs Exposes managed-driven text/selection mutation flags for selection-change guarding.
src/Uno.UI.Runtime.Skia.AppleUIKit/UI/Xaml/Controls/TextBox/MultilineInvisibleTextBoxDelegate.cs Adds SelectionChanged callback to forward native selection changes to the extension.
src/Uno.UI.Runtime.Skia.AppleUIKit/UI/Xaml/Controls/TextBox/InvisibleTextBoxViewExtension.cs Adds guarded selection sync entrypoint, tracks TextBox size while editing, matches proxy font size, and sizes proxy to the text area.
src/Uno.UI.Runtime.Skia.AppleUIKit/UI/Xaml/Controls/TextBox/IInvisibleTextBoxView.cs Extends the abstraction with Font and managed-mutation flags needed for the new guarding logic.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings July 29, 2026 18:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

src/Uno.UI.Runtime.Skia.AppleUIKit/UI/Xaml/Controls/TextBox/InvisibleTextBoxViewExtension.cs:276

  • OnNativeSelectionChanged runs NativeTextEquals on every native selection change, and NativeTextEquals is O(text length) due to the full-string scan. For large TextBox contents, frequent selection updates (e.g., trackpad drag) could cause noticeable UI-thread jank; consider replacing this with a cheaper "native edit in flight" flag (set around ProcessNativeTextInput / ShouldChange* paths) or caching the last (nativeText, managedText) equality result.
		// While a native edit is in flight the native text is ahead of the managed text, so
		// selection offsets don't map onto it; ProcessNativeTextInput applies the post-edit
		// selection itself (see SetPendingSelection).
		if (!NativeTextEquals(GetNativeText(), textBox.Text))
		{
			return;
		}

		SyncSelectionToTextBox();

src/Uno.UI.Runtime.Skia.AppleUIKit/UI/Xaml/Controls/TextBox/InvisibleTextBoxViewExtension.cs:87

  • OnTextBoxSizeChanged doesn't use either parameter; with analyzers enabled this can surface IDE0060/unused-parameter warnings (the repo treats warnings as errors). Consider using discard-style parameter names to avoid analyzer noise.
	private void OnTextBoxSizeChanged(object sender, SizeChangedEventArgs args) => InvalidateLayout();

@unodevops

Copy link
Copy Markdown
Contributor

🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here: https://unowasmprstaging.z20.web.core.windows.net/pr-23909/wasm-skia-net9/index.html

@unodevops

Copy link
Copy Markdown
Contributor

⚠️⚠️ The build 224820 has failed on Uno.UI - CI.

Copilot AI review requested due to automatic review settings July 30, 2026 05:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

@unodevops

Copy link
Copy Markdown
Contributor

🤖 Your WebAssembly Skia Sample App stage site is ready! Visit it here: https://unowasmprstaging.z20.web.core.windows.net/pr-23909/wasm-skia-net9/index.html

@unodevops

Copy link
Copy Markdown
Contributor

⚠️⚠️ The build 224911 has failed on Uno.UI - CI.

@MartinZikmund

MartinZikmund commented Jul 30, 2026

Copy link
Copy Markdown
Member

@ajpinedam I have tested the PR but noticed this introduces a behavioral difference vs iOS apps, which is probably not avoidable with this approach - in native apps the gesture does not commit the caret position until the gesture ends (so it acts as a kind of "preview"). Additionally, with custom fonts it may start misbehaving and line breaks can also become problematic. I made #23920 based on the review of the Flutter sources overriding the related system hooks and that should match the iOS behavior a bit closer (plus adjusting the caret color to "selection highlight color")

Update: I ported the viewport scroll behavior from your PR, as that was not emulating correctly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/skia ✏️ Categorizes an issue or PR as relevant to Skia platform/ios 🍎 Categorizes an issue or PR as relevant to the iOS platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[iOS][Skia] Keyboard spacebar trackpad gesture (hold space to move cursor) does not move the TextBox caret

4 participants