feat(hints): opt-in auto-refresh that keeps hints live on macOS

[gabrielecirulli]

Disclaimer: Both this code and this PR description were written by AI. I still need to review this description to make sure it is fully correct, and I will do that soon.

Draft: this branch also needs review and testing before it moves forward.

Rationale

Hints are scanned once when hints mode opens, so they go stale when the focused window changes afterward: a page finishes loading, a menu opens, a panel appears. This wires the AX observer service (#1017) to a new opt-in setting that re-scans hints when the focused app's UI changes, with no keypress. macOS only for now; on other platforms the observer backend is a stub and the setting is inert.

What this adds

The setting. [hints.auto_refresh] has two keys: enabled (off by default) and debounce_ms (how long a burst of changes settles before the re-scan, default 150). The set of accessibility notifications that wake a re-scan is not configurable: the observer watches a fixed DefaultMask defined in the axobserver package (#1017), which covers the structural notifications plus the web-content signals browsers post when a page changes, and omits value_changed because it fires continuously.

Debounce and coalescing. The refresh paths that re-enter through activateHintModeInternal (a --repeat re-entry, a modifier-passthrough re-scan, a bound hints re-launch, a cycle-hint re-scan, and the debounced observer) coalesce there, as Handler state rather than a separate coordinator:

• Leading edge: the first refresh in an idle burst scans immediately, so a manual refresh is never delayed.
• Trailing edge: refreshes arriving during the debounce window collapse into a single follow-up scan.
• Max-wait cap: a continuously-changing app still refreshes at a bounded rate.

While a search query is being typed, a refresh is held and released when the search ends, so the query and its filtered hint set are not swapped out from under the keystrokes. A pending hint-label selection is not held, because the --repeat re-scan right after a label is chosen must proceed or the overlay freezes on the stale filter.

Backoff until the page settles. An observer-driven scan can catch a web page mid-render and label only part of it. Browsers post a notification when content starts changing but none when it finishes, and single-page sites load content in waves a second or more after the click. So after each observer-driven scan, neru keeps re-scanning on a capped exponential backoff. Each re-scan fingerprints the resulting hint set (a hash of element bounds and roles, from hints the scan already produced, so it costs no extra accessibility calls). If the set changed, the interval restarts dense (250ms); if it is stable, the interval grows by 1.6x up to a 5s ceiling; the loop stops after two consecutive stable scans at that ceiling. Two ceilings bound a page that never settles: the scan count (25) and the elapsed window (30s) since the sequence began do not reset on change, so a continuously-changing live page winds down to dormant rather than re-scanning forever, until the next observer event starts a fresh sequence.

A manual or --repeat refresh takes priority over a running settle loop: it ends the settle and scans immediately as the leading edge, so selecting a hint while the settle is mid-flight is never deferred behind a pending re-check.

Safety. The observer callback runs on the accessibility thread and touches only a leaf mutex, never the mode lock, so it cannot deadlock a teardown that joins that thread. Observers arm on the focused app while hints are open and are disarmed on exit, so an idle neru holds no accessibility resources.

Tests

Cover the debounce (leading, trailing, max-wait, hold-during-search, no-hold-for-label), the backoff interval growth and clamp, the stop conditions (two stable at the cap, the scan-count ceiling, the window ceiling), the state machine, and a manual refresh pre-empting the settle.

Notes

This sits on top of the merged label-persistence fix (#1012). A stable re-scan that changes nothing redraws through the same incremental path, so it is not visible.

Testing the whole feature

This is the tip of the two-part stack (#1017 then this) and sits on top of main, so it is the branch to run to test the feature end to end. Pull and run feat/hints-auto-refresh-config (#1018).

macOS only. Set this in your neru config:

[hints.auto_refresh]
enabled = true          # re-scan hints when the focused app's UI changes

hints.auto_refresh.enabled = true is the only switch the feature needs. It refreshes native windows, menus, and browser and Electron web-page content alike: neru wakes the focused app's accessibility tree on focus, so page-content changes reach the observer without any extra setting. debounce_ms can stay at its default.

Then open hints mode and watch the labels update as the focused app changes, a page finishes loading, a menu opens, or a panel appears, with no keypress.

Potential improvements

• The fingerprint hashes exact element bounds, so one continuously-animating element on a page keeps the backoff pinned dense until the 25-scan ceiling instead of settling. Quantizing bounds to a coarse grid before hashing would let such a page settle early.
• On multi-monitor setups the re-scan resolves the target screen from the cursor location, the same as a fresh activation. Moving the cursor to another display during a settle can filter the window's hints off screen for that scan. Pinning a refresh to the screen the session started on would fix it.

Merge order

This PR and the observer PR are a two-part stack and must merge in order:

1. feat(axobserver): add push-based AX observer service #1017 — feat/ax-observer-service — push-based AX observer service (inert)
2. feat(hints): opt-in auto-refresh that keeps hints live on macOS #1018 — feat/hints-auto-refresh-config — opt-in hints auto-refresh ← this PR

Both target main, so until #1017 merges, this PR's diff also shows its commits.

Two earlier PRs from the original series have been closed: #1015 (show hints in any Electron app without whitelisting) and #1016 (rename hints.additional_ax_support to hints.web_content_hints). Upstream's bundle-type auto-detection (#1035) removed the additional_ax_support / web_content_hints machinery they depended on, so both became unnecessary; this branch was rebased directly onto main without them, and the feature no longer needs any browser-specific setting to reach web-page content.

[gabrielecirulli]

@greptileai ?

[greptile-apps]

Greptile Summary

This PR adds opt-in live hint refresh on macOS. The main changes are:

• Adds debounced AX observer-driven hint rescans.
• Adds bounded settle retries for asynchronously rendered content.
• Adds auto-refresh configuration, validation, defaults, and documentation.
• Renames browser accessibility settings to hints.web_content_hints.
• Adds macOS observer bridge support and non-macOS no-op behavior.

Confidence Score: 5/5

This looks safe to merge.

No blocking issues found in the changed code.

T-Rex Logs

What T-Rex did

• Blocked the focused-run attempt, recording the exact command, working directory, full compiler output, and exit code 1 in hints-auto-refresh-modes-01-before.log.
• Validated the approach with a passing fallback: the hints-auto-refresh-02-after.log records the exact command, working directory, complete verbose output, and exit code 0, and relevant auto-refresh validation tests passed at lines 512–521.

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
internal/app/modes/auto_refresh.go	Adds serialized debounce, typing holds, settle backoff, and observer refresh handling for hint mode.
internal/app/modes/hints.go	Routes in-mode hint refreshes through the new auto-refresh debounce gate.
internal/core/infra/axobserver/platform_darwin.go	Adds the Darwin observer adapter and maps configured notification masks to native AX observers.
internal/core/infra/platform/darwin/axobserver_darwin.m	Implements native AX observer registration, callback delivery, and run-loop teardown.
internal/config/service.go	Rejects renamed legacy accessibility configuration keys instead of silently ignoring them.
internal/core/infra/electron/electron.go	Consolidates per-application accessibility activation and PID-aware attribute caching.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant AX as AX observer
participant D as Debounce timer
participant H as Hint handler
participant S as Hint scanner
AX->>D: UI notification
D->>H: fire after debounce
H->>S: refresh hint collection
S-->>H: updated hints
H->>D: schedule settle recheck
D->>H: bounded backoff refresh

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant AX as AX observer
participant D as Debounce timer
participant H as Hint handler
participant S as Hint scanner
AX->>D: UI notification
D->>H: fire after debounce
H->>S: refresh hint collection
S-->>H: updated hints
H->>D: schedule settle recheck
D->>H: bounded backoff refresh

Loading

_{Reviews (1): Last reviewed commit: "feat(hints): re-scan auto-refresh with b..." | Re-trigger Greptile}