Skip to content

[eric] browser: speed + robustness + learning tiers + UX (session work)#61

Merged
ciregenz merged 192 commits into
mainfrom
eric/dev
Jun 8, 2026
Merged

[eric] browser: speed + robustness + learning tiers + UX (session work)#61
ciregenz merged 192 commits into
mainfrom
eric/dev

Conversation

@ciregenz

@ciregenz ciregenz commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

What this is

A large body of browser-agent work (191 commits, 107 files, ~13.5k insertions) accumulated on eric/dev. Opening as a draft for review before anything is merged or released. This is NOT validated end-to-end on a clean machine yet (see caveats), so it should not go straight to the stable channel.

Themes

Browser agent speed + robustness (the bulk)

  • Converged the speed-lever A/B: shipped the two proven winners (think-shorter, merge-verify) unconditionally, deleted the dead-end flags (cheap-laps, compose-helper, send-prior all proved neutral/negative).
  • Completion detection: a confirmed/clean Send-class click ends the run instead of stalling to re-verify (kills the post-send dither); patient target_only confirm so a late-rendering send is not falsely "not confirmed".
  • Send-button handoff: poll the interactives list for the real Send button after a composer fill and hand the model its index, so it stops hunting via CSS/JS.
  • Message integrity: never leave a tool_use unanswered (mid-turn backfill + two-way validator) so a 30s upstream reset stops 400-killing the run.
  • WS resilience: ride out a momentary dashboard WS drop instead of failing the run under load.
  • Surrogate sanitizer: strip orphan UTF-16 surrogates from webview text so emoji pages do not crash a turn.

Learning tiers

  • Seed playbooks shipped for 30 popular sites (judgment, not mechanical replay) so a fresh install starts warm.
  • Tier 3 cross-site meta-playbook (site-agnostic priors) + tier 4 self-audit of the learning machinery (proposes for human review, never self-edits, fires every 25 tasks in a daemon thread).

UX

  • Agent writes a plain human final message (no UI jargon / OUTCOME mechanics leaking to users).
  • Fast-path emits a synthetic BrowserAgent tool-call/result so the "Browser Agent" bubble always shows, even when the orchestrator is skipped.
  • Electron: silenced the per-CDP-node ipc.handle log spam (OPENSWARM_DIAG_IPC=1 restores it).

Test state

  • Backend suite: 1178 passing.
  • The benchmark went from ~582s first-success to a ~5-6 turn / ~30-47s true-cold floor (with the shipped seed/meta priors), and the cold-with-memory floor at ~55-60s.

Honest caveats (why draft, why not straight to stable)

  • This machine was under heavy load (Wallspace + Spotlight) for most of the session, so several behavior/UI fixes (friendly final message, send-handoff firing, the fast-path bubble) are unit-green but not visually confirmed on a clean machine. They need one clean-machine pass.
  • Some A/B runs were load-confounded; the holistic bench chart labels session-2 wall times as load-confounded for that reason.
  • Two files remain oversized (agent_manager.py 4357 lines, browser_agent.py 2203 lines); a phased split is filed as deferred tech-debt. The import tree itself is a clean downward DAG.

Next

Review the diff, then (once merged to main) cut an experimental/prerelease build first to soak before any stable promotion, rather than shipping all of this to every user at once.

ciregenz added 30 commits June 2, 2026 03:55
ciregenz added 29 commits June 7, 2026 14:28
…send isnt falsely 'not confirmed' under load
…-only spin even when the send cant be auto-detected
…ll, and commit to an already-open thread instead of re-verifying
… learning machinery (tier 4, proposes for human review, never self-edits)
…+ validate both directions) so a 30s upstream reset stops 400-killing the run
… human confirmation; drop hardcoded task-specific examples from prompts
…ent translates it, no more 'learned skill replay (N steps, no LLM)' leaking to users
…n/OUTCOME mechanics); delete the leaky send-ready advisory, dead prefix flag, and humanize aux experiment
…ever the raw action-log proof (indices/coords)
…button after a composer fill, kills the re-list turn and the late-Send miss
…the Send index after a fill); deleting it brought the Send-hunt back, it was NOT redundant with the wait
…fter a composer fill so the handoff actually fires (was 0/10, racing the lazy render)
…ds per page read); OPENSWARM_DIAG_IPC=1 restores the firehose
…sult so the 'Browser Agent' bubble always shows, even when the orchestrator is skipped
@ciregenz
ciregenz merged commit 4e78925 into main Jun 8, 2026
14 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant