Skip to content

feat: 'brow read' — clean text/content extraction (agents bypass snapshot for html+eval) #29

Description

@detrin

Problem

Analysis of 5,505 real brow calls across 11 agent projects shows agents overwhelmingly avoid snapshot for content extraction and fall back to raw html + hand-written parsers:

  • brow html: 1,241 calls — 99% with no --search/--locator (full-page DOM dumps)
  • Of those, 728 piped to python, 389 to grep
  • brow eval: 1,173 calls — 418 do inner_text/textContent, 92 json.dumps, 172 write a temp file

Root cause

SNAPSHOT_JS (routes/browser.py) is an interaction map, not a content reader. Text is captured only for unnamed nodes and truncated to 80 chars; there's a hard NODE_LIMIT of 200–400 and repeated items are collapsed. So whenever an agent needs prose/prices/embedded JSON, snapshot drops it and they rebuild extraction by hand every time.

Real example (what agents actually do)

brow html -s 74 | python3 -c "
import sys,re,html
t=sys.stdin.read()
for m in re.findall(r'\"commentary\".*?\"text\":\"(.*?)\"', t):
    print(html.unescape(m)[:3000])
"

Comments in the wild: "look for the post commentary text", "events have name+start_at+city in embedded json", "find sentences with market size / CAGR / USD / billion".

Proposal

Add brow read [--locator <sel>] [--format text|markdown] returning readability-style clean main content (text or markdown), dropping nav/scripts/styles. This replaces the dominant ~2,400-call html+eval pattern and is a big token win.

Tasks to work on

  • /browser/{sid}/read route + JS extractor (or Python-side html2text/readability)
  • brow read CLI command, --locator, --format
  • Tests: data-table page, article page, embedded-JSON page
  • README: position alongside snapshot ("snapshot to act, read to extract")

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions