From a2fd7b17f4c9eba89ae9c4d951f0aa81468f7d7f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Victor=20Gonz=C3=A1lez-Pacheco?= Date: Sat, 4 Jul 2026 09:03:51 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20add=20user=20manual=20=E2=80=94=20tutor?= =?UTF-8?q?ial=20walkthrough,=20digest-video=20example,=20troubleshooting?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The README covered install/config/command-reference but had no narrated end-to-end walkthrough, no worked digest-video example, and no troubleshooting. - docs/tutorial.md — zero-to-wiki walkthrough (sync → vocab/enrich/topics → media/describe → digest-video → dashboard), with expected output per step. - docs/digest-video.md — worked example (prereqs, run, the "## Video digest" note shape, model selection, video selection). - docs/troubleshooting.md — FAQ: expired X session, the re-saw-0 tripwire, rate limits, parakeet/ffmpeg PATH (incl. cron/launchd minimal-PATH), digest-video slowness/timeouts, silent-video "sin voz", iCloud generate hang, API keys. - README: link the three from Documentation + a "New to XBrain?" pointer at Quick start. Co-Authored-By: Claude Opus 4.8 (1M context) --- README.md | 7 +++ docs/digest-video.md | 111 ++++++++++++++++++++++++++++++++++ docs/troubleshooting.md | 110 ++++++++++++++++++++++++++++++++++ docs/tutorial.md | 128 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 356 insertions(+) create mode 100644 docs/digest-video.md create mode 100644 docs/troubleshooting.md create mode 100644 docs/tutorial.md diff --git a/README.md b/README.md index 4a54fe3..9d83bd1 100644 --- a/README.md +++ b/README.md @@ -324,6 +324,10 @@ uv run xbrain status # see the counts `sync` builds the mechanical layers. The LLM layers (`vocab`, `enrich`, `topics`) are run explicitly — see [The pipeline](#the-pipeline). +> **New to XBrain?** The [**Tutorial**](docs/tutorial.md) walks through the whole +> thing end-to-end — pull your posts, add topics, download + describe media, and +> digest a bookmarked talk — with the output you should see at each step. + --- ## Prerequisites @@ -1211,6 +1215,9 @@ client. Respect X's Terms of Service. | Document | Description | |----------|-------------| +| [docs/tutorial.md](docs/tutorial.md) | **Start here** — end-to-end walkthrough from install to a searchable wiki. | +| [docs/digest-video.md](docs/digest-video.md) | Worked example: turn a bookmarked talk into transcript + slide notes. | +| [docs/troubleshooting.md](docs/troubleshooting.md) | Common failures & fixes (auth, PATH, digest-video, iCloud). | | [ARCHITECTURE.md](ARCHITECTURE.md) | How XBrain is shaped: pipeline stages, artifacts, rubrics, executors, invariants. | | [CONTRIBUTING.md](CONTRIBUTING.md) | How to contribute — including PRs written with AI agents. | | [LICENSE](LICENSE) | MIT. | diff --git a/docs/digest-video.md b/docs/digest-video.md new file mode 100644 index 0000000..e2dbab6 --- /dev/null +++ b/docs/digest-video.md @@ -0,0 +1,111 @@ +# `digest-video` — turn bookmarked talks into readable notes + +`digest-video` manufactures **text** from a video so it flows through the normal +enrich → topics → generate pipeline like any other post. For each selected video +it does an **ephemeral** fetch, transcribes the audio with an external local +transcriber, attaches the transcript as an `x_video` content source, and +**discards the bytes** (the corpus never lands on disk). `--frames` adds a visual +layer: it extracts the slide key-frames and describes each with a vision model. + +## Prerequisites + +The heavy lifting is **external** — xbrain core carries no ML/ffmpeg dependency. +Install once (see [Local models for `digest-video`](../README.md#local-models-for-digest-video-apple-silicon)): + +```bash +brew install ffmpeg # frame extraction + audio probe +uv tool install parakeet-mlx # ASR (Apple Silicon) +uv tool install mlx-vlm # vision, only for --frames +``` + +and point `config.toml` at the wrappers: + +```toml +[transcribe] +command = "/abs/path/to/xbrain/scripts/xbrain-transcribe" # wraps parakeet-mlx + +[vision] +command = "/abs/path/to/xbrain/scripts/xbrain-vision" # local + cloud selector +model = "qwen-7b" +``` + +## Run it + +```bash +# Transcript only (no vision, no ffmpeg-frames) — fast: +uv run xbrain digest-video --all-pending + +# → Vídeos: transcritos 6, sin voz 2, ya digeridos 0, fallidos 0, sin vídeo 1, ... +# Dedup: 9 items ← 9 vídeos (6 transcritos este run). +``` + +Read the summary: **transcritos** = had speech, **sin voz** = silent (no audio +track — GIFs, muted clips; attached as `has_speech=false`, not a failure), +**fallidos** = a real transcribe failure, **sin vídeo** = the video couldn't be +fetched (deleted / unavailable). Videos are **deduped by identity** — N bookmarks +of the same clip are fetched + transcribed once. + +Add `--frames` for slide-heavy talks: + +```bash +uv run xbrain digest-video --all-pending --frames +# → ... Visual: 5 con slides, 4 talking-head (saltados). +``` + +`--frames` extracts key frames (ffmpeg scene-detection + interval sampling), +classifies the video as **slides** vs **talking-head** (talking-heads are skipped +— no vision calls wasted), and describes each slide of a slide video. The slide +images are embedded in the note like downloaded photos. + +Then render: + +```bash +uv run xbrain generate +``` + +## What you get + +The item's note gains a `## Video digest` section: + +```markdown +## Video digest: Elon Musk on the first thing to do when starting a company + +> Uh, the goal with Tesla was really to try to show what electric cars can do, +> because people had the wrong impression… (full transcript) + +![[_media/1874.../frames/0.png]] +> Slide: a line chart of Model S range vs. price, 2012–2015. +``` + +The transcript + slide descriptions are plain note text, so they feed `enrich` +(summary + topics) and are **searchable** in Obsidian. A silent video with no +slides degrades gracefully to a one-line "silent video" note. + +## Choosing the model, per run + +`config.toml` `[vision].model` is the default; `--vision-model` overrides it for +one run. The `scripts/xbrain-vision` selector routes the name: + +| `--vision-model` | Backend | Notes | +|------------------|---------|-------| +| `qwen-3b` / `qwen-7b` / `qwen-32b` / `` | local (mlx-vlm) | free, offline; `qwen-32b` needs ~20 GB RAM | +| `opus` / `sonnet` / `haiku` / `claude-` | cloud (Claude) | best quality; needs `ANTHROPIC_API_KEY`; frames leave the machine | + +```bash +uv run xbrain digest-video --ids --frames --vision-model opus +uv run xbrain digest-video --topic ai-coding --frames --vision-model qwen-7b +``` + +## Selecting which videos + +```bash +--ids a,b,c # specific item ids +--topic ai-coding # every video whose post is in that topic +--all-pending # every not-yet-digested video (idempotent; re-runs skip done ones) +--source bookmarks|tweets|all --limit N --language en +``` + +`digest-video` is destructive (rewrites `items.json`) → it auto-snapshots first. +Re-running skips videos already carrying an `x_video` source unless `--force`. + +Slow? See [Troubleshooting → digest-video](troubleshooting.md#digest-video-is-slow-or-times-out). diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md new file mode 100644 index 0000000..b2283af --- /dev/null +++ b/docs/troubleshooting.md @@ -0,0 +1,110 @@ +# Troubleshooting & FAQ + +Common failures and how to fix them. Most are environment issues (auth, PATH, +external tools), not bugs. + +## X session expired / auth fails + +Symptoms: `extract`/`sync` scrapes 0 posts, or `status` says it can't +authenticate. X sessions are short-lived. + +Fix — re-import cookies from a browser you're logged in to: + +```bash +# Chrome — log in to x.com in Chrome first, then: +.venv/bin/python scripts/import_chrome_session.py +# → "auth_token: OK" + +# Safari — log in in Safari, grant your terminal "Full Disk Access" +# (System Settings → Privacy & Security), then: +.venv/bin/python scripts/import_safari_session.py +``` + +`xbrain login` (in-app Playwright login) exists but is unreliable with +Google/SSO accounts — the automated browser gets blocked. Cookie import is the +recommended path. + +## "Re-saw 0 known items on a non-empty store" — the run aborts without saving + +A safety tripwire: extraction saw none of the items it already has, which almost +always means an **expired session** or an X GraphQL change, not that your +bookmarks vanished. It aborts rather than overwrite good data. Re-authenticate +(above) and re-run. If you're sure the store is stale, `--force` overrides it. + +## Getting rate-limited / the browser stalls + +`extract` runs **headful** (visible Chromium) by default to look human, paces +itself, and backs off on `429`. If you still hit limits, wait and re-run — the +store is incremental, so you lose nothing. Don't run many extracts back-to-back. + +## `parakeet-mlx` / `ffmpeg` not found (digest-video) + +``` +transcriber '.../xbrain-transcribe' exited 1: FileNotFoundError: 'parakeet-mlx' +``` + +The external tools aren't on `PATH`. Two cases: + +- **Interactive shell:** install them (`brew install ffmpeg`, + `uv tool install parakeet-mlx mlx-vlm`) and make sure `~/.local/bin` + + `/opt/homebrew/bin` are on your `PATH`. +- **cron / launchd / a scheduled job:** these run with a **minimal PATH** that + excludes `~/.local/bin` and `/opt/homebrew/bin`. Set the job's environment + explicitly — e.g. in a launchd plist: + + ```xml + EnvironmentVariables + + PATH/Users/you/.local/bin:/opt/homebrew/bin:/usr/bin:/bin + + ``` + + When testing a job, reproduce its env (`env -i HOME=$HOME PATH=... your-cmd`), + not your shell — your shell's full PATH hides the bug. + +## `digest-video` is slow or times out + +Local vision (`--frames`) is the bottleneck: a slide-heavy talk can have up to +40 key-frames, and a local VLM reloads the model per frame. On a 16 GB Mac, +`qwen-7b` is ~2 min/frame → a long talk takes over an hour. + +- **First run of a large model** can exceed the 300 s per-frame timeout while it + *downloads* — pre-pull once: `~/.local/share/uv/tools/mlx-vlm/bin/python -c + "from mlx_vlm import load; load('mlx-community/Qwen2.5-VL-7B-Instruct-4bit')"`. +- **Too slow overall?** Use a smaller model (`--vision-model qwen-3b`), or + transcript-only (drop `--frames`), or cloud (`--vision-model opus`, needs + `ANTHROPIC_API_KEY`). +- Frame extraction never hangs the run — ffmpeg is bounded by its own timeout. + +## Every video comes back `fallidos` / `sin voz` + +- `sin voz` (silent): the video has **no audio track** at the source (GIFs, + muted screencasts). This is expected — it attaches as `has_speech=false` + ("silent video"), not an error. Verify with `yt-dlp -f bestaudio ` + (errors = no audio exists). +- `fallidos` (real failures): usually `parakeet-mlx` not found (see the PATH + section above) — the fix is almost always the environment, not the video. + +## `generate` hangs or takes very long + +If your vault is on **iCloud** with "Optimize Mac Storage" on, files can be +evicted to the cloud (dataless), and reading/writing them blocks on +re-download — worst at night with no activity. Run `generate` while the machine +is active, or keep the vault folder materialized (turn off Optimize Storage for +it). `data/items.json` already holds every digest, so a slow `generate` never +loses data — just re-run it. + +## Do I need an API key? + +No. The default execution mode (`vocab`/`enrich`/`topics`/`describe`) uses a +**Claude Code session** — no key, no cost. `ANTHROPIC_API_KEY` is only for +`--executor api` (unattended LLM runs) and cloud vision (`--vision-model opus`). +`FIRECRAWL_API_KEY` is an optional fallback fetcher for JavaScript-heavy pages. + +## Where's the source of truth? Can I delete the vault notes? + +`data/items.json` is the hub — the markdown is **derived and disposable**. +Delete `items/`, `topics/`, `_index.md` and re-run `generate` any time. Every +destructive command auto-snapshots `items.json` first (see +[Snapshots & safety](../README.md#snapshots--safety)); restore from +`data/snapshots/` if needed. diff --git a/docs/tutorial.md b/docs/tutorial.md new file mode 100644 index 0000000..0c1909f --- /dev/null +++ b/docs/tutorial.md @@ -0,0 +1,128 @@ +# Tutorial — from zero to a searchable wiki + +A worked, end-to-end walkthrough: install XBrain, turn *your* X bookmarks into an +Obsidian knowledge base, and digest a bookmarked talk into readable notes. Every +command is copy-paste; the → lines show what you should see. + +New here? Do the [Quick start](../README.md#quick-start) first (install + +authenticate), then come back — this tutorial picks up from a logged-in install. + +--- + +## 1. Confirm you're set up + +```bash +uv run xbrain status +# → Items: 0 +# → con enlace: 0 +# → ... +``` + +An empty store with no error means config + auth are good. If `status` complains +about config, copy `config.toml.example` to `config.toml` and set your vault path ++ X handle. If it can't authenticate, re-run the cookie import (see +[Troubleshooting](troubleshooting.md#x-session-expired--auth-fails)). + +## 2. Pull your posts and build the mechanical wiki + +```bash +uv run xbrain sync # extract (scrape X) + fetch (article bodies) + generate +uv run xbrain status +# → Items: 812 +# → con enlace: 143 +# → última extracción bookmarks: 2026-07-04 ... +``` + +`sync` scrapes your bookmarks + own tweets into `data/items.json`, fetches the +linked article bodies, and writes one markdown note per post into your vault. +Open the vault in Obsidian — you already have `items/*.md` and `_index.md`. + +> `sync` runs **headful** by default (a visible Chromium) to look human; it +> paces itself and backs off on rate limits. First run scrolls your whole +> history, so it's the slow one. + +## 3. Add the topic layer (the LLM stages) + +The mechanical layers need no LLM. The *understanding* layers — a topic +vocabulary, per-post summaries + topics, and topic-page overviews — do: + +```bash +uv run xbrain vocab # induce ~45 topics from the corpus +uv run xbrain enrich # summary + topics for each post +uv run xbrain topics # write a topic page per cluster +uv run xbrain generate # re-render the vault with the new layers +``` + +By default these use the **claude-code execution mode** (no API key, no cost): +each stage exports a worksheet you fill in a Claude Code session, then +`--apply`. To run them unattended with the API instead, add `--executor api` +(needs `ANTHROPIC_API_KEY`). See [Execution modes](../README.md#execution-modes). + +Now your vault has three layers: `items/` (posts), `topics/` (thematic pages), +and `_index.md` (the map). Open `_index.md` in Obsidian and click into a topic. + +## 4. Download the media + +```bash +uv run xbrain media # download bookmarked photos +uv run xbrain download-videos --yes # download videos (prints a size gate first) +``` + +Photos embed under each post note. To make photos **searchable**, add vision +descriptions: + +```bash +uv run xbrain describe --executor claude-code # export a worksheet +# fill it in a Claude Code session, then: +uv run xbrain describe --apply data/describe-worksheet.json +uv run xbrain generate +``` + +Each photo now renders with a one-line caption under it — plain note text, so +Obsidian's search finds "that chart about pricing". + +## 5. Digest a bookmarked video + +This turns a saved talk into a readable, topic-linked note. It needs the local +tooling from [Local models for `digest-video`](../README.md#local-models-for-digest-video-apple-silicon) +(ffmpeg + parakeet-mlx, plus mlx-vlm for `--frames`). See the worked example in +[digest-video.md](digest-video.md). + +```bash +# Transcript only (fast): every bookmarked video → an x_video transcript source +uv run xbrain digest-video --all-pending + +# With the visual layer: also describe the slides of slide-heavy talks +uv run xbrain digest-video --all-pending --frames +uv run xbrain generate +# → the video's note now has a "## Video digest" section: transcript + slides +``` + +## 6. See the whole corpus at a glance + +`generate` also writes `dashboard.html` — a self-contained interactive dashboard +(counts, topics, authors, growth over time, photo thumbnails), with drill-down and +deep links back to each post + note. Open it from the **📊 Dashboard** link at the +top of `_index.md`, or directly in your browser: + +```bash +# //dashboard.html — from your config.toml [paths]: +open ~/Documents/Vault/vault/learnings/x-knowledge/dashboard.html +``` + +## Keeping it fresh + +Re-run periodically — everything is **incremental and idempotent**: + +```bash +uv run xbrain sync # pull new bookmarks/tweets, re-render +uv run xbrain enrich # enrich only the new posts +uv run xbrain topics # refresh topic pages +uv run xbrain generate +``` + +The markdown is **derived and disposable** — delete and regenerate any time. The +source of truth is `data/items.json` (snapshotted before every destructive op; +see [Snapshots & safety](../README.md#snapshots--safety)). + +Stuck? → [Troubleshooting](troubleshooting.md).