Release: docs user manual (tutorial, digest-video, troubleshooting)

.secrets.baseline

@@ -90,6 +90,10 @@
     {
       "path": "detect_secrets.filters.allowlist.is_line_allowlisted"
     },
+    {
+      "path": "detect_secrets.filters.common.is_baseline_file",
+      "filename": ".secrets.baseline"
+    },
     {
       "path": "detect_secrets.filters.common.is_ignored_due_to_verification_policies",
       "min_level": 2
@@ -129,23 +133,50 @@
         "filename": "scripts/check.sh",
         "hashed_secret": "e22586930a5b2f196cd9070b9a4af5c47c1380fa",
         "is_verified": false,
-        "line_number": 43
+        "line_number": 53
       },
       {
         "type": "Secret Keyword",
         "filename": "scripts/check.sh",
         "hashed_secret": "9d4e1e23bd5b727046a9e3b4b7db57bd8d6ee684",
         "is_verified": false,
-        "line_number": 127
+        "line_number": 414
       },
       {
         "type": "Secret Keyword",
         "filename": "scripts/check.sh",
         "hashed_secret": "a602e216eb44a3ac5e096036eeaaef6bb9159677",
         "is_verified": false,
-        "line_number": 415
+        "line_number": 418
+      }
+    ],
+    "tests/fixtures/art-Headcount_AI.json": [
+      {
+        "type": "Base64 High Entropy String",
+        "filename": "tests/fixtures/art-Headcount_AI.json",
+        "hashed_secret": "935c40c735878a3f840936083023bb1457a8f152",
+        "is_verified": false,
+        "line_number": 176
+      }
+    ],
+    "tests/fixtures/art-OpenWiki.json": [
+      {
+        "type": "Base64 High Entropy String",
+        "filename": "tests/fixtures/art-OpenWiki.json",
+        "hashed_secret": "cf1534db4a3700e7356f37e5b15459808dc52cb9",
+        "is_verified": false,
+        "line_number": 476
+      }
+    ],
+    "tests/fixtures/art-Wiki_Memory.json": [
+      {
+        "type": "Base64 High Entropy String",
+        "filename": "tests/fixtures/art-Wiki_Memory.json",
+        "hashed_secret": "b8cf2aa50cede3f008a49f610a8c90d5135530e8",
+        "is_verified": false,
+        "line_number": 424
       }
     ]
   },
-  "generated_at": "2026-05-18T17:05:58Z"
+  "generated_at": "2026-07-04T06:13:06Z"
 }

README.md

@@ -324,6 +324,10 @@ uv run xbrain status     # see the counts
 `sync` builds the mechanical layers. The LLM layers (`vocab`, `enrich`,
 `topics`) are run explicitly — see [The pipeline](#the-pipeline).
 
+> **New to XBrain?** The [**Tutorial**](docs/tutorial.md) walks through the whole
+> thing end-to-end — pull your posts, add topics, download + describe media, and
+> digest a bookmarked talk — with the output you should see at each step.
+
 ---
 
 ## Prerequisites
@@ -1211,6 +1215,9 @@ client. Respect X's Terms of Service.
 
 | Document | Description |
 |----------|-------------|
+| [docs/tutorial.md](docs/tutorial.md) | **Start here** — end-to-end walkthrough from install to a searchable wiki. |
+| [docs/digest-video.md](docs/digest-video.md) | Worked example: turn a bookmarked talk into transcript + slide notes. |
+| [docs/troubleshooting.md](docs/troubleshooting.md) | Common failures & fixes (auth, PATH, digest-video, iCloud). |
 | [ARCHITECTURE.md](ARCHITECTURE.md) | How XBrain is shaped: pipeline stages, artifacts, rubrics, executors, invariants. |
 | [CONTRIBUTING.md](CONTRIBUTING.md) | How to contribute — including PRs written with AI agents. |
 | [LICENSE](LICENSE) | MIT. |

docs/digest-video.md

@@ -0,0 +1,111 @@
+# `digest-video` — turn bookmarked talks into readable notes
+
+`digest-video` manufactures **text** from a video so it flows through the normal
+enrich → topics → generate pipeline like any other post. For each selected video
+it does an **ephemeral** fetch, transcribes the audio with an external local
+transcriber, attaches the transcript as an `x_video` content source, and
+**discards the bytes** (the corpus never lands on disk). `--frames` adds a visual
+layer: it extracts the slide key-frames and describes each with a vision model.
+
+## Prerequisites
+
+The heavy lifting is **external** — xbrain core carries no ML/ffmpeg dependency.
+Install once (see [Local models for `digest-video`](../README.md#local-models-for-digest-video-apple-silicon)):
+
+```bash
+brew install ffmpeg                # frame extraction + audio probe
+uv tool install parakeet-mlx       # ASR (Apple Silicon)
+uv tool install mlx-vlm            # vision, only for --frames
+```
+
+and point `config.toml` at the wrappers:
+
+```toml
+[transcribe]
+command = "/abs/path/to/xbrain/scripts/xbrain-transcribe"   # wraps parakeet-mlx
+
+[vision]
+command = "/abs/path/to/xbrain/scripts/xbrain-vision"       # local + cloud selector
+model   = "qwen-7b"
+```
+
+## Run it
+
+```bash
+# Transcript only (no vision, no ffmpeg-frames) — fast:
+uv run xbrain digest-video --all-pending
+
+# → Vídeos: transcritos 6, sin voz 2, ya digeridos 0, fallidos 0, sin vídeo 1, ...
+#   Dedup: 9 items ← 9 vídeos (6 transcritos este run).
+```
+
+Read the summary: **transcritos** = had speech, **sin voz** = silent (no audio
+track — GIFs, muted clips; attached as `has_speech=false`, not a failure),
+**fallidos** = a real transcribe failure, **sin vídeo** = the video couldn't be
+fetched (deleted / unavailable). Videos are **deduped by identity** — N bookmarks
+of the same clip are fetched + transcribed once.
+
+Add `--frames` for slide-heavy talks:
+
+```bash
+uv run xbrain digest-video --all-pending --frames
+# → ... Visual: 5 con slides, 4 talking-head (saltados).
+```
+
+`--frames` extracts key frames (ffmpeg scene-detection + interval sampling),
+classifies the video as **slides** vs **talking-head** (talking-heads are skipped
+— no vision calls wasted), and describes each slide of a slide video. The slide
+images are embedded in the note like downloaded photos.
+
+Then render:
+
+```bash
+uv run xbrain generate
+```
+
+## What you get
+
+The item's note gains a `## Video digest` section:
+
+```markdown
+## Video digest: Elon Musk on the first thing to do when starting a company
+
+> Uh, the goal with Tesla was really to try to show what electric cars can do,
+> because people had the wrong impression… (full transcript)
+
+![[_media/1874.../frames/0.png]]
+> Slide: a line chart of Model S range vs. price, 2012–2015.
+```
+
+The transcript + slide descriptions are plain note text, so they feed `enrich`
+(summary + topics) and are **searchable** in Obsidian. A silent video with no
+slides degrades gracefully to a one-line "silent video" note.
+
+## Choosing the model, per run
+
+`config.toml` `[vision].model` is the default; `--vision-model` overrides it for
+one run. The `scripts/xbrain-vision` selector routes the name:
+
+| `--vision-model` | Backend | Notes |
+|------------------|---------|-------|
+| `qwen-3b` / `qwen-7b` / `qwen-32b` / `<hf/repo>` | local (mlx-vlm) | free, offline; `qwen-32b` needs ~20 GB RAM |
+| `opus` / `sonnet` / `haiku` / `claude-<id>` | cloud (Claude) | best quality; needs `ANTHROPIC_API_KEY`; frames leave the machine |
+
+```bash
+uv run xbrain digest-video --ids <slide-heavy-id> --frames --vision-model opus
+uv run xbrain digest-video --topic ai-coding      --frames --vision-model qwen-7b
+```
+
+## Selecting which videos
+
+```bash
+--ids a,b,c        # specific item ids
+--topic ai-coding  # every video whose post is in that topic
+--all-pending      # every not-yet-digested video (idempotent; re-runs skip done ones)
+--source bookmarks|tweets|all   --limit N   --language en
+```
+
+`digest-video` is destructive (rewrites `items.json`) → it auto-snapshots first.
+Re-running skips videos already carrying an `x_video` source unless `--force`.
+
+Slow? See [Troubleshooting → digest-video](troubleshooting.md#digest-video-is-slow-or-times-out).

docs/troubleshooting.md

@@ -0,0 +1,110 @@
+# Troubleshooting & FAQ
+
+Common failures and how to fix them. Most are environment issues (auth, PATH,
+external tools), not bugs.
+
+## X session expired / auth fails
+
+Symptoms: `extract`/`sync` scrapes 0 posts, or `status` says it can't
+authenticate. X sessions are short-lived.
+
+Fix — re-import cookies from a browser you're logged in to:
+
+```bash
+# Chrome — log in to x.com in Chrome first, then:
+.venv/bin/python scripts/import_chrome_session.py
+# → "auth_token: OK"
+
+# Safari — log in in Safari, grant your terminal "Full Disk Access"
+# (System Settings → Privacy & Security), then:
+.venv/bin/python scripts/import_safari_session.py
+```
+
+`xbrain login` (in-app Playwright login) exists but is unreliable with
+Google/SSO accounts — the automated browser gets blocked. Cookie import is the
+recommended path.
+
+## "Re-saw 0 known items on a non-empty store" — the run aborts without saving
+
+A safety tripwire: extraction saw none of the items it already has, which almost
+always means an **expired session** or an X GraphQL change, not that your
+bookmarks vanished. It aborts rather than overwrite good data. Re-authenticate
+(above) and re-run. If you're sure the store is stale, `--force` overrides it.
+
+## Getting rate-limited / the browser stalls
+
+`extract` runs **headful** (visible Chromium) by default to look human, paces
+itself, and backs off on `429`. If you still hit limits, wait and re-run — the
+store is incremental, so you lose nothing. Don't run many extracts back-to-back.
+
+## `parakeet-mlx` / `ffmpeg` not found (digest-video)
+
+```
+transcriber '.../xbrain-transcribe' exited 1: FileNotFoundError: 'parakeet-mlx'
+```
+
+The external tools aren't on `PATH`. Two cases:
+
+- **Interactive shell:** install them (`brew install ffmpeg`,
+  `uv tool install parakeet-mlx mlx-vlm`) and make sure `~/.local/bin` +
+  `/opt/homebrew/bin` are on your `PATH`.
+- **cron / launchd / a scheduled job:** these run with a **minimal PATH** that
+  excludes `~/.local/bin` and `/opt/homebrew/bin`. Set the job's environment
+  explicitly — e.g. in a launchd plist:
+
+  ```xml
+  <key>EnvironmentVariables</key>
+  <dict>
+    <key>PATH</key><string>/Users/you/.local/bin:/opt/homebrew/bin:/usr/bin:/bin</string>
+  </dict>
+  ```
+
+  When testing a job, reproduce its env (`env -i HOME=$HOME PATH=... your-cmd`),
+  not your shell — your shell's full PATH hides the bug.
+
+## `digest-video` is slow or times out
+
+Local vision (`--frames`) is the bottleneck: a slide-heavy talk can have up to
+40 key-frames, and a local VLM reloads the model per frame. On a 16 GB Mac,
+`qwen-7b` is ~2 min/frame → a long talk takes over an hour.
+
+- **First run of a large model** can exceed the 300 s per-frame timeout while it
+  *downloads* — pre-pull once: `~/.local/share/uv/tools/mlx-vlm/bin/python -c
+  "from mlx_vlm import load; load('mlx-community/Qwen2.5-VL-7B-Instruct-4bit')"`.
+- **Too slow overall?** Use a smaller model (`--vision-model qwen-3b`), or
+  transcript-only (drop `--frames`), or cloud (`--vision-model opus`, needs
+  `ANTHROPIC_API_KEY`).
+- Frame extraction never hangs the run — ffmpeg is bounded by its own timeout.
+
+## Every video comes back `fallidos` / `sin voz`
+
+- `sin voz` (silent): the video has **no audio track** at the source (GIFs,
+  muted screencasts). This is expected — it attaches as `has_speech=false`
+  ("silent video"), not an error. Verify with `yt-dlp -f bestaudio <tweet-url>`
+  (errors = no audio exists).
+- `fallidos` (real failures): usually `parakeet-mlx` not found (see the PATH
+  section above) — the fix is almost always the environment, not the video.
+
+## `generate` hangs or takes very long
+
+If your vault is on **iCloud** with "Optimize Mac Storage" on, files can be
+evicted to the cloud (dataless), and reading/writing them blocks on
+re-download — worst at night with no activity. Run `generate` while the machine
+is active, or keep the vault folder materialized (turn off Optimize Storage for
+it). `data/items.json` already holds every digest, so a slow `generate` never
+loses data — just re-run it.
+
+## Do I need an API key?
+
+No. The default execution mode (`vocab`/`enrich`/`topics`/`describe`) uses a
+**Claude Code session** — no key, no cost. `ANTHROPIC_API_KEY` is only for
+`--executor api` (unattended LLM runs) and cloud vision (`--vision-model opus`).
+`FIRECRAWL_API_KEY` is an optional fallback fetcher for JavaScript-heavy pages.
+
+## Where's the source of truth? Can I delete the vault notes?
+
+`data/items.json` is the hub — the markdown is **derived and disposable**.
+Delete `items/`, `topics/`, `_index.md` and re-run `generate` any time. Every
+destructive command auto-snapshots `items.json` first (see
+[Snapshots & safety](../README.md#snapshots--safety)); restore from
+`data/snapshots/` if needed.