Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
116 commits
Select commit Hold shift + click to select a range
e5c6035
[#33] add Media tagged union with legacy migration
VGonPa May 24, 2026
f129766
[#33] add media download orchestrator with cascade fallback
VGonPa May 24, 2026
ae2625b
[#33] add xbrain media CLI command with snapshot trigger
VGonPa May 24, 2026
d755666
[#33] render downloaded photos inline in Obsidian notes
VGonPa May 24, 2026
4da4589
[#33] extend xbrain diff to report media state counts
VGonPa May 24, 2026
9d2540b
[#33] document Phase A media downloader in README + ARCHITECTURE
VGonPa May 24, 2026
7690acf
[#33] apply ruff format to Phase A files
VGonPa May 24, 2026
0fa494e
[#33] refactor download_all to bring complexity down to B
VGonPa May 24, 2026
dcc5532
[#33] declare requests as an explicit runtime dependency
VGonPa May 24, 2026
738390d
[#33] address silent-failure review: narrow exceptions, log warnings,…
VGonPa May 24, 2026
e053be5
[#33] simplify media.py: inline single-call helpers, drop unused params
VGonPa May 24, 2026
a1a6aa7
[#33] harden Media types: gt=0 ranges, path-traversal, attempts>=1
VGonPa May 24, 2026
6399f37
[#33] strip issue-number rot and rewrite lying docstrings
VGonPa May 24, 2026
c8e3f45
[#33] strengthen media test suite: cascade, resume, atomic-write, ver…
VGonPa May 24, 2026
12eb9ce
[#33] silence vulture on @field_validator cls and apply ruff format
VGonPa May 24, 2026
0165076
[#33] strip stray issue-number references from test docstrings
VGonPa May 24, 2026
e1ec720
[#33] fix path-traversal validator docstring positioning
VGonPa May 24, 2026
a892da5
[#33] add validator-rejection tests for new Media constraints
VGonPa May 24, 2026
2caf1dd
Merge pull request #35 from VGonPa/ws-33-images-phase-a
VGonPa May 24, 2026
3378cc2
[#33] reduce download_all + _download_one complexity below grade C
VGonPa May 24, 2026
5490bbc
Merge pull request #36 from VGonPa/ws-radon-cleanup-media
VGonPa May 24, 2026
aac3d83
[#34] add MediaPhotoDescribed variant and discriminator updates
VGonPa May 24, 2026
deb094d
[#34] add describe-image rubric with {language} placeholder
VGonPa May 24, 2026
b59d610
[#34] add xbrain describe orchestrator + Anthropic client wiring
VGonPa May 24, 2026
3c302f8
[#34] add xbrain describe CLI command + snapshot trigger
VGonPa May 24, 2026
49287ac
[#34] integrate image descriptions into enrich + topics prompts
VGonPa May 24, 2026
110b365
[#34] extend xbrain diff with described counts
VGonPa May 24, 2026
d71a30a
[#34] update README + ARCHITECTURE for the describe stage
VGonPa May 24, 2026
a215fb6
[#34] dedupe SUMMARY + harden MediaPhotoDescribed invariants
VGonPa May 24, 2026
260c883
[#34] refactor orchestrator + SDK refusal + language drift + Protocol
VGonPa May 24, 2026
a373eff
[#34] inline _user_directive_text into _build_user_blocks
VGonPa May 24, 2026
4c09fff
[#34] add negative tests for MediaPhotoDescribed invariants + media_t…
VGonPa May 24, 2026
98eee46
Merge pull request #37 from VGonPa/ws-34-images-phase-b
VGonPa May 24, 2026
839a8d2
[cleanup] split validate_judgment into focused sub-validators
VGonPa May 25, 2026
c9472eb
[cleanup] extract archive parsing helpers from _archive_tweet_to_item
VGonPa May 25, 2026
cb8c6ae
[cleanup] extract topic-frequency computation from _render_index
VGonPa May 25, 2026
f26ec88
[cleanup] silence deptry DEP003 for optional browser_cookie3 import
VGonPa May 25, 2026
f5029e3
Merge pull request #38 from VGonPa/ws-cleanup-radon-deptry
VGonPa May 25, 2026
526bbaa
[#40] capture playable video URL from video_info.variants, not the po…
VGonPa Jun 25, 2026
e2e96e4
[#40] share video-variant helper across extract + archive; harden bit…
VGonPa Jun 25, 2026
ed27add
Merge pull request #41 from VGonPa/feat/40-video-playable-url
VGonPa Jun 25, 2026
9528fa0
[#40] add xbrain refresh-media to backfill playable video on existing…
VGonPa Jun 25, 2026
ce2b489
[#40] refresh-media: empty-capture guard + no-degrade swap; review round
VGonPa Jun 25, 2026
fb126b2
Merge pull request #42 from VGonPa/feat/40-refresh-media-backfill
VGonPa Jun 25, 2026
4245e97
[#40] add xbrain download-videos: download mp4 video bytes (part 2)
VGonPa Jun 25, 2026
d314e30
[#40] download-videos: validate video bytes, add --max-size, review f…
VGonPa Jun 26, 2026
43eca9d
[#40] download-videos: harden mid-download failures (review round 2)
VGonPa Jun 26, 2026
05dc3bd
Merge pull request #43 from VGonPa/feat/40-download-videos
VGonPa Jun 26, 2026
d693386
feat(extract): headful by default + randomized 429 backoff (anti-ban)
VGonPa Jun 30, 2026
f5f84e0
fix(extract): close data-loss + headless-regression found in review
VGonPa Jun 30, 2026
304a474
Merge pull request #45 from VGonPa/feat/headful-default-rate-limit-ba…
VGonPa Jun 30, 2026
3d3e615
chore: add resynth-topic-overviews workflow
VGonPa Jul 1, 2026
1751def
Merge pull request #48 from VGonPa/chore/resynth-topic-overviews-work…
VGonPa Jul 1, 2026
3d84730
[#44] PR1: add list-videos + fetch-video (selection + ephemeral fetch)
VGonPa Jul 1, 2026
4016ba8
[#44] PR1 review round: data-safety + machine-contract fixes
VGonPa Jul 1, 2026
a09e045
[#44] Merge PR1: list-videos + fetch-video (selection + ephemeral fetch)
VGonPa Jul 1, 2026
60a4d58
[#44] PR2: add digest-video (fetch → transcribe → x_video content sou…
VGonPa Jul 1, 2026
aa17399
[#44] PR2 review round: fix transcriber contract (silent data-loss) +…
VGonPa Jul 1, 2026
3221559
[#44] Merge PR2: digest-video (fetch → transcribe → x_video content s…
VGonPa Jul 1, 2026
95a6380
[#44] PR3: enrich/topics/generate consume the x_video transcript
VGonPa Jul 1, 2026
3c2d139
[#44] PR3 review round: pin re-enrichment settle + close test/doc gaps
VGonPa Jul 1, 2026
823b832
Merge PR 3: enrich/topics/generate consume the x_video transcript (#44)
VGonPa Jul 1, 2026
93f05fc
[#44] PR4: visual layer — content-type-aware key-frame slides (--fram…
VGonPa Jul 1, 2026
ed84ba4
feat(dashboard): generate an interactive metrics dashboard (#49)
VGonPa Jul 1, 2026
d556a68
fix(dashboard): address review-panel findings (XSS breakout, isolation)
VGonPa Jul 1, 2026
0baf7c8
Merge pull request #51 from VGonPa/feat/49-metrics-dashboard
VGonPa Jul 1, 2026
e4660df
[#44] PR4 review round: kill the cold-run flake at its root + harden …
VGonPa Jul 1, 2026
0899404
Merge PR 4: content-type-aware visual layer (--frames slides) (#44)
VGonPa Jul 1, 2026
c2231be
[#44] Re-enrich hygiene: advance content.fetched_at only on a MATERIA…
VGonPa Jul 1, 2026
e70d88e
[#44] Re-enrich: model-derived (deny-list) material-content signature
VGonPa Jul 1, 2026
5589942
Merge re-enrich hygiene: material-content refetch signature (#44)
VGonPa Jul 1, 2026
06a143d
feat(describe): claude-code worksheet track + render descriptions in …
VGonPa Jul 1, 2026
175102b
fix(describe): harden worksheet apply + multi-line captions (PR #53 r…
VGonPa Jul 1, 2026
f2592d0
Merge pull request #53 from VGonPa/feat/52-describe-worksheet
VGonPa Jul 1, 2026
321888f
[#44] Final-review polish: doc fixes + caption newline + coverage
VGonPa Jul 1, 2026
5e4da02
fix(dashboard): count described photos as downloaded + show captions …
VGonPa Jul 1, 2026
5257efc
Merge origin/develop into feat/44-video-digest-module before PR (#44)
VGonPa Jul 1, 2026
003d789
fix(dashboard): typed isinstance for caption + assert empty desc (PR …
VGonPa Jul 1, 2026
4227810
Merge pull request #54 from VGonPa/feat/54-dashboard-described-photos
VGonPa Jul 1, 2026
de66abe
Merge pull request #55 from VGonPa/feat/44-video-digest-module
VGonPa Jul 1, 2026
d713ce8
fix(generate): link dashboard from _index with an absolute file:// UR…
VGonPa Jul 2, 2026
9e46905
docs(generate): note the dashboard file:// link is machine-local (PR …
VGonPa Jul 2, 2026
07efed8
Merge pull request #60 from VGonPa/fix/55-dashboard-index-link
VGonPa Jul 2, 2026
dca1851
[#46] fix --until date-only boundary to include the whole final day
VGonPa Jul 2, 2026
d1a13f9
[#46] harden date-only detection + pin boundary/per-command wiring
VGonPa Jul 2, 2026
ce0c6ef
Merge pull request #61 from VGonPa/feat/46-until-boundary
VGonPa Jul 2, 2026
d22ed93
[#34] Flow image descriptions through the worksheet enrich/topics tracks
VGonPa Jul 2, 2026
c277dd7
[#34] Wire the claude-code topic consumers for image_descriptions (+ …
VGonPa Jul 2, 2026
8980a7d
[#34] Image descriptions (and video transcripts) through the workshee…
VGonPa Jul 2, 2026
8de2415
[#39] PR1: article blocks model (ArticleTextBlock | ArticleImageBlock)
VGonPa Jul 2, 2026
edeeea0
Merge #39 PR1: article-blocks model seam (ArticleTextBlock | ArticleI…
VGonPa Jul 2, 2026
7f8a1a5
[#39] PR2: detect X Article entity → canonical /i/article link
VGonPa Jul 2, 2026
5d09a97
feat(digest): multi-backend vision model selector + docs (#56)
VGonPa Jul 2, 2026
fe883d8
[#39] PR2 hardening: normalize dedup, __typename + numeric-rest_id gu…
VGonPa Jul 2, 2026
a4e9d84
Merge #39 PR2: detect X Article entity → canonical /i/article link
VGonPa Jul 2, 2026
71e0315
fix(digest): harden vision selector per PR #63 review (2 panels, no 🔴)
VGonPa Jul 2, 2026
4c1789e
test(digest): assert usage exit code + ANSI-stripped msg for --vision…
VGonPa Jul 2, 2026
621f5f1
Merge pull request #63 from VGonPa/feat/56-vision-model-selector
VGonPa Jul 2, 2026
ce6e8f6
[#39] PR3: fetch article body → ordered blocks (+ text==concat invari…
VGonPa Jul 2, 2026
668ef26
[#39] PR3 review round: observability, truncation tripwire, degrade-n…
VGonPa Jul 2, 2026
86181e3
[#39] PR3: guard truncation tripwire against trafilatura raising (deg…
VGonPa Jul 2, 2026
a9b92c9
Merge #39 PR3: fetch article body → ordered blocks (+ text==concat va…
VGonPa Jul 2, 2026
d7e3a86
[#39] PR4: download inline article images (reuse media engine)
VGonPa Jul 2, 2026
51cc4c6
[#39] PR4 review round: dedicated article_images_skipped, limit-in-ge…
VGonPa Jul 2, 2026
d918358
Merge #39 PR4: download inline article images (reuse media engine)
VGonPa Jul 2, 2026
99582e9
[#39] PR5: render x_article as an ordered blogpost with inline images
VGonPa Jul 2, 2026
6beb169
fix(digest): handle silent (no-audio) videos instead of failing them …
VGonPa Jul 2, 2026
2125efb
fix(digest): content-based output detection + confirm-silence for tra…
VGonPa Jul 2, 2026
2294ce0
Merge pull request #64 from VGonPa/fix/57-transcribe-silent-videos
VGonPa Jul 2, 2026
de9f2be
[#39] PR5 review fixes: suppress empty article heading, pin defensive…
VGonPa Jul 2, 2026
e039605
Merge #39 PR5: render x_article as an ordered blogpost with inline im…
VGonPa Jul 2, 2026
ac46002
[#39] Fix round: consolidate paragraph separator + nav-failure diagno…
VGonPa Jul 2, 2026
7d03bb8
Merge origin/develop into feat/39-x-articles-blogpost before PR (#39)
VGonPa Jul 2, 2026
ae68459
Merge pull request #65 from VGonPa/feat/39-x-articles-blogpost
VGonPa Jul 2, 2026
2b212ac
docs: make digest-video external dependencies explicit (ffmpeg, parak…
VGonPa Jul 4, 2026
c7755be
Merge pull request #69 from VGonPa/docs/66-digest-dependencies
VGonPa Jul 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .claude/skills/enriching-x-knowledge/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,15 @@ each topic page — also via a worksheet, at no API cost.
with their current post lists. If it reports 0 topics pending, stop.

2. **Read `data/topic-worksheet.json`.** It has `rubric` and `topics` (each with
a `slug`, a `description` and the `summaries` of its posts).
a `slug`, a `description`, the `summaries` of its posts, and — when the posts
carry them — `image_descriptions` (prose for the topic's content-bearing
photos) and `video_transcripts` (bounded transcript excerpts of its videos)).

3. **Produce one judgment per topic**, following the embedded rubric exactly:
- `overview`: 1-3 paragraphs of plain Spanish prose, faithful to the
summaries.
summaries. When a topic carries `image_descriptions` or `video_transcripts`,
weigh them as visual/transcript evidence alongside the summaries — they carry
topic signal a one-line summary may only hint at.
- `notes`: 0-15 plain-prose strings, one important idea each.
- Emit only `{slug, overview, notes}`. **Never write a wikilink (`[[...]]`),
a filename or any identifier** — you have summaries, not posts, and the
Expand Down
109 changes: 109 additions & 0 deletions .claude/workflows/resynth-topic-overviews.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
/*
* resynth-topic-overviews — re-synthesize stale xbrain topic-page overviews,
* one agent per topic (each reads only its own topic's post summaries).
*
* WHY: `xbrain topics` refreshes the mechanical post-lists on every run, but it
* does NOT rewrite the prose `overview` / `notes`. As the corpus grows, those
* overviews go stale and the pages show "⚠ Overview desactualizado: +N posts
* desde la última síntesis". `topics --resynth` rewrites them from the current
* post set. Fanning the work out — one agent per topic — lets each agent spend
* its full context on a single topic (some carry 200-326 post summaries) and
* runs them in parallel, instead of one agent diluting across all of them.
*
* The agents synthesize; they do NOT write the worksheet (N writers on one JSON
* would corrupt it). Each RETURNS {slug, overview, notes}; a single assemble +
* validate step (step 4 below) writes the worksheet once.
*
* RUNBOOK (from the xbrain repo root, with the venv):
*
* # 1. Export the resynth worksheet (stale topics + their summaries + rubric):
* .venv/bin/xbrain topics --resynth --executor claude-code
*
* # 2. Dump the topic slugs to feed as this workflow's `args`:
* .venv/bin/python -c "import json;print(json.dumps([t['slug'] for t in json.load(open('data/topic-worksheet.json'))['topics']]))"
*
* # 3. Run this workflow (Claude drives it):
* # Workflow({ name: 'resynth-topic-overviews', args: <the slug array from step 2> })
* # It returns [{slug, overview, notes}, ...] — one entry per topic.
*
* # 4. Save that returned array to data/overviews-result.json, then assemble +
* # validate into the worksheet (coverage, no wikilinks, cap notes to 15):
* .venv/bin/python - <<'PY'
* import json
* res = json.load(open('data/overviews-result.json'))
* ws = json.load(open('data/topic-worksheet.json'))
* assert {t['slug'] for t in ws['topics']} == {r['slug'] for r in res}, 'cobertura incompleta'
* for r in res:
* blob = r['overview'] + ' '.join(r['notes'])
* assert r['overview'].strip() and '[[' not in blob, r['slug'] # no vacío, sin wikilinks
* ws['judgments'] = [{'slug': r['slug'], 'overview': r['overview'], 'notes': r['notes'][:15]} for r in res]
* json.dump(ws, open('data/topic-worksheet.json', 'w'), ensure_ascii=False, indent=2)
* print('worksheet:', len(ws['judgments']), 'judgments')
* PY
*
* # 5. Apply + regenerate the vault, then confirm zero stale warnings:
* .venv/bin/xbrain topics --apply data/topic-worksheet.json
* .venv/bin/xbrain generate
* grep -rl "Overview desactualizado" <vault>/learnings/x-knowledge/topics | wc -l # -> 0
*
* Model: agents inherit the session model. Pin `model: 'opus'` (below) so a
* resynth always runs on Opus regardless of the session's model.
*/

export const meta = {
name: 'resynth-topic-overviews',
description: 'Re-synthesize stale xbrain topic overviews, one Opus agent per topic',
phases: [{ title: 'Synthesize', detail: 'one agent per topic reads its own posts', model: 'opus' }],
}

// The standard export location written by `xbrain topics --resynth --executor claude-code`.
const WORKSHEET = '/Users/vgonpa/devel/xbrain/data/topic-worksheet.json'

const OVERVIEW_SCHEMA = {
type: 'object',
additionalProperties: false,
required: ['overview', 'notes'],
properties: {
overview: {
type: 'string',
description: '1 a 3 párrafos en español, prosa plana, sin wikilinks ni headings.',
},
notes: {
type: 'array',
items: { type: 'string' },
description: '0 a 15 frases cortas en español, una idea por nota, sin bullets ni wikilinks.',
},
},
}

// `args` may arrive as a real array or as a JSON string — parse defensively.
const slugs = Array.isArray(args) ? args : JSON.parse(args)
log(`Re-sintetizando ${slugs.length} overviews de topic, un agente Opus por topic`)
phase('Synthesize')

const results = await parallel(
slugs.map((slug) => () =>
agent(
`Eres un sintetizador experto de una base de conocimiento personal (xbrain de Víctor). Tu única tarea: escribir el OVERVIEW del topic "${slug}", leyendo SOLO los posts de ESE topic.

PASO 1 — Extrae tu topic y la rúbrica ejecutando exactamente:
cd /Users/vgonpa/devel/xbrain && .venv/bin/python -c "import json; d=json.load(open('${WORKSHEET}')); t=[x for x in d['topics'] if x['slug']=='${slug}'][0]; print('=== RUBRICA ==='); print(d['rubric']); print('=== DESCRIPTION ==='); print(t['description']); print('=== SUMMARIES ('+str(len(t['summaries']))+') ==='); [print('- '+s) for s in t['summaries']]; img=t.get('image_descriptions') or []; print('=== IMAGES ('+str(len(img))+') ==='); [print('- '+s) for s in img]; vid=t.get('video_transcripts') or []; print('=== VIDEO TRANSCRIPTS ('+str(len(vid))+') ==='); [print('- '+s) for s in vid]"

PASO 2 — Lee la RÚBRICA entera y respétala al pie de la letra. Lee la DESCRIPTION y TODOS los SUMMARIES: cada summary es el resumen de un post del topic — ESOS son "los posts" que debes leer y sintetizar. El comando también imprime dos bloques de evidencia visual/audiovisual cuando existen — trátalos como parte de "los posts", con el mismo peso que los summaries:
- IMAGES — "Images across the content-bearing photos in this topic": descripciones en prosa de las fotos con contenido (gráficas, capturas, diagramas) de los posts del topic. Aportan señal temática que el summary a veces solo insinúa.
- VIDEO TRANSCRIPTS — "Video transcripts across the videos in this topic": extractos (truncados) de las transcripciones de los vídeos del topic.
No leas otros ficheros; trabaja solo con lo que imprime ese comando.

PASO 3 — Sintetiza para el topic "${slug}":
- overview: 1 a 3 párrafos en español. Qué es este topic EN ESTE CORPUS: las ideas recurrentes, el arco temporal, las tensiones o debates. Escrito para alguien que decide si leer los posts. Integra la evidencia de IMAGES y VIDEO TRANSCRIPTS junto con los summaries. FIEL a los summaries y a esa evidencia — nunca inventes nombres, números ni hechos que no estén en ellos. Prosa plana: sin wikilinks [[...]], sin nombres de fichero, sin identificadores, sin headings markdown. Si el topic es "misc" o de verdad no tiene núcleo temático, dilo con naturalidad en un párrafo y deja notes corto o vacío — no fabriques temas.
- notes: lista de 0 a 15 frases cortas en español, una idea/hilo/patrón importante por nota, frase plana sin bullets ni wikilinks.

Devuelve SOLO el objeto {overview, notes}.`,
{ label: `synth:${slug}`, phase: 'Synthesize', schema: OVERVIEW_SCHEMA, model: 'opus' }
).then((r) => (r ? { slug, overview: r.overview, notes: r.notes } : null))
)
)

const filled = results.filter(Boolean)
log(`Completados ${filled.length}/${slugs.length} overviews`)
return filled
4 changes: 4 additions & 0 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ on:
jobs:
quality:
runs-on: ubuntu-latest
env:
# Pin the Python hash seed so the gate is byte-reproducible run-to-run
# (determinism; check.sh pins the same default for local runs).
PYTHONHASHSEED: "0"
steps:
- uses: actions/checkout@v4
- name: Install uv
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,6 @@ auth/storage_state.json
data/*
!data/.gitkeep
config.toml

# Working docs (not tracked)
zz-support-files/
Loading
Loading