vault-brain

Three-stage memory maintenance for markdown vaults: walk, reconcile, link.

An agent that remembers across sessions needs its memory maintained, not just appended. Session captures pile up as prose; the same project ends up with five differently named notes; nothing connects to anything; wrong or stale facts stay live forever. vault-brain is a small, dependency-free Python toolkit that runs nightly over an Obsidian-style vault and turns that pile into a maintained knowledge graph. It reads and writes plain markdown with YAML-ish frontmatter and [[wikilinks]], so the result stays readable and editable in any vault tool.

Judgement is delegated to an LLM through one configurable shell command. Everything that must not fail (capture, ledgers, file moves, link rewriting) is deterministic code.

The three stages

Run them in this order. Each is a governed pass: it checks a budget signal first, does a capped amount of work, records what it did, and stops. The next run catches up.

1. Walk (vault_brain.walk) Reads: unprocessed session episodes in INBOX_DIR (notes whose frontmatter type: is one of EPISODE_TYPES and that have no brain_processed: stamp), plus the inboxes of sibling vaults if enabled. Asks the LLM to decompose each episode into typed atoms: decision, risk, fact, person, opportunity, hypothesis, open_question. Each atom names the durable entity it is about and a coarse confidence (verified or inferred). Writes: one bullet per atom under ## Atoms in the entity's note in LONG_TERM_DIR/<node>/<entity>.md, creating the note if missing (person atoms go to PEOPLE_NODE, everything else to DEFAULT_NODE). Stamps the episode brain_processed: <date> when writable and appends its key to the .brain-processed.txt ledger.

2. Reconcile (vault_brain.reconcile) Reads: every active type: entity note under LONG_TERM_DIR (never the archive). Blocks them into candidate clusters by shared name tokens (deterministic; hub tokens that would chain everything together are dropped, and oversized clusters are re-blocked more strictly). For each multi-member cluster the LLM says which members are the same thing and picks a canonical name. Unsure means no merge. Writes: absorbed atoms folded into the canonical note (exact duplicate bullets dropped, sources unioned, earliest created kept, merged_from: stamped); every [[old name]], [[old name|alias]] and [[old name#heading]] in other active notes repointed to the canonical; the absorbed note moved to LONG_TERM_DIR/_Archive/_superseded/ with status: superseded, superseded_by: and a reason (never deleted). Notes not confirmed for BRAIN_STALE_DAYS are flipped to status: stale in place. A merge map is appended to Timeline/<date> - Reconcile.md.

3. Link and abstract (vault_brain.link) Reads: the same active entity notes. For each note, finds the other entities it mentions by name (whole phrase, case-insensitive) and asks the LLM to type each real relationship from a fixed vocabulary (depends-on, blocks, supersedes, contradicts, about, same-entity, caused, resulted-in, follows-from, continues) with a one-line reason. No type or no reason means no edge. Writes: - <type>:: [[target]] - <reason> bullets under a ## Related heading in the source note. Then any entity with at least BRAIN_ABSTRACT_MIN_EVIDENCE connections is offered to the LLM as a possible durable lesson; the default answer is NONE. Accepted lessons become type: insight hub notes in LONG_TERM_DIR/Growth/, linked to their evidence. An audit map goes to Timeline/<date> - Link.md.

Expected vault layout

Folder names are the defaults and are all configurable. Paths are relative to VAULT_ROOT.

<vault>/
  2 Memory/
    .brain-processed.txt        ledgers (BRAIN_LEDGER_DIR)
    .brain-reconciled.txt
    .brain-linked.txt
    Inbox/                      INBOX_DIR: session episodes land here
    Long-term/                  LONG_TERM_DIR: entity notes
      Decisions & Synthesis/    DEFAULT_NODE
      People/Unsorted/          PEOPLE_NODE
      Growth/                   GROWTH_SUBDIR: insight hubs
      Timeline/                 TIMELINE_SUBDIR: audit notes
      _Archive/_superseded/     ARCHIVE_SUBDIR: tombstoned notes

A session episode is any markdown file in the inbox with frontmatter like:

---
type: session-episode
session_id: 2026-01-05-abc
---

session_id is the dedup key when present, otherwise the filename. Entity notes carry type: entity, entity:, status:, created:, last_confirmed: and source: in their frontmatter; the walk creates them in that shape.

Idempotency and the governor

Every stage keeps a ledger in BRAIN_LEDGER_DIR, and the ledger, not the in-file stamp, is the source of truth. The walk ledgers episode keys, so an episode in a read-only or synced vault that cannot be stamped is still never reprocessed. Reconcile ledgers the hash of each cluster the LLM declined to merge, so it is not re-asked until the cluster's membership changes. Link ledgers the hash of (source, candidate set), skips any [[target]] already present in the note, and deepens an existing insight rather than creating a second one. Re-running any stage on an unchanged vault is a no-op.

The governor is a budget kill switch. Before doing anything, each stage reads an effective tier (green, amber or red) and skips its whole pass at red, catching up next run. The tier comes from GOVERNOR_TIER (a static override), or a GET to GOVERNOR_URL returning JSON with an effectiveTier field, or green when neither is set or the URL is unreachable. It fails open because it can only ever reduce work. Per-run caps (*_MAX_PER_RUN) pace a large backlog across nights.

Configuration

All settings are environment variables. Relative paths are resolved against VAULT_ROOT.

Variable	Default	Meaning
VAULT_ROOT	(required)	The vault directory
LLM_COMMAND	(required)	Shell command that reads a prompt on stdin and prints the reply on stdout
LLM_TIMEOUT_SECS	600	Per-call timeout
AGENT_NAME	agent	Persona name used in the prompts
GOVERNOR_TIER	(unset)	Static tier override: green, amber, red
GOVERNOR_URL	(unset)	URL returning {"effectiveTier": "..."}
INBOX_DIR	2 Memory/Inbox	Where episodes are read from
LONG_TERM_DIR	2 Memory/Long-term	Where entity notes live
BRAIN_LEDGER_DIR	2 Memory	Where the three ledger files live
ARCHIVE_SUBDIR	_Archive	Under LONG_TERM_DIR; tombstoned notes
GROWTH_SUBDIR	Growth	Under LONG_TERM_DIR; insight hubs
TIMELINE_SUBDIR	Timeline	Under LONG_TERM_DIR; audit notes
PEOPLE_NODE	People/Unsorted	Under LONG_TERM_DIR; person atoms
DEFAULT_NODE	Decisions & Synthesis	Under LONG_TERM_DIR; all other atoms
EPISODE_TYPES	session-episode,session-capture	Frontmatter type: values treated as episodes
DISCOVER_SIBLING_VAULTS	0	Set to 1 to also scan sibling directories of VAULT_ROOT for inboxes
SIBLING_INBOX_SUBDIRS	2 Memory/Inbox:00 Inbox	Inbox paths to look for inside sibling vaults (os.pathsep separated)
BRAIN_WALK_INBOXES	(unset)	Explicit extra inbox directories; disables discovery
BRAIN_WALK_MAX_PER_RUN	50	Episodes per run (0 = unlimited)
BRAIN_RECONCILE_MAX_PER_RUN	50	Clusters adjudicated per run
BRAIN_STALE_DAYS	90	Days since last_confirmed before status: stale
BRAIN_BLOCK_DF_CAP	6	Tokens shared by more notes than this are ignored when blocking
BRAIN_BLOCK_SIZE_CAP	25	Largest cluster sent to the LLM
BRAIN_LINK_MAX_PER_RUN	50	Source notes typed per run
BRAIN_ABSTRACT_MIN_EVIDENCE	4	Minimum connections before a note is offered for abstraction
BRAIN_ABSTRACT_MAX_PER_RUN	5	Abstraction calls per run

LLM_COMMAND is any program that behaves like a filter: prompt in, text out, for example claude -p, llm -m <model>, or ollama run <model>. The prompts ask for JSON only, so a plain-text-answering model works. Credentials are the command's own business; vault-brain handles none. Setting LLM_COMMAND=cat echoes the prompt back and is useful only to smoke-test the plumbing.

Running

pip install .                       # or run from the checkout with python -m
export VAULT_ROOT=/path/to/vault
export LLM_COMMAND='claude -p'

python -m vault_brain walk --once
python -m vault_brain reconcile --once       # --backlog for an unlimited first pass
python -m vault_brain link --dry-run         # report what would be linked, write nothing
python -m vault_brain link --once            # --backlog for an unlimited first pass

The installed vault-brain script takes the same arguments. A stage exits early with a clear message if VAULT_ROOT or LLM_COMMAND is missing, so a misconfigured run cannot ledger episodes as processed.

Nightly, in order, with a gap for each stage to finish:

30 2 * * * cd /path/to/vault-brain && VAULT_ROOT=/path/to/vault LLM_COMMAND='claude -p' python3 -m vault_brain walk --once >> /path/to/logs/walk.log 2>&1
0  3 * * * cd /path/to/vault-brain && VAULT_ROOT=/path/to/vault LLM_COMMAND='claude -p' python3 -m vault_brain reconcile --once >> /path/to/logs/reconcile.log 2>&1
30 3 * * * cd /path/to/vault-brain && VAULT_ROOT=/path/to/vault LLM_COMMAND='claude -p' python3 -m vault_brain link --once >> /path/to/logs/link.log 2>&1

Take a copy of the vault before the first --backlog run; merges are reversible through the supersession links, but a snapshot is the fast way back.

Tests: pip install pytest then python3 -m pytest -q. The suite runs against a small fixture vault and never calls a real model; it stubs LLM_COMMAND with cat or a fixed-output shell snippet.

What it deliberately does not do

• It does not capture sessions. Something else must write episodes into the inbox; this toolkit only consolidates them.
• It does not delete. Retired notes are moved to the archive with a reason and a link to what replaced them.
• It does not do semantic search or embeddings. Candidates for merging and linking come from names and mentions, which is sparse, cheap and explainable.
• It does not invent structure. Bad or missing JSON from the model means no atoms, no merge, no edge, no insight.
• It does not manage model credentials or providers. One command on stdin is the whole interface.
• It does not write outside LONG_TERM_DIR, the inbox stamps and the ledgers.

Licence

MIT. See LICENSE.