diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 2989fe9..554f92c 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -10,7 +10,7 @@ { "name": "mnemo", "description": "7 memory skills for Claude Code and Codex via Obsidian — vault audit, knowledge search, bounded cross-runtime recall, connection discovery, session handoff, memory routing, review orchestration, and setup", - "version": "1.2.10", + "version": "1.2.12", "source": "./plugins/mnemo", "category": "productivity", "keywords": [ diff --git a/.gitignore b/.gitignore index 991c592..b74e608 100644 --- a/.gitignore +++ b/.gitignore @@ -12,3 +12,7 @@ __pycache__/ # working goal-state files — local, not part of the public plugin docs/plans/ .claude/loop-gate + +# personal permission grants — never ship these in a public plugin: they would +# hand every cloner a pre-approved path to scripts that rewrite a vault +.claude/settings.local.json diff --git a/CHANGELOG.md b/CHANGELOG.md index a46b4e8..d8503a2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,32 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ## [Unreleased] +## [1.2.12] - 2026-07-25 + +### Changed + +- **An unfinished tail now belongs to its own session note; the handoff gets a pointer.** `session` Step 5 and `depth-contract.md` previously routed open threads *into* the shared handoff. Measured on a live vault, that left only **9%** of fresh open items present in their own session note (34% for older ones) — the handoff had become the sole home of forward state, which is precisely how it reached 805 KiB and then could be neither read nor shrunk. A pointer line is bounded by the *number of sessions*; a copied tail is bounded by nothing. `hot-scan` (v1.2.11) collects the tails from where they now live, so the digest keeps working unchanged. + +### Added + +- **`handoff-index-upsert` action in `vault-write.py`** — writes one idempotent pointer line per session (`- 2026-07-25 · mnemo · open 3 · [[Session — …]]`), keyed on the session link so a mid-task checkpoint refreshes its own line instead of appending a twin. This also repairs a contract that was already broken: Step 5 demanded "the exact old section copied from read", but a large handoff read comes back truncated to a preview in which no complete section appears. Bounds — `handoff.maxLines` (180), `handoff.maxLineBytes` (200), `handoff.hardCapBytes` (38912) — are enforced in **bytes**, and trimming sacrifices the project label, never the `[[Session — …]]` link (a cut wikilink is a dead link). The block path (`archive-handoff`, `mode=blocks`) is untouched, along with its post-v1.1.11 regression suite. +- **`scripts/migrate-handoff-to-index.py`** — one-shot migration, `--dry-run` by default. Moves blocks **verbatim and whole** into the archive (never checkbox-extracted: 90 flat bullets and ~200 prose lines on the live file carry live state with no `- [ ]`), never edits a session note, never deletes, writes `.bak` first, and verifies afterwards that every migrated block is byte-present in the archive. Measured on the live handoff: **826 916 B → 35 910 B (−95.7%)**, 186 pointer lines. +- **`scripts/restore-handoff-from-bak.py`** — the rehearsed undo. The vault has no version control, so the migration's rollback is written and exercised *before* the migration may run; a restore itself saves a `.pre-restore` copy. Verified end-to-end on a copy of the live file: migrate → restore returns it byte-for-byte (md5 match). +- **`scripts/split-handoff-archive.py`** — splits the cold archive into per-month notes plus a small hub. A cold file is not harmless: at 717 KiB it is past the 256 KB read limit, so "it's still in the archive" was a promise nothing could keep, and it matched nearly every content search while being unopenable. A month is not automatically a readable unit either — June alone was 425 KB — so oversized months split into numbered parts under `--max-part-bytes` (default 200 KB). +- **43 new tests** across `test-handoff-index.py`, `test-migrate-handoff.py`, and `test-split-archive.py`, pinning the safety contract itself: dry-run writes nothing, blocks move verbatim, nothing is lost, backups precede writes, restore is byte-exact. Mutation-tested: counting characters instead of bytes, or dropping the hard cap, each fails the suite. + +## [1.2.11] - 2026-07-25 + +### Added + +- **Open-tails digest at SessionStart (`hot-scan.py` + `hooks.hotDigest`, default true).** Continuity finally has an automatic *reader*. `hot-scan.py` collects still-open tails from the pending sections of recent session notes and `mnemo-context.sh` injects a byte-capped, project-scoped summary next to the memory nudge — so a new session opens knowing where the last one stopped, instead of relying on someone remembering to open a note. Config: `hot.scope` (`project`/`all`, default project), `hot.windowDays` (7), `hot.maxKB` (8). Two measured properties are designed in: inside a pending section a bare `- bullet` counts as a tail (the v1.1.10 class — "live" is not always a `- [ ]`; 90 such bullets exist on the live vault), and every cap is measured in **bytes**, because `len(str)` under-counts a Cyrillic vault by ~34%. Any failure (no Obsidian, slow CLI, missing plugin root) degrades to nudge-only. The injected digest always carries the stale-premise warning — a handoff is a snapshot, not current state. +- **`handoff-resolver.py` + `health` Step 7.6 — report-only handoff triage.** A handoff does not shrink by archiving harder: the archiver may never move a block holding an open `- [ ]`, so it shrinks only when someone *decides* what those items still mean. The resolver reports the ceiling (what a complete resolution would free), blocks by payoff class (`fully-resolvable` / `partial` / `no-anchor` / `prose-pinned` / `fresh`), and the items an external arbiter could actually settle. Offline, read-only, no credentials — and permanently report-only, because closing someone's unfinished work is a human decision. It imports the keep-hot invariant from `vault-write.py` rather than copying it, so the report can never drift from the archiver it predicts. 🛑 It separates an item's **own** anchors from ones **inherited** from the block header: counting inherited anchors inflates the resolvable share from 27.6% to 60.6%. + +### Fixed + +- **`docs/session.md` promised a startup read that never existed.** It stated "when the next session starts, it reads `Meta — Session Handoff`" — no hook or skill ever did, and past ~256 KB no file read can. Corrected to describe the digest above, which is the reader that claim always assumed. +- **"Token bomb read every session" was the wrong reason to bound the handoff** (`config-schema.md`, `session/SKILL.md`). A host runtime truncates a large read to a preview, so the real cost of an un-rotated handoff is **unreadability**, not tokens. `session/SKILL.md` now also states the guard's own limit plainly: because a single open `- [ ]` pins a block, the archiver can legitimately be a no-op while the file sits 20× over `handoff.maxKB` (measured: 805 KiB, 863 open items, 0 eligible blocks). Size is bounded by the format of what gets written, not by the rotation guard. + ## [1.2.10] - 2026-07-23 ### Added @@ -939,7 +965,9 @@ Frontmatter now includes `session_id: {CLAUDE_SESSION_ID}` — disambiguates sam - `config.example.json` - MIT License -[Unreleased]: https://github.com/jojoprison/mnemo/compare/v1.2.10...HEAD +[Unreleased]: https://github.com/jojoprison/mnemo/compare/v1.2.12...HEAD +[1.2.12]: https://github.com/jojoprison/mnemo/compare/v1.2.11...v1.2.12 +[1.2.11]: https://github.com/jojoprison/mnemo/compare/v1.2.10...v1.2.11 [1.2.10]: https://github.com/jojoprison/mnemo/compare/v1.2.9...v1.2.10 [1.2.9]: https://github.com/jojoprison/mnemo/compare/v1.2.8...v1.2.9 [1.2.8]: https://github.com/jojoprison/mnemo/compare/v1.2.7...v1.2.8 diff --git a/docs/session.md b/docs/session.md index 7f9c219..2fae076 100644 --- a/docs/session.md +++ b/docs/session.md @@ -43,11 +43,14 @@ Handoff contents: ## Cross-Session Continuity -When the next session starts, it reads `Meta — Session Handoff`: +The next session starts with the **open-tails digest**: `hooks/mnemo-context.sh` runs `hot-scan.py` over the pending sections of recent session notes and injects a byte-capped summary (`hot.windowDays`, `hot.maxKB`) next to the memory nudge, scoped to the repo you are in (`hot.scope`). + - Picks up pending items - Has context about what happened - No more "what was I doing yesterday?" +**The handoff note itself is not read at startup, and never was.** That expectation was documented here for a long time, but no such path existed in any hook or skill — and a grown handoff cannot be read even on request (a file read caps at 256 KB / 25,000 tokens). Measured on a live vault in July 2026, an 805 KiB handoff was opened for continuity in 6% of sessions. The digest above is the automatic reader that claim always assumed; `Meta — Session Handoff` remains the durable index that `session` writes and `health` Step 7.6 triages. + ## When to Use - ✅ After completing a feature / PR / fix diff --git a/plugins/mnemo/.claude-plugin/plugin.json b/plugins/mnemo/.claude-plugin/plugin.json index 1c2df33..83cc7c2 100644 --- a/plugins/mnemo/.claude-plugin/plugin.json +++ b/plugins/mnemo/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "mn", "displayName": "mnemo", - "version": "1.2.10", + "version": "1.2.12", "description": "Persistent memory layer for Claude Code and Codex via Obsidian — 7 skills for vault health, knowledge search, verified cross-runtime project recall, connection discovery, session continuity, memory routing, skill-aware session review, and interactive setup. Bundled JSON-stdin atomic vault writer, argv-safe indexed reads, dual-runtime helpers, hybrid model routing, parallel CLI calls, SessionStart prewarm, incremental JSONL parsing, shared references, and semantic taxonomy roles.", "author": { "name": "jojoprison", diff --git a/plugins/mnemo/.codex-plugin/plugin.json b/plugins/mnemo/.codex-plugin/plugin.json index e692f1c..52881fe 100644 --- a/plugins/mnemo/.codex-plugin/plugin.json +++ b/plugins/mnemo/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "mnemo", - "version": "1.2.10", + "version": "1.2.12", "description": "Persistent memory layer for Codex and Claude Code via Obsidian: vault health, knowledge search, verified cross-runtime project recall, link discovery, session notes, memory routing, review orchestration, and setup.", "author": { "name": "jojoprison", diff --git a/plugins/mnemo/hooks/mnemo-context.sh b/plugins/mnemo/hooks/mnemo-context.sh index 1f97916..74513e6 100755 --- a/plugins/mnemo/hooks/mnemo-context.sh +++ b/plugins/mnemo/hooks/mnemo-context.sh @@ -30,8 +30,19 @@ else RUNTIME="claude" fi -OUT=$(python3 - "$CONFIG" "$RUNTIME" <<'PY' 2>/dev/null -import json, sys +# Project label for the open-tails digest. Use the git COMMON dir, not the +# worktree root: in a worktree `basename $(git rev-parse --show-toplevel)` is the +# worktree's name (floating-frolicking-goose), not the project's (mnemo). +PROJECT="" +if command -v git >/dev/null 2>&1; then + COMMON_DIR=$(git rev-parse --path-format=absolute --git-common-dir 2>/dev/null || true) + [ -n "$COMMON_DIR" ] && PROJECT=$(basename "$(dirname "$COMMON_DIR")") +fi + +PLUGIN_DIR="${PLUGIN_ROOT:-${CLAUDE_PLUGIN_ROOT:-}}" + +OUT=$(python3 - "$CONFIG" "$RUNTIME" "$PLUGIN_DIR" "$PROJECT" <<'PY' 2>/dev/null +import json, os, subprocess, sys try: cfg = json.load(open(sys.argv[1])) except Exception: @@ -52,6 +63,52 @@ msg = ( f"capture findings, decisions, and gotchas as they happen with {save} so a future " "session doesn't relearn them." ) + + +def hot_digest() -> str: + """Open tails from recent session notes — the forward-state half of memory. + + Best-effort and byte-capped: any failure (no Obsidian, no plugin root, slow + CLI) yields an empty string, so the nudge above is never lost to it. + Gate: hooks.hotDigest, defaults to true. + """ + if cfg.get("hooks", {}).get("hotDigest", True) is False: + return "" + plugin_root = sys.argv[3] if len(sys.argv) > 3 else "" + script = os.path.join(plugin_root, "scripts", "hot-scan.py") + if not plugin_root or not os.path.isfile(script): + return "" + try: + probe = subprocess.run( + ["obsidian", "vault", f"vault={vault}"], + capture_output=True, text=True, timeout=3, check=False) + except Exception: + return "" + vault_path = "" + for line in probe.stdout.splitlines(): + field, separator, value = line.partition("\t") + if field == "path" and separator and value: + vault_path = value + break + if not vault_path or not os.path.isdir(vault_path): + return "" + command = [sys.executable, script, vault_path] + project = sys.argv[4] if len(sys.argv) > 4 else "" + # Project scope keeps the digest about the repo in front of you; "all" is + # for people who want the cross-project view every session. + if cfg.get("hot", {}).get("scope", "project") == "project" and project: + command += ["--project", project] + try: + result = subprocess.run( + command, capture_output=True, text=True, timeout=5, check=False) + except Exception: + return "" + return result.stdout.strip() if result.returncode == 0 else "" + + +digest = hot_digest() +if digest: + msg = f"{msg}\n\n{digest}" payload = { "hookSpecificOutput": { "hookEventName": "SessionStart", diff --git a/plugins/mnemo/references/config-schema.md b/plugins/mnemo/references/config-schema.md index 55dc59e..9d9639e 100644 --- a/plugins/mnemo/references/config-schema.md +++ b/plugins/mnemo/references/config-schema.md @@ -132,8 +132,15 @@ The `recall` section is optional and ships off. `recall.codeGraph` (default `nul | `hooks.stopNudge` | At session end, if the session looks worth-saving but the save and/or session skill never ran, block once with the current runtime's native commands. This is **opt-in, default false**; an anti-loop governor prevents repeated blocking | hooks/mnemo-stop-nudge.sh | | `hooks.invocationEcho` | Claude Code only: on a `/mn:*` slash command, emit a `systemMessage` line (`🧠 mnemo: /mn:save → skill body loaded`) via the Claude-only `UserPromptExpansion` hook — a **deterministic** invocation confirmation that, unlike the in-body marker, does not depend on model compliance. Codex does not load this unsupported event. Default **true**; set false to silence | hooks/mnemo-skill-echo.sh | | `memory.indexWarnKB` | Early loaded-content byte warning for Claude `MEMORY.md`. Current loader limits are 200 lines or 25,000 bytes after stripping leading YAML frontmatter and block-level HTML comments; health always reports either hard-limit breach independently of this threshold. Default **22** | health | -| `handoff.maxKB` | Size ceiling (KB) before `vault-write.py archive-handoff` rotates CLOSED old blocks into ` Archive` (cold). The handoff is a live index, not a store; un-rotated it becomes a token bomb read every session. Default **40** | session | +| `handoff.maxKB` | Size ceiling (KB) before `vault-write.py archive-handoff` rotates CLOSED old blocks into ` Archive` (cold). The handoff is a live index, not a store. What an un-rotated one costs is **unreadability, not tokens** — a host runtime truncates a large read to a preview, and a file read refuses outright past 256 KB / 25,000 tokens, so past a point nothing can consult it and live items drown under months of closed history. Default **40** | session | | `handoff.keepDays` | Blocks newer than this stay hot regardless of status; older **and** closed (no open `- [ ]`) move to the archive. Default **14** | session | +| `handoff.maxLines` | Pointer lines kept in the handoff index before the oldest rotate out. Default **180** — at the maintainer's measured pace (5.9 sessions/day lifetime, 12.4 over the last week) that is a 14-31 day window that never needs retuning as the pace changes | session | +| `handoff.maxLineBytes` | Per-line ceiling for an index line, in **bytes**. Default **200** (measured: the longest real session-note link is 177 B, so the pointer always fits). Trimming sacrifices the project label — never the `[[Session — …]]` link, because a cut wikilink is a dead link | session | +| `handoff.hardCapBytes` | Whole-file ceiling for the index, in **bytes**; oldest pointers drop until it fits. Default **38912** (38 KiB, under the 40 KB `maxKB` guard) | session | +| `hooks.hotDigest` | Inject the open-tails digest alongside the SessionStart nudge — the forward-state half of memory ("where did I stop"), collected from recent session notes' pending sections. This is the automatic reader continuity never had: a full handoff cannot be read at all (a file read caps at 256 KB / 25,000 tokens), so before this the digest existed only if someone opened the note by hand. Degrades to nudge-only on any failure. Default **true**; set false to silence | hooks/mnemo-context.sh | +| `hot.scope` | `"project"` limits the digest to the repo you are in (project label = the basename of the git **common** dir, so a worktree reports its project, not the worktree name); `"all"` gives the cross-project view. Default **project** | hooks/mnemo-context.sh | +| `hot.windowDays` | How far back the digest looks for open tails. Default **7** | hot-scan | +| `hot.maxKB` | Hard **byte** cap on the rendered digest; oldest tails drop first until it fits. Measured in bytes, never characters — `len(str)` under-counts a Cyrillic vault by ~34%. Default **8** | hot-scan | | `review.full.autoConnect` | In `review --full`, the internal `connect` step **applies** its link suggestions without a per-suggestion prompt instead of only showing them (connect Step 5.5, review Step 9A). The user typing `--full` **and** setting this flag is the consent — mirroring `review.lint.autoStampReviewed`, the opt-in that lets a skill write. A standalone `/mn:connect` is unaffected and never auto-applies. Default **false** (non-destructive default; connect stays suggest-only for everyone who doesn't opt in) | review, connect | | `review.staleDays.default` | Days before a note becomes a review candidate when its type has no specific entry. Default **30**. (`review.staleDays` may also be a bare integer — a single uniform threshold for every type.) | health | | `review.staleDays.{type}` | Per-type staleness cadence (key = a taxonomy `type` you actually use: `atom`/`molecule`/`source`/`session`/`moc`). A fast-moving fact ages quicker than an architectural decision | health | diff --git a/plugins/mnemo/references/depth-contract.md b/plugins/mnemo/references/depth-contract.md index ed74624..2202670 100644 --- a/plugins/mnemo/references/depth-contract.md +++ b/plugins/mnemo/references/depth-contract.md @@ -28,7 +28,7 @@ Step 0b, applied across the whole close-out. | An actionable never-X / always-Y rule tied to code | `save` Step 3.5 → `.claude/rules/` | path-scoped auto-inject, not recall | | The session narrative (what happened, the arc, decisions-in-context) | `session` | one note + handoff, never a duplicate of today's | | Connections between the new notes (incl. non-obvious) | `connect` | suggest-only links (auto-applied only under `review.full.autoConnect`), no orphans | -| A future-useful thread not finished | `session` handoff `- [ ]` pending | thin live index | +| A future-useful thread not finished | the session note's own `## Next steps / pending` | the handoff gets a pointer line, never a copy | The business-logic / pains / mental-model layer is the crown material — it routes to `save`'s typed `principle` / `pain` / `stance` atoms, **not** the session narrative. That is how "capture the way I diff --git a/plugins/mnemo/scripts/handoff-resolver.py b/plugins/mnemo/scripts/handoff-resolver.py new file mode 100644 index 0000000..7b74aef --- /dev/null +++ b/plugins/mnemo/scripts/handoff-resolver.py @@ -0,0 +1,263 @@ +#!/usr/bin/env python3 +"""Report-only triage view of a block-format session handoff. + +A handoff grows because the archiver may never touch a block that still holds an +open `- [ ]` — an unkept promise must not slip silently into cold storage. So the +file is not cleaned by archiving harder; it is cleaned by *deciding* what those +open items still mean. This script produces that worklist, plus honest +arithmetic about what deciding can actually buy. + +It is deliberately **offline and read-only**: no network, no writes, no shell. +Whether a PR merged or an issue closed is a question for an agent with the right +credentials — this script only says which items carry an answerable anchor. + +Two measured properties shape the output (live vault, 2026-07-25): + +- **Anchor inheritance lies.** Counting anchors found in a block *header* as the + item's own inflates "resolvable" from 27.6% to 60.6%. Item anchors and + inherited anchors are reported separately; only item anchors count. +- **One anchorless item pins a whole block.** Blocks are classed by payoff, so + the report never promises bytes the archiver would refuse to free. + +The keep-hot rule itself is imported from `vault-write.py` rather than copied: +a second implementation would drift from the archiver it is predicting. + +Usage: + handoff-resolver.py [--keep-days N] [--limit N] + [--json] [--today YYYY-MM-DD] +""" +from __future__ import annotations + +import argparse +import datetime as dt +import importlib.util +import json +import os +import re +import sys + +SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__)) +WRITER_PATH = os.path.join(SCRIPT_DIR, "vault-write.py") + +# Anchors an external arbiter could actually answer. +LINEAR_RE = re.compile(r"\b([A-Z]{2,7})-(\d{1,5})\b") +PR_STRONG_RE = re.compile(r"(?:\bPR\s*#?|\bpull/|\bpull request\s*#?)(\d{1,6})\b", re.IGNORECASE) +PR_BARE_RE = re.compile(r"(? dict[str, list[str]]: + linear = sorted({f"{m.group(1)}-{m.group(2)}" for m in LINEAR_RE.finditer(text)}) + pr = sorted({m.group(1) for m in PR_STRONG_RE.finditer(text)} + | {m.group(1) for m in PR_BARE_RE.finditer(text)}) + paths = sorted({m.group(0) for m in PATH_RE.finditer(text)}) + return {"linear": linear, "pr": pr, "path": paths} + + +def has_external_anchor(anchors: dict[str, list[str]]) -> bool: + """Only Linear keys and PR numbers can be settled by an external arbiter. + + A file path proves the tail is about code, not that anything closed it. + """ + return bool(anchors["linear"] or anchors["pr"]) + + +def analyse(body: str, *, writer, today: dt.date, keep_days: int) -> dict: + header, blocks = writer.split_handoff(body) + cutoff = today - dt.timedelta(days=keep_days) + + block_rows: list[dict] = [] + items: list[dict] = [] + for chunk in blocks: + date = writer.handoff_block_date(chunk) + first_line = chunk.split("\n", 1)[0] + inherited = anchors_of(first_line) + prose_live = writer.HEADER_PENDING_RE.search(first_line) is not None + size = len(chunk.encode()) + + open_texts = [ + line.split("- [ ]", 1)[1].strip() if "- [ ]" in line else line.strip() + for line in chunk.splitlines() + if writer.OPEN_TODO_RE.search(line) + ] + block_items = [] + for text in open_texts: + own = anchors_of(text) + resolvable = has_external_anchor(own) + block_items.append({ + "text": text[:300], + "block_date": date.isoformat() if date else None, + "anchors": own, + "inherited": inherited, + "resolvable": resolvable, + "needs_person": bool(PEOPLE_RE.search(text)), + }) + + fresh = date is None or date >= cutoff + if fresh: + payoff = "fresh" + elif prose_live: + # Prose-live blocks stay hot even with every checkbox closed, so + # resolving their items frees nothing. Never promise those bytes. + payoff = "prose-pinned" + elif not block_items: + payoff = "already-cold" + elif all(i["resolvable"] for i in block_items): + payoff = "fully-resolvable" + elif any(i["resolvable"] for i in block_items): + payoff = "partial" + else: + payoff = "no-anchor" + + block_rows.append({ + "date": date.isoformat() if date else None, + "bytes": size, + "open": len(block_items), + "prose_live": prose_live, + "payoff": payoff, + "header": first_line[:160], + }) + items.extend(block_items) + + freed = sum(b["bytes"] for b in block_rows if b["payoff"] == "fully-resolvable") + total = len(body.encode()) + resolvable_items = sum(1 for i in items if i["resolvable"]) + anchorless = sum( + 1 for i in items + if not (i["anchors"]["linear"] or i["anchors"]["pr"] or i["anchors"]["path"]) + ) + + by_payoff: dict[str, dict] = {} + for row in block_rows: + entry = by_payoff.setdefault(row["payoff"], {"blocks": 0, "bytes": 0, "open": 0}) + entry["blocks"] += 1 + entry["bytes"] += row["bytes"] + entry["open"] += row["open"] + + return { + "bytes": total, + "header_bytes": len(header.encode()), + "blocks_total": len(block_rows), + "open_total": len(items), + "keep_days": keep_days, + "anchors": { + "resolvable": resolvable_items, + "resolvable_share": round(resolvable_items / len(items), 4) if items else 0, + "none": anchorless, + "needs_person": sum(1 for i in items if i["needs_person"]), + }, + "ceiling": { + "current_bytes": total, + "freed_bytes": freed, + "freed_pct": round(100 * freed / total, 2) if total else 0, + "floor_bytes": total - freed, + }, + "classes": by_payoff, + "blocks": block_rows, + "items": items, + } + + +def render(report: dict, limit: int) -> str: + lines = [ + f"handoff: {report['bytes']} B · {report['blocks_total']} блоков · " + f"{report['open_total']} открытых (keepDays={report['keep_days']})", + "", + "Что даст полный резолв открытых пунктов:", + f" освободится {report['ceiling']['freed_bytes']} B " + f"({report['ceiling']['freed_pct']}%) → пол {report['ceiling']['floor_bytes']} B", + "", + "Блоки по классу выигрыша:", + ] + order = ["fully-resolvable", "partial", "no-anchor", "prose-pinned", "fresh", "already-cold"] + for name in order: + entry = report["classes"].get(name) + if entry: + lines.append( + f" {name:<18} {entry['blocks']:>4} блоков · {entry['bytes']:>8} B · " + f"{entry['open']:>4} открытых") + anchors = report["anchors"] + lines += [ + "", + f"Якоря: резолвимо внешним арбитром {anchors['resolvable']}/{report['open_total']} " + f"({round(100 * anchors['resolvable_share'], 1)}%) · " + f"без якоря {anchors['none']} · требует человека {anchors['needs_person']}", + "", + f"Первые {limit} пунктов с якорем (кандидаты на проверку):", + ] + shown = 0 + for item in report["items"]: + if not item["resolvable"] or shown >= limit: + continue + keys = ", ".join(item["anchors"]["linear"] + [f"#{n}" for n in item["anchors"]["pr"]]) + lines.append(f" [{item['block_date']}] {keys} — {item['text'][:110]}") + shown += 1 + lines.append("") + lines.append("Report-only: ничего не изменено. Решения принимает человек.") + return "\n".join(lines) + + +def main() -> int: + parser = argparse.ArgumentParser( + description="Report-only triage view of a block-format handoff.") + parser.add_argument("handoff_path") + parser.add_argument("--keep-days", type=int, default=14) + parser.add_argument("--limit", type=int, default=25) + parser.add_argument("--json", action="store_true") + parser.add_argument("--today", help="override today's date (tests)") + args = parser.parse_args() + + path = os.path.expanduser(args.handoff_path) + if not os.path.isfile(path): + print(f"error: not a file: {path}", file=sys.stderr) + return 1 + try: + with open(path, encoding="utf-8") as handle: + body = handle.read() + except OSError as error: + print(f"error: cannot read {path}: {error}", file=sys.stderr) + return 1 + + try: + writer = load_writer() + except Exception as error: # noqa: BLE001 — a broken import must not traceback + print(f"error: cannot load vault-write.py: {error}", file=sys.stderr) + return 1 + + if args.today: + try: + today = dt.date.fromisoformat(args.today) + except ValueError: + print("error: --today must be YYYY-MM-DD", file=sys.stderr) + return 1 + else: + today = dt.date.today() + + report = analyse(body, writer=writer, today=today, keep_days=args.keep_days) + if args.json: + print(json.dumps(report, ensure_ascii=False)) + else: + print(render(report, args.limit)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/plugins/mnemo/scripts/hot-scan.py b/plugins/mnemo/scripts/hot-scan.py new file mode 100644 index 0000000..49a66e4 --- /dev/null +++ b/plugins/mnemo/scripts/hot-scan.py @@ -0,0 +1,352 @@ +#!/usr/bin/env python3 +"""Collect still-open tails from recent session notes into a small digest. + +This is the *reader* the handoff never had. Measured on the live vault +(2026-07-25): `Meta — Session Handoff` was read for continuity in 6% of +sessions, because nothing automatic could read it — 805 KiB exceeds both the +256 KB and the 25000-token caps of a file read. A digest small enough to inject +at SessionStart turns "forward state" from an artifact nobody opens into one +line of context every session. + +Two failure classes are designed against, both measured, not assumed: + +- **"Live" is not always a checkbox** (the v1.1.10 class). Pending sections + carry flat bullets with no `- [ ]` — 90 of them across 14 handoff blocks. + A `- [ ]`-only scan silently drops them, which is exactly the bug the + archiver's prose-pending detector was added to fix. So inside a pending + section a bare `- bullet` counts too; outside one, only an explicit `- [ ]`. + +- **Sizing in characters** under-counts a Cyrillic vault by ~34% (UTF-8 is + ~1.35 B/char here). Every cap in this file is in BYTES. + +Read-only: never writes, never follows symlinks out of the vault, never runs a +shell. Stdlib only (system pip is PEP-668 locked). + +Usage: + hot-scan.py [--project NAME] [--window-days N] [--max-kb N] + [--handoff NOTE] [--config PATH] [--json] [--today YYYY-MM-DD] + +Default output is the injectable digest text; `--json` returns the structure. +""" +from __future__ import annotations + +import argparse +import datetime as dt +import json +import os +import re +import sys + +DEFAULT_WINDOW_DAYS = 7 +DEFAULT_MAX_KB = 8 +MAX_ITEM_CHARS = 200 +SKIP_DIRS = {".obsidian", ".trash", ".git", "node_modules"} + +# A heading that opens a section of unfinished work. Deliberately generous: +# missing a pending section costs a lost tail, a false positive costs one noisy +# line. Russian and English, because the vault is both. +PENDING_HEADING_RE = re.compile( + r"^#{1,6}\s*.*(?:" + r"pending|next\s*steps?|to\s?-?do|follow[\s-]?up|open\s+(?:items?|threads?)|" + r"остал|жд[её]м|хвост|незакрыт|в\s+процессе|не\s+закры" + r")", + re.IGNORECASE, +) +OPEN_CHECKBOX_RE = re.compile(r"^\s*[-*]\s+\[\s\]\s*(.+?)\s*$") +CLOSED_CHECKBOX_RE = re.compile(r"^\s*[-*]\s+\[[xX~-]\]") +FLAT_BULLET_RE = re.compile(r"^\s*[-*]\s+(?!\[)(.+?)\s*$") +HEADING_RE = re.compile(r"^#{1,6}\s") +# A bullet already carrying a done/rejected marker is not a tail. +DONE_MARKER_RE = re.compile(r"^\s*(?:✅|❌|~~|DONE\b|СДЕЛАНО\b)", re.IGNORECASE) + +STALE_PREMISE_WARNING = ( + "⚠️ премиса могла протухнуть — сверь с origin/main и Linear перед действием." +) + + +def parse_frontmatter(path: str) -> dict[str, str] | None: + """Minimal scalar parse of a leading `---` block; {} when absent.""" + try: + with open(path, encoding="utf-8") as handle: + if handle.readline().strip() != "---": + return {} + fields: dict[str, str] = {} + for index, line in enumerate(handle): + if index >= 100 or line.strip() == "---": + break + key, _, value = line.partition(":") + key, value = key.strip(), value.strip().strip('"').strip("'") + if key and value: + fields[key] = value + return fields + except OSError: + return None + + +def parse_date(value: str) -> dt.date | None: + try: + return dt.datetime.strptime(value[:10], "%Y-%m-%d").date() + except (ValueError, TypeError): + return None + + +def clip(text: str) -> str: + """Bound one item so a single runaway line cannot eat the whole cap.""" + return text if len(text) <= MAX_ITEM_CHARS else text[: MAX_ITEM_CHARS - 1] + "…" + + +def extract_tails(body: str) -> list[str]: + """Open tails from a note body. + + Inside a pending section: any bullet. Outside it: only an explicit `- [ ]` + (a bare bullet elsewhere is narrative — Findings, links, evidence). + """ + tails: list[str] = [] + in_pending = False + for line in body.splitlines(): + if HEADING_RE.match(line): + in_pending = bool(PENDING_HEADING_RE.match(line)) + continue + if CLOSED_CHECKBOX_RE.match(line): + continue + open_box = OPEN_CHECKBOX_RE.match(line) + if open_box: + text = open_box.group(1).strip() + if text and not DONE_MARKER_RE.match(text): + tails.append(clip(text)) + continue + if not in_pending: + continue + bullet = FLAT_BULLET_RE.match(line) + if bullet: + text = bullet.group(1).strip() + if text and not DONE_MARKER_RE.match(text): + tails.append(clip(text)) + return tails + + +def split_handoff_blocks(body: str) -> list[tuple[dt.date, str]]: + """Dated `## YYYY-MM-DD` blocks of the legacy handoff, if one is scanned.""" + blocks: list[tuple[dt.date, str]] = [] + for chunk in re.split(r"(?m)(?=^## \d{4}-\d{2}-\d{2})", body): + match = re.match(r"## (\d{4}-\d{2}-\d{2})", chunk) + if match is None: + continue + date = parse_date(match.group(1)) + if date is not None: + blocks.append((date, chunk)) + return blocks + + +def scan_vault( + vault: str, + *, + cutoff: dt.date, + project: str | None, +) -> list[dict]: + """Every open tail from session notes dated on/after `cutoff`.""" + found: list[dict] = [] + for root, dirs, files in os.walk(vault): + dirs[:] = [d for d in dirs if d not in SKIP_DIRS] + for name in files: + if not name.endswith(".md"): + continue + path = os.path.join(root, name) + fields = parse_frontmatter(path) + if not fields or fields.get("type") != "session": + continue + date = parse_date(fields.get("date", "")) + if date is None or date < cutoff: + continue + note_project = fields.get("project") or "—" + if project and note_project != project: + continue + try: + with open(path, encoding="utf-8") as handle: + body = handle.read() + except OSError: + continue + for text in extract_tails(body): + found.append({ + "text": text, + "project": note_project, + "date": date.isoformat(), + "note": name[:-3], + }) + return found + + +def scan_handoff(path: str, *, cutoff: dt.date, project: str | None) -> list[dict]: + """Open tails from the legacy block-format handoff, window-limited. + + Kept as an opt-in source: 66% of its open items are not covered by any + session note, so until the migration lands they exist nowhere else. + """ + try: + with open(path, encoding="utf-8") as handle: + body = handle.read() + except OSError: + return [] + found: list[dict] = [] + for date, chunk in split_handoff_blocks(body): + if date < cutoff: + continue + header = chunk.split("\n", 1)[0] + # `## 2026-07-25 — mnemo: …` → "mnemo"; the project is prose here, so + # this is a best-effort label, never a filter people depend on. + label = re.sub(r"^## \d{4}-\d{2}-\d{2}\s*[—-]?\s*", "", header) + block_project = re.split(r"[::]", label, maxsplit=1)[0].strip()[:40] or "—" + if project and block_project != project: + continue + for text in extract_tails(chunk): + found.append({ + "text": text, + "project": block_project, + "date": date.isoformat(), + "note": None, + }) + return found + + +def dedupe(items: list[dict]) -> list[dict]: + """Same tail recorded in both a session note and the handoff — keep one.""" + seen: set[str] = set() + unique: list[dict] = [] + for item in items: + # Compare the WHOLE normalized text: truncating the key merges distinct + # tails that happen to share a prefix (a long quoted path, a repeated + # "Проверить, что …" opener) — a silent loss, not a dedupe. + key = re.sub(r"\W+", "", item["text"].lower()) + if key in seen: + continue + seen.add(key) + unique.append(item) + return unique + + +def build_groups(items: list[dict]) -> list[dict]: + """Group by project, newest project first, newest item first inside it.""" + by_project: dict[str, list[dict]] = {} + for item in items: + by_project.setdefault(item["project"], []).append(item) + groups = [] + for name, group_items in by_project.items(): + group_items.sort(key=lambda i: i["date"], reverse=True) + groups.append({ + "project": name, + "count": len(group_items), + "newest": group_items[0]["date"], + "items": group_items, + }) + groups.sort(key=lambda g: g["newest"], reverse=True) + return groups + + +def render(groups: list[dict], *, window_days: int, max_bytes: int) -> tuple[str, list[dict]]: + """Render the digest under a hard BYTE cap, dropping oldest items first. + + Returns the text plus the groups that actually survived the cap, so the + JSON payload can never claim more than the digest shows. + """ + if not groups: + return "", [] + + total = sum(g["count"] for g in groups) + head = f"📌 Открытые хвосты: {total} (окно {window_days} дн.)" + tail = STALE_PREMISE_WARNING + + def compose(items: list[dict]) -> tuple[str, list[dict]]: + surviving = build_groups(items) + lines = [head] + for group in surviving: + lines.append(f"{group['project']} · {group['count']}:") + lines.extend(f" · {item['text']}" for item in group["items"]) + lines.append(tail) + return "\n".join(lines), surviving + + # Newest first, so dropping from the end drops the oldest. + ranked = sorted( + (item for group in groups for item in group["items"]), + key=lambda i: i["date"], + reverse=True, + ) + # Measure the RENDERED text, not the sum of item costs: every group adds a + # header line, so per-item accounting silently overshoots the cap once the + # digest spans several projects (measured: 8335 B against an 8192 B cap). + while ranked: + text, surviving = compose(ranked) + if len(text.encode()) <= max_bytes: + return text, surviving + ranked.pop() + return "", [] + + +def main() -> int: + parser = argparse.ArgumentParser( + description="Digest of still-open tails from recent session notes.") + parser.add_argument("vault_path", help="path to the Obsidian vault root") + parser.add_argument("--project", help="keep only this project's tails") + parser.add_argument("--window-days", type=int, help=f"default {DEFAULT_WINDOW_DAYS}") + parser.add_argument("--max-kb", type=int, help=f"hard byte cap, default {DEFAULT_MAX_KB}") + parser.add_argument("--handoff", help="also scan this legacy handoff note (path)") + parser.add_argument("--config", default="~/.mnemo/config.json") + parser.add_argument("--json", action="store_true", help="emit the structure") + parser.add_argument("--today", help="override today's date (tests)") + args = parser.parse_args() + + vault = os.path.expanduser(args.vault_path) + if not os.path.isdir(vault): + print(f"error: not a directory: {vault}", file=sys.stderr) + return 1 + + config: dict = {} + try: + with open(os.path.expanduser(args.config)) as handle: + config = json.load(handle).get("hot", {}) or {} + except (OSError, json.JSONDecodeError, AttributeError): + config = {} + + def setting(name: str, cli_value, fallback: int) -> int: + if cli_value is not None: + return cli_value + try: + return int(config.get(name, fallback)) + except (TypeError, ValueError): + return fallback + + window_days = setting("windowDays", args.window_days, DEFAULT_WINDOW_DAYS) + max_kb = setting("maxKB", args.max_kb, DEFAULT_MAX_KB) + + today = parse_date(args.today) if args.today else dt.date.today() + if today is None: + print("error: --today must be YYYY-MM-DD", file=sys.stderr) + return 1 + cutoff = today - dt.timedelta(days=window_days) + + items = scan_vault(vault, cutoff=cutoff, project=args.project) + if args.handoff: + items += scan_handoff( + os.path.expanduser(args.handoff), cutoff=cutoff, project=args.project) + items = dedupe(items) + + groups = build_groups(items) + digest, surviving = render( + groups, window_days=window_days, max_bytes=max_kb * 1024) + + if args.json: + print(json.dumps({ + "digest": digest, + "groups": surviving, + "total": sum(g["count"] for g in groups), + "shown": sum(g["count"] for g in surviving), + "windowDays": window_days, + "maxKB": max_kb, + "bytes": len(digest.encode()), + }, ensure_ascii=False)) + elif digest: + print(digest) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/plugins/mnemo/scripts/vault-write.py b/plugins/mnemo/scripts/vault-write.py index 96ccb94..f04324f 100755 --- a/plugins/mnemo/scripts/vault-write.py +++ b/plugins/mnemo/scripts/vault-write.py @@ -37,7 +37,14 @@ MAX_COMPONENT_BYTES = 255 CLI_TIMEOUT_SECONDS = 15 RESERVED_DIRS = {".obsidian", ".trash"} -VALID_ACTIONS = {"create", "replace", "insert", "append", "archive-handoff"} +VALID_ACTIONS = { + "create", + "replace", + "insert", + "append", + "archive-handoff", + "handoff-index-upsert", +} DATE_HEADER_RE = re.compile(r"^## (\d{4}-\d{2}-\d{2})") OPEN_TODO_RE = re.compile(r"\[ \]") HEADER_PENDING_RE = re.compile( @@ -689,6 +696,131 @@ def transform_append(current: CurrentFile, payload: dict[str, Any]) -> str: return current.body + string(payload, "content", allow_empty=True) +INDEX_LINE_RE = re.compile(r"^- (\d{4}-\d{2}-\d{2}) · ") +INDEX_DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$") +DEFAULT_INDEX_MAX_LINES = 180 +DEFAULT_INDEX_MAX_LINE_BYTES = 200 +DEFAULT_INDEX_HARD_CAP_BYTES = 38912 + + +def clip_to_bytes(value: str, budget: int) -> str: + """Longest prefix of `value` fitting `budget` UTF-8 bytes. + + Byte-wise, never character-wise: a Cyrillic label is ~2 B/char, so a + character budget admits roughly twice what it promises. + """ + if budget <= 0: + return "" + encoded = value.encode() + if len(encoded) <= budget: + return value + trimmed = value + while trimmed and len(trimmed.encode()) > budget: + trimmed = trimmed[:-1] + return trimmed + + +def build_index_line( + *, + date: str, + project: str, + open_count: int, + session_note: str, + max_line_bytes: int, +) -> str: + """One pointer line: `- DATE · project · open N · [[Session — …]]`. + + The link is load-bearing and never truncated — a cut wikilink is a dead + link, strictly worse than a long line. Only the project label gives way. + """ + link = f"[[{session_note}]]" + suffix = f" · open {open_count} · {link}" + fixed = f"- {date}" + separator = " · " + # Budget = ceiling minus every part that is NOT the project label. Measured + # in bytes: "·" alone is 2 bytes, and a Cyrillic label ~2 B/char. + overhead = ( + len(fixed.encode()) + len(separator.encode()) + len(suffix.encode()) + ) + label = clip_to_bytes(project.strip(), max_line_bytes - overhead) + if label: + return f"{fixed} · {label}{suffix}" + return f"{fixed}{suffix}" + + +def transform_handoff_index(current: CurrentFile, payload: dict[str, Any]) -> str: + """Upsert one session's pointer line into the handoff index. + + Idempotency keys on `[[session_note]]`, so a mid-session checkpoint refreshes + its own line instead of appending a twin. This is what the previous + read-then-replace-exact-section contract could not do on a grown handoff: + a large read comes back truncated to a preview, so the section to copy was + frequently not even visible to the caller. + + Bounds are enforced in this order — per-line bytes, then line count, then a + whole-file byte cap — and every one of them counts bytes. + """ + session_note = string(payload, "session_note") + if any(ch in session_note for ch in ("[", "]", "|", "\n", "\r", "\0")): + fail("input_error", "session_note must not contain wikilink syntax or newlines") + date = string(payload, "date") + if INDEX_DATE_RE.match(date) is None: + fail("input_error", "date must be YYYY-MM-DD") + project = optional_string(payload, "project") or "" + if any(ch in project for ch in ("\n", "\r", "\0")): + fail("input_error", "project must not contain newlines") + open_count = nonnegative_integer(payload, "open_count", 0) + max_lines = nonnegative_integer(payload, "max_lines", DEFAULT_INDEX_MAX_LINES) + max_line_bytes = nonnegative_integer( + payload, "max_line_bytes", DEFAULT_INDEX_MAX_LINE_BYTES + ) + hard_cap = nonnegative_integer( + payload, "hard_cap_bytes", DEFAULT_INDEX_HARD_CAP_BYTES + ) + + lines = current.body.split("\n") + positions = [i for i, line in enumerate(lines) if INDEX_LINE_RE.match(line)] + entries = [lines[i] for i in positions] + link = f"[[{session_note}]]" + entries = [entry for entry in entries if link not in entry] + entries.append( + build_index_line( + date=date, + project=project, + open_count=open_count, + session_note=session_note, + max_line_bytes=max_line_bytes, + ) + ) + # Newest first; a stable sort keeps same-day sessions in insertion order. + entries.sort(key=lambda e: INDEX_LINE_RE.match(e).group(1), reverse=True) + if max_lines: + entries = entries[:max_lines] + + if positions: + anchor = positions[0] + rest = [line for i, line in enumerate(lines) if i not in set(positions)] + # `anchor` counts positions in the ORIGINAL list; recompute where that + # lands once the old index lines are gone. + before = len([i for i in range(anchor) if i not in set(positions)]) + head, tail = rest[:before], rest[before:] + else: + head, tail = lines, [] + while head and not head[-1].strip(): + head.pop() + head.append("") + + def assemble(rows: list[str]) -> str: + return "\n".join(head + rows + tail) + + body = assemble(entries) + # Whole-file cap last: drop oldest pointers until the file fits. + while entries and len(body.encode()) > hard_cap: + entries.pop() + body = assemble(entries) + return body + + def split_handoff(body: str) -> tuple[str, list[str]]: match = re.search(r"^## \d{4}-\d{2}-\d{2}", body, re.MULTILINE) if match is None: @@ -951,7 +1083,8 @@ def write(payload: dict[str, Any]) -> dict[str, Any]: if action not in VALID_ACTIONS: fail( "input_error", - "action must be create, replace, insert, append, or archive-handoff", + "action must be create, replace, insert, append, archive-handoff, " + "or handoff-index-upsert", ) ensure_supported_platform() vault = discover_vault(string(payload, "vault")) @@ -988,6 +1121,8 @@ def write(payload: dict[str, Any]) -> dict[str, Any]: body = transform_replace(current, payload) elif action == "insert": body = transform_insert(current, payload) + elif action == "handoff-index-upsert": + body = transform_handoff_index(current, payload) else: body = transform_append(current, payload) raw = ensure_text_size(body, "resulting note") diff --git a/plugins/mnemo/skills/health/SKILL.md b/plugins/mnemo/skills/health/SKILL.md index 36bd672..aa095c4 100644 --- a/plugins/mnemo/skills/health/SKILL.md +++ b/plugins/mnemo/skills/health/SKILL.md @@ -176,6 +176,25 @@ Emit a verdict per candidate: Verdicts are **triage, not truth** — on `haiku` especially, expect false positives; even on `opus`, surface them as questions. The **only** write health ever performs is the `reviewed:` auto-stamp above, and only ever on a **still-valid** verdict — never content, never on update-needed/contradicts. It fires only inside this lint pass (which is itself off unless `review.lint.enabled` is true), and can be turned back to suggest-only with `autoStampReviewed: false`. The user stays in control. +### Step 7.6: Handoff Triage (report-only, skipped when there is no handoff) + +Steps 7/7.5 judge whether *notes* have rotted. This judges whether the **handoff** has — a different failure with a different cause. The archiver may never move a block that still holds an open `- [ ]` (an unkept promise must not slip silently into cold storage), so a handoff does not shrink by archiving harder; it shrinks by someone *deciding* what those open items still mean. Run it whenever `handoff_note` is configured and the file exceeds `handoff.maxKB`: + +```bash +python3 "/scripts/handoff-resolver.py" "{vault_path}/{handoff_note}.md" \ + --keep-days {handoff.keepDays} --limit 25 +``` + +Offline and read-only — it never writes and never calls a network. Report its three blocks as-is: + +- **Ceiling.** How many bytes a *complete* resolution of every open item would free. Quote it honestly even when it is small: on the maintainer's live vault it was **2.47%** (20 KB of 805 KB). Triage buys correctness, not size — say so instead of implying a cleanup will fix a bloated file. +- **Blocks by payoff.** `fully-resolvable` (every open item carries an external anchor → resolving frees the whole block) · `partial` (one anchorless item pins the rest — the common case) · `no-anchor` · `prose-pinned` (kept hot by its header, so closing checkboxes frees nothing) · `fresh`. +- **Anchored candidates.** Items whose own text carries a Linear key or PR number, i.e. the ones an arbiter could actually settle. + +🛑 **Never quote the inherited-anchor number as resolvability.** An anchor in the *block header* is not the item's anchor: counting it inflates the resolvable share from 27.6% to 60.6% (measured), e.g. a tail about `docs/plans/…` counted as resolvable only because its header ended with "(PR 8)". The script reports `anchors` (the item's own) and `inherited` separately — only the former is resolvable. + +Report-only, permanently. Closing someone's open item is a decision about their unfinished work, and the vault is human-authored: surface the worklist, never tick a box. Checking whether an anchor actually closed (a merged PR, a Done issue) needs credentials this script deliberately does not have — that is a job for an agent acting with the user's go-ahead. + ### Step 8: Top Hubs Resolve the `moc` semantic role through `taxonomy_roles`, enumerate notes by that mapped type's configured prefix, then count backlinks for each: @@ -247,6 +266,10 @@ Other (no configured taxonomy tag): {N} (snooze a still-valid note: add `reviewed: {today}` to its frontmatter) 🔬 Content lint: {N judged} — {S} still-valid, {U} update-needed, {C} contradicts ← only when review.lint.enabled + +📮 Handoff: {N} KB / {N} blocks / {N} open items ← only when a handoff_note is configured and over handoff.maxKB + full resolution would free {N} KB ({N}%) — triage buys correctness, not size + {N} fully-resolvable · {N} partial (one anchorless item pins the block) · {N} no-anchor · {N} prose-pinned (counts MUST equal the lint's actual verdicts — never default to all-still-valid; see Step 7.5) - {fact_prefix}API X gotcha → UPDATE-NEEDED: superseded by [[{fact_prefix}API X v2]] - {source_prefix}vendor API pricing → still-valid (stamped reviewed: {today}) diff --git a/plugins/mnemo/skills/session/SKILL.md b/plugins/mnemo/skills/session/SKILL.md index d6a752c..d242b30 100644 --- a/plugins/mnemo/skills/session/SKILL.md +++ b/plugins/mnemo/skills/session/SKILL.md @@ -127,31 +127,27 @@ If the anchor is missing/non-unique or the file changes during publication, re-r ### Step 5: Update Session Handoff -**Read handoff through the shell-free CLI adapter:** +🛑 **Unfinished work belongs to its own session note, not to the handoff.** Write every open tail into this session's `## Next steps / pending` section (Step 4) — that is its home. The handoff gets a **pointer**, not a copy. -```bash -python3 "/scripts/safe-read.py" read <<'JSON' -{"file":"{handoff_note}","vault":"{vault}"} -JSON -``` +Why this is the contract and not a preference: when tails were written into the shared handoff instead, only **9%** of the live vault's fresh open items existed in their own session note (34% for older ones). The handoff became the sole home of forward state, which is exactly how it reached 805 KiB — and why it could then be neither read nor shrunk. A pointer line is bounded by the *number of sessions*; a copied tail is bounded by nothing. -**Update with `vault-write.py replace/insert` — targeted, not blind append:** - -- Remove completed pending items from previous sessions -- Add new pending items from current session (if any) -- Update context carry-over section +**Upsert this session's pointer line — the handoff index:** ```bash python3 "/scripts/vault-write.py" <<'JSON' -{"action":"replace","vault":"{vault}","note":"{handoff_note}","old_str":"{exact old section copied from read}","new_str":"{updated section as one JSON-escaped string}"} +{"action":"handoff-index-upsert","vault":"{vault}","note":"{handoff_note}","session_note":"{this session's note name}","date":"{YYYY-MM-DD}","project":"{project}","open_count":{count of open items in the session note}} JSON ``` -If the handoff note does not exist, create it via `vault-write.py create` with the same structure as `setup` Step 6. +It writes one line — `- 2026-07-25 · mnemo · open 3 · [[Session — …]]` — and is **idempotent on the session link**: a mid-task checkpoint refreshes its own line instead of appending a twin. Count `open_count` from the session note you just wrote; do not estimate it. Bounds (`max_lines` 180, `max_line_bytes` 200, `hard_cap_bytes` 38912, all overridable, all in **bytes**) are enforced by the writer, so the file cannot grow past its ceiling no matter how busy the week is. The session link is never truncated — only the project label gives way. + +If the handoff note does not exist, create it via `vault-write.py create` with the same structure as `setup` Step 6, then upsert. + +**Legacy block format.** A handoff still holding `## YYYY-MM-DD` blocks keeps them: the upsert adds its index line without touching them, and the size-guard below still rotates closed blocks. Migrating the blocks is a deliberate, separately-run operation — never a side effect of writing a session note. **Size-guard — keep the handoff thin (run every session):** -The handoff is a LIVE index, not an archive. After updating it, run the archival helper so closed history doesn't accumulate into a multi-MB token bomb (it no-ops when the note is at/under `handoff.maxKB`): +The handoff is a LIVE index, not an archive. After updating it, run the archival helper so closed history doesn't accumulate into a note nothing can read (it no-ops when the note is at/under `handoff.maxKB`): ```bash python3 "/scripts/vault-write.py" <<'JSON' @@ -195,7 +191,7 @@ Output summary: - **Include session_id in frontmatter** — disambiguates same-day sessions - **No session notes for trivial work** — but "trivial" = mechanical one-liners only (typo, single rename). A research / exploration / curiosity session counts as significant even with zero code; default to creating. - **Branch field optional** — research sessions don't have branches -- **Handoff = thin live index, not an archive** — targeted optimistic replacement, not blind append. Named ceiling: when it exceeds `handoff.maxKB` (default 40KB), `vault-write.py archive-handoff` (Step 5) rotates CLOSED blocks older than `handoff.keepDays` into `{handoff_note} Archive` (cold); open `- [ ]` + recent stay hot. Prevents multi-MB token-bomb accumulation without a second writer. +- **Handoff = thin live index, not an archive** — targeted optimistic replacement, not blind append. Named ceiling: when it exceeds `handoff.maxKB` (default 40KB), `vault-write.py archive-handoff` (Step 5) rotates CLOSED blocks older than `handoff.keepDays` into `{handoff_note} Archive` (cold); open `- [ ]` + recent stay hot. 🛑 **The guard is necessary but not sufficient, by its own invariant:** a block with even one open `- [ ]` is never archived, so on a real vault the archiver can be a no-op while the file is 20× over the ceiling (measured: 805 KiB, 863 open items, 0 blocks eligible). Size is bounded by the *format* of what gets written here; the guard only rotates what is already closed. `health` Step 7.6 reports what the guard cannot move and why. - **Links section is mandatory** — at least one MOC link, else the note orphans (invisible to graph navigation) - **Ghost notes generously** — wrap projects, technologies, people in `[[wikilinks]]` - **Thorough by routing, not volume** — the note stays a narrative; atom-worthy material (decisions, business-logic, pains, how-the-user-thinks) goes to `save`, links to `connect`, unfinished work to handoff. Full contract in `/references/depth-contract.md` diff --git a/scripts/migrate-handoff-to-index.py b/scripts/migrate-handoff-to-index.py new file mode 100644 index 0000000..d5faf70 --- /dev/null +++ b/scripts/migrate-handoff-to-index.py @@ -0,0 +1,220 @@ +#!/usr/bin/env python3 +"""One-shot migration: block-format handoff → pointer index + verbatim archive. + +Why a migration exists at all: the block format cannot be bounded. A block stays +hot while it holds a single open `- [ ]`, so on the maintainer's vault the +archiver was correctly freeing **zero** bytes while the file sat at 805 KiB — +20x its own ceiling, and past the point where anything can read it. + +What this does NOT do, deliberately: + +- **It never extracts checkboxes out of a block.** Measured on the live file, 90 + flat bullets under pending headings and ~200 prose lines carry live state with + no `- [ ]` at all; pulling out only checkboxes would silently drop them. Blocks + move **verbatim, whole**. +- **It never edits a session note.** Those are human-authored, and the vault has + no version control to undo a bad sweep. +- **It never deletes.** Everything moves to the archive, and a `.bak` of both + files is written before the first byte changes. + +Safety contract: `--dry-run` is the default. `--apply` requires the operator to +have a restore path in hand (`restore-handoff-from-bak.py`), writes `.bak` files +first, and verifies afterwards that every migrated block is byte-present in the +archive — refusing to leave a half-migration behind. + +Usage: + migrate-handoff-to-index.py [--archive PATH] [--apply] + [--keep-blocks N] [--today YYYY-MM-DD] +""" +from __future__ import annotations + +import argparse +import datetime as dt +import os +import re +import shutil +import sys + +BLOCK_SPLIT_RE = re.compile(r"(?m)(?=^## \d{4}-\d{2}-\d{2})") +BLOCK_DATE_RE = re.compile(r"^## (\d{4}-\d{2}-\d{2})") +SESSION_LINK_RE = re.compile(r"\[\[((?:Session|Сессия) — [^\]]+)\]\]") +OPEN_TODO_RE = re.compile(r"(?m)^\s*[-*]\s+\[ \]") +MAX_LINE_BYTES = 200 + + +def split_blocks(body: str) -> tuple[str, list[str]]: + match = re.search(r"(?m)^## \d{4}-\d{2}-\d{2}", body) + if match is None: + return body, [] + header = body[: match.start()] + blocks = [b for b in BLOCK_SPLIT_RE.split(body[match.start():]) if b.strip()] + return header, blocks + + +def clip_to_bytes(value: str, budget: int) -> str: + if budget <= 0: + return "" + trimmed = value + while trimmed and len(trimmed.encode()) > budget: + trimmed = trimmed[:-1] + return trimmed + + +def index_line(block: str) -> str: + """A pointer for one block: date · label · open N · [[Session — …]]. + + The session link is preferred as the pointer because it leads to the + narrative; when a block has none, the header text is kept as the label so + the line still says what it was about. + """ + date_match = BLOCK_DATE_RE.match(block) + date = date_match.group(1) if date_match else "0000-00-00" + header = block.split("\n", 1)[0] + label = re.sub(r"^## \d{4}-\d{2}-\d{2}\s*[—-]?\s*", "", header).strip() + open_count = len(OPEN_TODO_RE.findall(block)) + link_match = SESSION_LINK_RE.search(block) + if link_match: + suffix = f" · open {open_count} · [[{link_match.group(1)}]]" + else: + suffix = f" · open {open_count} · (без session-заметки)" + overhead = len(f"- {date}".encode()) + len(" · ".encode()) + len(suffix.encode()) + label = clip_to_bytes(label, MAX_LINE_BYTES - overhead) + return f"- {date} · {label}{suffix}" if label else f"- {date}{suffix}" + + +def archive_lead(today: str, count: int, byte_size: int) -> str: + return ( + f"\n> ❄️ UPDATE {today}: сюда перенесены {count} блоков ({byte_size} B) " + "из горячего handoff при переходе на индекс-формат. Перенос verbatim — " + "текст блоков не изменён, ничего не удалено. Живое состояние теперь " + "живёт в pending-секциях самих session-заметок, а горячий handoff держит " + "по одной строке-указателю на сессию.\n\n" + ) + + +def block_date(block: str) -> str: + match = BLOCK_DATE_RE.match(block) + return match.group(1) if match else "0000-00-00" + + +def plan(handoff_body: str, archive_body: str, *, today: str, keep_blocks: int): + header, blocks = split_blocks(handoff_body) + # Order by date, never by file position: a real handoff is not sorted (the + # live file had a 2026-07-19 block sitting among March ones), so "the newest + # N" must be computed, not assumed from the top of the file. + by_date = sorted(blocks, key=block_date, reverse=True) + keep = by_date[:keep_blocks] if keep_blocks else [] + kept = set(map(id, keep)) + move = [b for b in by_date if id(b) not in kept] + + lines = [index_line(b) for b in by_date] + new_handoff = header.rstrip("\n") + "\n\n" + "\n".join(lines) + "\n" + if keep: + new_handoff += "\n" + "".join( + b if b.endswith("\n") else b + "\n" for b in keep) + + moved_text = "".join(b if b.endswith("\n") else b + "\n" for b in move) + if archive_body.strip(): + arch_header, arch_blocks = split_blocks(archive_body) + new_archive = ( + arch_header.rstrip("\n") + + archive_lead(today, len(move), len(moved_text.encode())) + + moved_text + + "".join(b if b.endswith("\n") else b + "\n" for b in arch_blocks) + ) + else: + new_archive = ( + "---\ntype: meta\ntags: [meta, handoff, archive, cold]\n---\n\n" + "# Meta — Session Handoff Archive\n" + + archive_lead(today, len(move), len(moved_text.encode())) + + moved_text + ) + return new_handoff, new_archive, blocks, move + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + parser.add_argument("handoff") + parser.add_argument("--archive") + parser.add_argument("--apply", action="store_true", + help="actually write (default is a dry run)") + parser.add_argument("--keep-blocks", type=int, default=0, + help="keep the N newest blocks in full, verbatim") + parser.add_argument("--today") + args = parser.parse_args() + + handoff_path = os.path.expanduser(args.handoff) + if not os.path.isfile(handoff_path): + print(f"error: not a file: {handoff_path}", file=sys.stderr) + return 1 + archive_path = os.path.expanduser(args.archive) if args.archive else ( + handoff_path[:-3] + " Archive.md") + today = args.today or dt.date.today().isoformat() + + handoff_body = open(handoff_path, encoding="utf-8").read() + archive_body = (open(archive_path, encoding="utf-8").read() + if os.path.isfile(archive_path) else "") + + _, existing_blocks = split_blocks(handoff_body) + if not existing_blocks: + # Already migrated (or never block-format). Re-running must be a true + # no-op: a second pass used to append a "moved 0 blocks" note to the + # archive and stray blank lines to the handoff — harmless to the data, + # but noise a re-run should never produce. + print("handoff уже в индекс-формате — мигрировать нечего, файлы не тронуты") + return 0 + + new_handoff, new_archive, blocks, moved = plan( + handoff_body, archive_body, today=today, keep_blocks=args.keep_blocks) + + before, after = len(handoff_body.encode()), len(new_handoff.encode()) + print(f"handoff: {before} B → {after} B " + f"(−{round(100 * (before - after) / before, 1)}%)") + print(f"блоков: {len(blocks)} всего · {len(moved)} в архив · " + f"{len(blocks) - len(moved)} остаются целиком") + print(f"архив: {len(archive_body.encode())} B → {len(new_archive.encode())} B") + index_lines = [l for l in new_handoff.splitlines() if l.startswith("- 20")] + over = [l for l in index_lines if len(l.encode()) > MAX_LINE_BYTES] + print(f"индекс: {len(index_lines)} строк, " + f"max {max((len(l.encode()) for l in index_lines), default=0)} B/строка " + f"(бюджет {MAX_LINE_BYTES}; превышают {len(over)} — длинная ссылка не режется)") + print("\nпервые 5 строк индекса:") + for line in [l for l in new_handoff.splitlines() if l.startswith("- 20")][:5]: + print(f" {line}") + + if not args.apply: + print("\n🔍 DRY RUN — ничего не записано. Для применения: --apply") + return 0 + + stamp = today + for path in (handoff_path, archive_path): + if os.path.isfile(path): + backup = f"{path}.bak-migrate-{stamp}" + index = 1 + while os.path.exists(backup): + index += 1 + backup = f"{path}.bak-migrate-{stamp}-{index}" + shutil.copy2(path, backup) + print(f"бэкап: {backup}") + + # Archive first: if this run dies between the two writes, the blocks exist + # in the archive and the handoff is still intact — never the reverse. + with open(archive_path, "w", encoding="utf-8") as handle: + handle.write(new_archive) + with open(handoff_path, "w", encoding="utf-8") as handle: + handle.write(new_handoff) + + written_archive = open(archive_path, encoding="utf-8").read() + missing = [b.split("\n", 1)[0] for b in moved if b.rstrip("\n") not in written_archive] + if missing: + print(f"\n🛑 ПРОВЕРКА НЕ ПРОШЛА: {len(missing)} блоков не найдены в архиве " + f"дословно. Восстанови из .bak немедленно:", file=sys.stderr) + for header in missing[:5]: + print(f" {header}", file=sys.stderr) + return 2 + print(f"\n✅ применено; все {len(moved)} блоков дословно присутствуют в архиве") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/restore-handoff-from-bak.py b/scripts/restore-handoff-from-bak.py new file mode 100644 index 0000000..af1f332 --- /dev/null +++ b/scripts/restore-handoff-from-bak.py @@ -0,0 +1,101 @@ +#!/usr/bin/env python3 +"""Restore a handoff (and its archive) from the backups a migration wrote. + +This exists so the migration is reversible *before* it is ever run — the vault +is not under version control, Time Machine is not configured on this machine, +and Obsidian's File Recovery keeps only its own snapshot cadence. A migration +whose undo has not been rehearsed is not a migration, it is a gamble. + +Default is a dry run: it shows which backup would be restored where, and how +the sizes compare. `--apply` restores, and before overwriting it saves the +*current* file as `.pre-restore-` — so even an unwanted restore is itself +undoable. + +Usage: + restore-handoff-from-bak.py [--archive PATH] [--stamp STR] + [--list] [--apply] +""" +from __future__ import annotations + +import argparse +import datetime as dt +import glob +import os +import shutil +import sys + + +def backups_for(path: str) -> list[str]: + return sorted(glob.glob(f"{path}.bak*"), key=os.path.getmtime, reverse=True) + + +def pick(path: str, stamp: str | None) -> str | None: + candidates = backups_for(path) + if not candidates: + return None + if stamp: + matching = [c for c in candidates if stamp in c] + return matching[0] if matching else None + return candidates[0] + + +def describe(path: str, backup: str | None) -> None: + current = os.path.getsize(path) if os.path.isfile(path) else 0 + if backup is None: + print(f" {os.path.basename(path)}: бэкапов не найдено (сейчас {current} B)") + return + size = os.path.getsize(backup) + when = dt.datetime.fromtimestamp(os.path.getmtime(backup)).isoformat(timespec="seconds") + print(f" {os.path.basename(path)}: сейчас {current} B → восстановится {size} B") + print(f" из {os.path.basename(backup)} ({when})") + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + parser.add_argument("handoff") + parser.add_argument("--archive") + parser.add_argument("--stamp", help="substring of the backup name to pick") + parser.add_argument("--list", action="store_true", help="list backups and exit") + parser.add_argument("--apply", action="store_true") + args = parser.parse_args() + + handoff = os.path.expanduser(args.handoff) + archive = os.path.expanduser(args.archive) if args.archive else ( + handoff[:-3] + " Archive.md") + targets = [handoff] + ([archive] if os.path.isfile(archive) + or backups_for(archive) else []) + + if args.list: + for path in targets: + print(os.path.basename(path)) + for backup in backups_for(path): + when = dt.datetime.fromtimestamp(os.path.getmtime(backup)) + print(f" {os.path.getsize(backup):>9} B {when:%Y-%m-%d %H:%M} " + f"{os.path.basename(backup)}") + return 0 + + chosen = {path: pick(path, args.stamp) for path in targets} + print("План восстановления:") + for path in targets: + describe(path, chosen[path]) + + if not any(chosen.values()): + print("\n🛑 нечего восстанавливать", file=sys.stderr) + return 1 + if not args.apply: + print("\n🔍 DRY RUN — ничего не записано. Для применения: --apply") + return 0 + + stamp = dt.datetime.now().strftime("%Y%m%d-%H%M%S") + for path, backup in chosen.items(): + if backup is None: + continue + if os.path.isfile(path): + shutil.copy2(path, f"{path}.pre-restore-{stamp}") + shutil.copy2(backup, path) + print(f"✅ {os.path.basename(path)} ← {os.path.basename(backup)}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/split-handoff-archive.py b/scripts/split-handoff-archive.py new file mode 100644 index 0000000..ef1a759 --- /dev/null +++ b/scripts/split-handoff-archive.py @@ -0,0 +1,174 @@ +#!/usr/bin/env python3 +"""Split a monolithic handoff archive into per-month notes plus a small hub. + +A cold archive is not harmless just because nothing reads it at startup. The +live one is 717 KiB (1.5 MiB after migration), which makes it: + +- **unreadable** — a file read refuses past 256 KB / 25,000 tokens, so the + "you can always find it in the archive" promise silently fails, +- **noisy** — a vault-wide content scan matches it on nearly any query (it + appeared in 3 of 3 probes), so it crowds recall results with a hit nobody can + then open. + +Per-month notes fix both without deleting anything: each part stays whole and +verbatim, and the hub keeps one navigable line per month. Same safety contract +as the migration — `--dry-run` by default, `.bak` before writing, and a +verbatim-presence check afterwards. + +Usage: + split-handoff-archive.py [--apply] [--today YYYY-MM-DD] +""" +from __future__ import annotations + +import argparse +import collections +import datetime as dt +import os +import re +import shutil +import sys + +BLOCK_SPLIT_RE = re.compile(r"(?m)(?=^## \d{4}-\d{2}-\d{2})") +BLOCK_DATE_RE = re.compile(r"^## (\d{4}-\d{2}-\d{2})") +OPEN_TODO_RE = re.compile(r"(?m)^\s*[-*]\s+\[ \]") + + +def split_blocks(body: str) -> tuple[str, list[str]]: + match = re.search(r"(?m)^## \d{4}-\d{2}-\d{2}", body) + if match is None: + return body, [] + return body[: match.start()], [ + b for b in BLOCK_SPLIT_RE.split(body[match.start():]) if b.strip() + ] + + +def month_of(block: str) -> str: + match = BLOCK_DATE_RE.match(block) + return match.group(1)[:7] if match else "undated" + + +def part_body(hub_name: str, month: str, blocks: list[str]) -> str: + opens = sum(len(OPEN_TODO_RE.findall(b)) for b in blocks) + return ( + f"---\ntype: meta\ntags: [meta, handoff, archive, cold]\ndate: {month}-01\n---\n\n" + f"# {hub_name} {month}\n\n" + f"> ❄️ Холодный архив за {month}: {len(blocks)} блоков, {opens} незакрытых пунктов. " + f"Текст перенесён дословно. Навигация — [[{hub_name}]].\n\n" + + "".join(b if b.endswith("\n") else b + "\n" for b in blocks) + ) + + +def hub_body(hub_name: str, prefix: str, months: dict[str, list[str]], today: str) -> str: + lines = [ + f"> ❄️ UPDATE {today}: архив разбит помесячно — монолит перестал открываться " + "(чтение файла отказывает после 256 KB) и засорял поиск попаданием на любой запрос. " + "Блоки перенесены дословно, ничего не удалено.\n", + "", + "| Месяц | Блоков | Незакрытых | Размер |", + "|---|---|---|---|", + ] + for month in sorted(months, reverse=True): + blocks = months[month] + opens = sum(len(OPEN_TODO_RE.findall(b)) for b in blocks) + size = sum(len(b.encode()) for b in blocks) + lines.append( + f"| [[{hub_name} {month}]] | {len(blocks)} | {opens} | {round(size / 1024)} KB |") + lines += ["", "## Связи", "", f"- [[{hub_name.replace(' Archive', '')}]] — горячий индекс"] + return prefix.rstrip("\n") + "\n\n" + "\n".join(lines) + "\n" + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + parser.add_argument("archive") + parser.add_argument("--apply", action="store_true") + parser.add_argument("--today") + parser.add_argument("--max-part-bytes", type=int, default=200 * 1024, + help="split a month once it exceeds this (default 200 KB, " + "under the 256 KB read limit)") + args = parser.parse_args() + + path = os.path.expanduser(args.archive) + if not os.path.isfile(path): + print(f"error: not a file: {path}", file=sys.stderr) + return 1 + today = args.today or dt.date.today().isoformat() + directory, filename = os.path.split(path) + hub_name = filename[:-3] + + body = open(path, encoding="utf-8").read() + prefix, blocks = split_blocks(body) + if not blocks: + print("в архиве нет датированных блоков — делить нечего") + return 0 + + by_month: dict[str, list[str]] = collections.defaultdict(list) + for block in blocks: + by_month[month_of(block)].append(block) + + # A month is not automatically a readable unit: on the live archive June + # alone was 425 KB, still past the 256 KB read limit. Split any oversized + # month into numbered parts so every piece can actually be opened. + months: dict[str, list[str]] = {} + for month in sorted(by_month): + month_blocks = by_month[month] + if sum(len(b.encode()) for b in month_blocks) <= args.max_part_bytes: + months[month] = month_blocks + continue + part, used, index = [], 0, 1 + for block in month_blocks: + size = len(block.encode()) + if part and used + size > args.max_part_bytes: + months[f"{month} ч{index}"] = part + part, used, index = [], 0, index + 1 + part.append(block) + used += size + if part: + months[f"{month} ч{index}"] = part + + print(f"архив: {len(body.encode())} B · {len(blocks)} блоков · {len(months)} месяцев") + print(f"{'месяц':<10} {'блоков':>7} {'размер':>10}") + for month in sorted(months, reverse=True): + size = sum(len(b.encode()) for b in months[month]) + print(f"{month:<10} {len(months[month]):>7} {size:>9} B") + new_hub = hub_body(hub_name, prefix, months, today) + print(f"\nхаб станет {len(new_hub.encode())} B (было {len(body.encode())} B)") + biggest = max(len(part_body(hub_name, m, b).encode()) for m, b in months.items()) + print(f"крупнейшая часть: {biggest} B " + f"({'читаема' if biggest < 256 * 1024 else '🛑 всё ещё за пределом чтения 256 KB'})") + + if not args.apply: + print("\n🔍 DRY RUN — ничего не записано. Для применения: --apply") + return 0 + + backup = f"{path}.bak-split-{today}" + index = 1 + while os.path.exists(backup): + index += 1 + backup = f"{path}.bak-split-{today}-{index}" + shutil.copy2(path, backup) + print(f"бэкап: {backup}") + + # Parts first: a crash between writes must leave the monolith intact. + written = [] + for month, month_blocks in months.items(): + part_path = os.path.join(directory, f"{hub_name} {month}.md") + with open(part_path, "w", encoding="utf-8") as handle: + handle.write(part_body(hub_name, month, month_blocks)) + written.append((part_path, month_blocks)) + with open(path, "w", encoding="utf-8") as handle: + handle.write(new_hub) + + missing = 0 + for part_path, month_blocks in written: + content = open(part_path, encoding="utf-8").read() + missing += sum(1 for b in month_blocks if b.rstrip("\n") not in content) + if missing: + print(f"\n🛑 ПРОВЕРКА НЕ ПРОШЛА: {missing} блоков отсутствуют дословно. " + f"Восстанови: cp '{backup}' '{path}'", file=sys.stderr) + return 2 + print(f"\n✅ применено; {len(blocks)} блоков дословно разложены по {len(months)} частям") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/test-context-hook.py b/scripts/test-context-hook.py new file mode 100644 index 0000000..dea1221 --- /dev/null +++ b/scripts/test-context-hook.py @@ -0,0 +1,161 @@ +#!/usr/bin/env python3 +"""Tests for mnemo-context.sh — the SessionStart nudge plus open-tails digest. + +The digest half is new: it is the first automatic reader the handoff/forward +state ever had (measured 2026-07-25: continuity reads happened in 6% of +sessions, because 805 KiB cannot be read at all). These tests pin the +properties that make injecting it safe: + +- it never costs the nudge — any failure degrades to nudge-only, +- it is gated (`hooks.hotDigest`) and scoped (`hot.scope`), +- it stays under the byte cap it advertises. + +The Obsidian CLI is mocked on PATH, so the tests neither need Obsidian running +nor touch the real vault. + +Stdlib-only (unittest + subprocess), run directly: + + python3 scripts/test-context-hook.py +""" +import json +import os +import subprocess +import sys +import tempfile +import unittest + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +HOOK = os.path.join(REPO, "plugins", "mnemo", "hooks", "mnemo-context.sh") +PLUGIN_ROOT = os.path.join(REPO, "plugins", "mnemo") + +SESSION_NOTE = ( + "---\ntype: session\ntags: [session]\ndate: {date}\nproject: {project}\n---\n\n" + "# Заголовок\n\n## Next steps / pending\n\n- [ ] {tail}\n" +) + + +def write(path: str, text: str) -> None: + with open(path, "w", encoding="utf-8") as handle: + handle.write(text) + + +class ContextHookTest(unittest.TestCase): + def setUp(self) -> None: + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.home = self.tmp.name + os.makedirs(os.path.join(self.home, ".mnemo")) + + self.vault = os.path.join(self.home, "vault") + os.makedirs(self.vault) + write(os.path.join(self.vault, "s.md"), SESSION_NOTE.format( + date="2026-07-25", project="mnemo", tail="уникальный незакрытый хвост")) + write(os.path.join(self.vault, "other.md"), SESSION_NOTE.format( + date="2026-07-25", project="другой-проект", tail="чужой хвост")) + + # Mock the Obsidian CLI: `obsidian vault vault=` → tab-separated path. + self.bin = os.path.join(self.home, "bin") + os.makedirs(self.bin) + self.obsidian = os.path.join(self.bin, "obsidian") + self.write_cli(f'printf "name\\tmain\\npath\\t{self.vault}\\n"') + + def write_cli(self, body: str) -> None: + write(self.obsidian, f"#!/bin/sh\n{body}\n") + os.chmod(self.obsidian, 0o755) + + def config(self, **overrides) -> None: + payload = {"vault": "main"} + payload.update(overrides) + write(os.path.join(self.home, ".mnemo", "config.json"), json.dumps(payload)) + + def run_hook(self, cwd: str | None = None) -> str: + env = dict(os.environ) + env["HOME"] = self.home + env["PATH"] = self.bin + os.pathsep + env["PATH"] + env["CLAUDE_PLUGIN_ROOT"] = PLUGIN_ROOT + env.pop("PLUGIN_ROOT", None) + env.pop("CODEX_THREAD_ID", None) + env.pop("CODEX_SESSION_ID", None) + result = subprocess.run( + ["bash", HOOK], capture_output=True, text=True, env=env, + cwd=cwd or REPO, stdin=subprocess.DEVNULL) + self.assertEqual(0, result.returncode, result.stderr) + if not result.stdout.strip(): + return "" + payload = json.loads(result.stdout) + return payload.get("hookSpecificOutput", {}).get("additionalContext", "") + + # --- the digest ------------------------------------------------------ + + def test_digest_is_injected_next_to_the_nudge(self): + self.config() + context = self.run_hook() + self.assertIn("mnemo memory is set up here", context) + self.assertIn("уникальный незакрытый хвост", context) + + def test_digest_carries_the_stale_premise_warning(self): + self.config() + self.assertIn("протухн", self.run_hook()) + + def test_project_scope_is_the_default(self): + """Run from the mnemo repo → only the mnemo project's tails.""" + self.config() + context = self.run_hook() + self.assertIn("уникальный незакрытый хвост", context) + self.assertNotIn("чужой хвост", context) + + def test_scope_all_shows_every_project(self): + self.config(hot={"scope": "all"}) + context = self.run_hook() + self.assertIn("уникальный незакрытый хвост", context) + self.assertIn("чужой хвост", context) + + # --- gates ----------------------------------------------------------- + + def test_hot_digest_gate_suppresses_only_the_digest(self): + self.config(hooks={"hotDigest": False}) + context = self.run_hook() + self.assertIn("mnemo memory is set up here", context) + self.assertNotIn("уникальный незакрытый хвост", context) + + def test_session_start_gate_suppresses_everything(self): + self.config(hooks={"sessionStartNudge": False}) + self.assertEqual("", self.run_hook()) + + # --- degradation ------------------------------------------------------ + + def test_missing_obsidian_cli_keeps_the_nudge(self): + self.config() + os.remove(self.obsidian) + context = self.run_hook() + self.assertIn("mnemo memory is set up here", context) + self.assertNotIn("📌", context) + + def test_failing_obsidian_cli_keeps_the_nudge(self): + self.config() + self.write_cli("exit 3") + self.assertIn("mnemo memory is set up here", self.run_hook()) + + def test_garbage_from_obsidian_cli_keeps_the_nudge(self): + self.config() + self.write_cli('printf "surprise\\n"') + context = self.run_hook() + self.assertIn("mnemo memory is set up here", context) + self.assertNotIn("📌", context) + + def test_no_vault_key_stays_silent(self): + write(os.path.join(self.home, ".mnemo", "config.json"), json.dumps({})) + self.assertEqual("", self.run_hook()) + + def test_digest_respects_the_byte_cap(self): + self.config(hot={"scope": "all", "maxKB": 1}) + for i in range(20): + write(os.path.join(self.vault, f"n{i}.md"), SESSION_NOTE.format( + date="2026-07-25", project="mnemo", tail="я" * 150 + f" {i}")) + context = self.run_hook() + digest = context.split("📌", 1)[1] if "📌" in context else "" + self.assertLessEqual(len("📌".encode() + digest.encode()), 1024) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/scripts/test-handoff-index.py b/scripts/test-handoff-index.py new file mode 100644 index 0000000..e8dad12 --- /dev/null +++ b/scripts/test-handoff-index.py @@ -0,0 +1,217 @@ +#!/usr/bin/env python3 +"""Tests for vault-write.py's handoff-index-upsert action. + +The index format exists because the block format cannot be bounded. A block is +kept hot by a single open `- [ ]`, so on a real vault the archiver legitimately +freed **zero** bytes while the file sat 20x over its ceiling (measured +2026-07-25: 805 KiB, 863 open items). An index line is a pointer: its size is a +function of the number of sessions, not of how much was written in them. + +Pinned here: + +- **Idempotency keys on the session link.** A second `/mn:session` for the same + session must refresh its line, never append a twin — the old `read`+`old_str` + contract could not do this reliably, because a large handoff read returns a + preview and the section to copy is simply not in it. +- **Every bound is in BYTES.** A 150-character Cyrillic line is ~270 bytes; a + char-counted cap silently admits ~2x what it promises. One test writes exactly + such a line and fails if the accounting is done in characters. +- **The link is never truncated.** Trimming a line must sacrifice the project + label, never the `[[Session — …]]` pointer — a cut wikilink is a dead link, + which is worse than a long line. +- **The block path is untouched.** `archive-handoff` keeps its own regression + suite (`test-handoff-archive.py`, written after the v1.1.11 corruption + incident); this action must not disturb it. + +Stdlib-only (unittest + subprocess) — run directly: + + python3 scripts/test-handoff-index.py +""" +import json +import os +import re +import subprocess +import sys +import tempfile +import unittest + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +SCRIPT = os.path.join(REPO, 'plugins', 'mnemo', 'scripts', 'vault-write.py') + +HEADER = ( + '---\ntype: meta\ntags: [meta, handoff]\n---\n\n' + '# Meta — Session Handoff\n\n' + '> 🔥 HOT index. Detail lives in the linked session notes.\n\n' +) +INDEX_LINE_RE = re.compile(r'^- \d{4}-\d{2}-\d{2} · ') + + +class FakeVault: + """A vault dir plus a stub `obsidian` CLI that resolves it.""" + + def __init__(self, tmp: str) -> None: + self.root = os.path.join(tmp, 'vault') + os.makedirs(self.root) + self.bin = os.path.join(tmp, 'bin') + os.makedirs(self.bin) + cli = os.path.join(self.bin, 'obsidian') + with open(cli, 'w', encoding='utf-8') as handle: + handle.write(f'#!/bin/sh\nprintf "name\\tmain\\npath\\t{self.root}\\n"\n') + os.chmod(cli, 0o755) + + def env(self) -> dict: + env = dict(os.environ) + env['PATH'] = self.bin + os.pathsep + env['PATH'] + return env + + +class IndexUpsertTests(unittest.TestCase): + def setUp(self) -> None: + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.vault = FakeVault(self.tmp.name) + self.note = 'Meta — Session Handoff' + self.path = os.path.join(self.vault.root, f'{self.note}.md') + self.write_handoff(HEADER) + + def write_handoff(self, body: str) -> None: + with open(self.path, 'w', encoding='utf-8') as handle: + handle.write(body) + + def read_handoff(self) -> str: + with open(self.path, encoding='utf-8') as handle: + return handle.read() + + def index_lines(self) -> list[str]: + return [l for l in self.read_handoff().splitlines() if INDEX_LINE_RE.match(l)] + + def upsert(self, *, session_note: str, date: str, project: str = 'mnemo', + open_count: int = 0, expect_ok: bool = True, **extra) -> dict: + payload = { + 'action': 'handoff-index-upsert', + 'vault': 'main', + 'note': self.note, + 'session_note': session_note, + 'date': date, + 'project': project, + 'open_count': open_count, + } + payload.update(extra) + result = subprocess.run( + [sys.executable, SCRIPT], input=json.dumps(payload), + capture_output=True, text=True, env=self.vault.env()) + parsed = json.loads(result.stdout) + if expect_ok: + self.assertTrue(parsed.get('ok'), parsed) + return parsed + + # --- basic upsert ----------------------------------------------------- + + def test_first_upsert_creates_the_line(self): + self.upsert(session_note='Session — 2026-07-25 реформа', date='2026-07-25', + open_count=3) + lines = self.index_lines() + self.assertEqual(1, len(lines)) + self.assertIn('[[Session — 2026-07-25 реформа]]', lines[0]) + self.assertIn('open 3', lines[0]) + self.assertIn('mnemo', lines[0]) + + def test_header_is_preserved_verbatim(self): + self.upsert(session_note='Session — A', date='2026-07-25') + self.assertTrue(self.read_handoff().startswith(HEADER.rstrip('\n'))) + + def test_second_upsert_of_same_session_updates_in_place(self): + """The idempotency property the read+old_str contract could not hold.""" + self.upsert(session_note='Session — A', date='2026-07-25', open_count=1) + self.upsert(session_note='Session — A', date='2026-07-25', open_count=4) + lines = self.index_lines() + self.assertEqual(1, len(lines), lines) + self.assertIn('open 4', lines[0]) + self.assertNotIn('open 1', lines[0]) + + def test_distinct_sessions_get_distinct_lines(self): + self.upsert(session_note='Session — A', date='2026-07-25') + self.upsert(session_note='Session — B', date='2026-07-25') + self.assertEqual(2, len(self.index_lines())) + + def test_lines_are_sorted_newest_first(self): + for note, date in (('Session — old', '2026-07-01'), + ('Session — new', '2026-07-25'), + ('Session — mid', '2026-07-10')): + self.upsert(session_note=note, date=date) + dates = [l.split(' · ')[0][2:] for l in self.index_lines()] + self.assertEqual(sorted(dates, reverse=True), dates) + + # --- bounds ----------------------------------------------------------- + + def test_rotation_drops_the_oldest_line(self): + for day in range(1, 6): + self.upsert(session_note=f'Session — {day}', date=f'2026-07-0{day}') + self.upsert(session_note='Session — 6', date='2026-07-06', max_lines=3) + lines = self.index_lines() + self.assertEqual(3, len(lines)) + self.assertIn('Session — 6', lines[0]) + self.assertNotIn('Session — 1', '\n'.join(lines)) + + def test_line_budget_is_bytes_not_characters(self): + """150 Cyrillic chars ≈ 270 B: a char-counted budget lets it through.""" + self.upsert(session_note='Session — тест', date='2026-07-25', + project='я' * 150, max_line_bytes=200) + for line in self.index_lines(): + self.assertLessEqual(len(line.encode('utf-8')), 200, line) + + def test_trimming_never_cuts_the_session_link(self): + long_note = 'Session — ' + 'ю' * 90 + self.upsert(session_note=long_note, date='2026-07-25', + project='проект' * 20, max_line_bytes=200) + body = self.read_handoff() + self.assertIn(f'[[{long_note}]]', body) + + def test_hard_cap_bounds_the_whole_file(self): + for day in range(1, 10): + self.upsert(session_note=f'Session — {"ж" * 60} {day}', + date=f'2026-07-0{day}', project='проект-длинный', + hard_cap_bytes=len(HEADER.encode()) + 900) + self.assertLessEqual(len(self.read_handoff().encode('utf-8')), + len(HEADER.encode()) + 900) + + # --- validation -------------------------------------------------------- + + def test_rejects_session_note_with_wikilink_breaking_characters(self): + result = self.upsert(session_note='Session — bad]] injection', date='2026-07-25', + expect_ok=False) + self.assertFalse(result.get('ok')) + self.assertEqual('input_error', result['error']['code']) + + def test_rejects_malformed_date(self): + result = self.upsert(session_note='Session — A', date='25.07.2026', + expect_ok=False) + self.assertFalse(result.get('ok')) + + def test_rejects_negative_open_count(self): + result = self.upsert(session_note='Session — A', date='2026-07-25', + open_count=-1, expect_ok=False) + self.assertFalse(result.get('ok')) + + def test_missing_handoff_fails_closed_without_creating_it(self): + os.remove(self.path) + result = self.upsert(session_note='Session — A', date='2026-07-25', + expect_ok=False) + self.assertFalse(result.get('ok')) + self.assertFalse(os.path.exists(self.path)) + + # --- coexistence with the block path ----------------------------------- + + def test_block_format_content_is_left_alone(self): + """Upserting into a still-block-format handoff must not eat the blocks.""" + blocks = HEADER + '## 2026-05-01 — старый блок\n- [ ] незакрытое\n' + self.write_handoff(blocks) + self.upsert(session_note='Session — A', date='2026-07-25') + body = self.read_handoff() + self.assertIn('## 2026-05-01 — старый блок', body) + self.assertIn('- [ ] незакрытое', body) + self.assertEqual(1, len(self.index_lines())) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/scripts/test-handoff-resolver.py b/scripts/test-handoff-resolver.py new file mode 100644 index 0000000..da107e7 --- /dev/null +++ b/scripts/test-handoff-resolver.py @@ -0,0 +1,183 @@ +#!/usr/bin/env python3 +"""Tests for handoff-resolver.py — the report-only triage view of a handoff. + +Report-only forever: it never writes, never calls the network. What it produces +is a worklist plus honest arithmetic about what triage can and cannot buy. + +Two measured findings are pinned here as behaviour: + +1. **Anchor inheritance lies.** Counting an anchor found in the *block header* + as the checkbox's own anchor inflates "resolvable" from 27.6% to 60.6% — + e.g. a tail about `docs/plans/...` counted as resolvable merely because its + block header ended with "(PR 8)". Item anchors and inherited anchors are + reported as separate fields, and only item anchors count as resolvable. + +2. **One unresolvable checkbox pins a whole block.** The archiver keeps a block + HOT if *any* `- [ ]` remains, so a block that is 90% resolvable frees zero + bytes. Block payoff classes must reflect that, otherwise the report promises + space it cannot deliver. + +Stdlib-only (unittest + subprocess), run directly: + + python3 scripts/test-handoff-resolver.py +""" +import json +import os +import subprocess +import sys +import tempfile +import unittest + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +SCRIPT = os.path.join(REPO, 'plugins', 'mnemo', 'scripts', 'handoff-resolver.py') + +HEADER = '---\ntype: meta\n---\n\n🛡️ SIZE-GUARD line.\n\n' + + +def write(path: str, text: str) -> None: + with open(path, 'w', encoding='utf-8') as handle: + handle.write(text) + + +def run(path: str, *args: str) -> dict: + proc = subprocess.run( + [sys.executable, SCRIPT, path, '--json', '--today', '2026-07-25', *args], + capture_output=True, text=True) + if proc.returncode != 0: + raise AssertionError(f'exit {proc.returncode}: {proc.stderr}') + return json.loads(proc.stdout) + + +class ResolverTests(unittest.TestCase): + def setUp(self) -> None: + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.path = os.path.join(self.tmp.name, 'handoff.md') + + def blocks_by_date(self, payload: dict) -> dict: + return {b['date']: b for b in payload['blocks']} + + # --- parsing --------------------------------------------------------- + + def test_counts_blocks_and_open_items(self): + write(self.path, HEADER + + '## 2026-05-01 — старый блок\n- [ ] первый\n- [x] закрытый\n\n' + '## 2026-07-24 — свежий блок\n- [ ] второй\n') + payload = run(self.path) + self.assertEqual(2, payload['blocks_total']) + self.assertEqual(2, payload['open_total']) + + def test_sizes_are_bytes_not_characters(self): + """Cyrillic is ~2 B/char in UTF-8; a char count under-reports by a third.""" + body = HEADER + '## 2026-05-01 — блок\n- [ ] ' + 'я' * 500 + '\n' + write(self.path, body) + payload = run(self.path) + self.assertEqual(len(body.encode('utf-8')), payload['bytes']) + self.assertGreater(payload['bytes'], len(body)) + + def test_prose_live_block_is_flagged(self): + write(self.path, HEADER + '## 2026-05-01 — сервер (WAITING ответа)\nтекст без чекбоксов\n') + block = self.blocks_by_date(run(self.path))['2026-05-01'] + self.assertTrue(block['prose_live']) + self.assertEqual(0, block['open']) + + # --- anchors --------------------------------------------------------- + + def test_extracts_linear_and_pr_anchors_from_item_text(self): + write(self.path, HEADER + '## 2026-05-01 — блок\n' + '- [ ] добить BTS-250 и смержить PR #443\n') + item = run(self.path)['items'][0] + self.assertIn('BTS-250', item['anchors']['linear']) + self.assertIn('443', item['anchors']['pr']) + + def test_header_anchor_is_inherited_not_owned(self): + """The 60.6% artifact: an anchor in the header is not the item's anchor.""" + write(self.path, HEADER + '## 2026-07-25 — ACP: предполёт (PR 8)\n' + '- [ ] Фаза-0 из docs/plans/acp-buildout-state.md\n') + item = run(self.path)['items'][0] + self.assertEqual([], item['anchors']['pr']) + self.assertIn('8', item['inherited']['pr']) + self.assertFalse(item['resolvable']) + + def test_item_without_any_anchor_is_reported_as_such(self): + write(self.path, HEADER + '## 2026-05-01 — блок\n' + '- [ ] Уточнить у Леонида кто реальный заказчик\n') + payload = run(self.path) + self.assertFalse(payload['items'][0]['resolvable']) + self.assertEqual(1, payload['anchors']['none']) + + def test_resolvable_share_counts_only_item_anchors(self): + write(self.path, HEADER + '## 2026-05-01 — блок про BTS-1\n' + '- [ ] с якорем BTS-250\n- [ ] без якоря вообще\n') + payload = run(self.path) + self.assertEqual(1, payload['anchors']['resolvable']) + self.assertEqual(0.5, payload['anchors']['resolvable_share']) + + # --- payoff arithmetic ------------------------------------------------ + + def test_block_with_one_unresolvable_item_is_partial(self): + """One anchorless tail pins the whole block — payoff must say so.""" + write(self.path, HEADER + '## 2026-05-01 — блок\n' + '- [ ] BTS-250 закрыть\n- [ ] спросить Наталью\n') + block = self.blocks_by_date(run(self.path))['2026-05-01'] + self.assertEqual('partial', block['payoff']) + + def test_block_with_all_anchored_items_is_fully_resolvable(self): + write(self.path, HEADER + '## 2026-05-01 — блок\n' + '- [ ] BTS-250 закрыть\n- [ ] смержить PR #443\n') + block = self.blocks_by_date(run(self.path))['2026-05-01'] + self.assertEqual('fully-resolvable', block['payoff']) + + def test_fresh_blocks_are_not_counted_as_payoff(self): + """A block inside keepDays stays hot regardless — it is not a saving.""" + write(self.path, HEADER + '## 2026-07-24 — свежий\n- [ ] BTS-250\n') + payload = run(self.path, '--keep-days', '14') + self.assertEqual('fresh', self.blocks_by_date(payload)['2026-07-24']['payoff']) + self.assertEqual(0, payload['ceiling']['freed_bytes']) + + def test_ceiling_reports_what_full_resolution_would_free(self): + old_resolvable = '## 2026-05-01 — блок\n- [ ] BTS-250 закрыть\n' + old_pinned = '## 2026-05-02 — блок\n- [ ] спросить Наталью\n' + write(self.path, HEADER + old_resolvable + '\n' + old_pinned) + ceiling = run(self.path, '--keep-days', '14')['ceiling'] + self.assertEqual(len(old_resolvable.encode()) + 1, ceiling['freed_bytes']) + self.assertGreater(ceiling['floor_bytes'], 0) + + # --- safety ----------------------------------------------------------- + + def test_never_modifies_the_file(self): + body = HEADER + '## 2026-05-01 — блок\n- [ ] BTS-250\n' + write(self.path, body) + before = os.stat(self.path) + run(self.path) + after = os.stat(self.path) + self.assertEqual(body, open(self.path, encoding='utf-8').read()) + self.assertEqual(before.st_mtime_ns, after.st_mtime_ns) + + def test_missing_file_exits_nonzero_without_traceback(self): + proc = subprocess.run( + [sys.executable, SCRIPT, os.path.join(self.tmp.name, 'nope.md'), '--json'], + capture_output=True, text=True) + self.assertNotEqual(0, proc.returncode) + self.assertNotIn('Traceback', proc.stderr) + + def test_handoff_without_dated_blocks_is_not_a_crash(self): + write(self.path, HEADER + 'просто текст без блоков\n') + payload = run(self.path) + self.assertEqual(0, payload['blocks_total']) + self.assertEqual(0, payload['open_total']) + + def test_uses_the_shipped_hot_invariant(self): + """Regression guard: the resolver must not re-implement keep-hot. + + vault-write.py owns `keep = fresh OR open- [ ] OR prose-pending`. A copy + here would drift the day that rule changes, and the report would then + promise savings the archiver refuses to make. + """ + source = open(SCRIPT, encoding='utf-8').read() + self.assertIn('vault-write.py', source) + self.assertNotIn('re.compile(r"\\[ \\]")', source) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/scripts/test-hot-scan.py b/scripts/test-hot-scan.py new file mode 100644 index 0000000..17c081e --- /dev/null +++ b/scripts/test-hot-scan.py @@ -0,0 +1,226 @@ +#!/usr/bin/env python3 +"""Tests for hot-scan.py — the open-thread digest that feeds the SessionStart nudge. + +Pins the two failure classes measured on the live vault (2026-07-25): + +1. **v1.1.10 blind spot.** "Live" is not always a `- [ ]`. A pending section may + hold flat bullets (`- foo`) with no checkbox — 90 such bullets across 14 + handoff blocks, plus 200 prose lines. A scanner that only greps `- [ ]` + reproduces the exact bug the archiver was patched for in v1.1.10. + +2. **Byte-vs-char sizing.** Cyrillic in UTF-8 is ~1.35 B/char, so `len(str)` + under-counts a Russian vault by a third. Every cap here is in BYTES. + +Stdlib-only (unittest + subprocess) — run directly: + + python3 scripts/test-hot-scan.py +""" +import json +import os +import subprocess +import sys +import tempfile +import unittest + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +SCRIPT = os.path.join(REPO, 'plugins', 'mnemo', 'scripts', 'hot-scan.py') + + +def write(path: str, body: str) -> None: + with open(path, 'w', encoding='utf-8') as f: + f.write(body) + + +def session_note(date: str, project: str, body: str) -> str: + return ( + f'---\ntype: session\ntags: [session]\ndate: {date}\n' + f'project: {project}\n---\n\n# Заголовок\n\n{body}\n' + ) + + +def run(vault: str, *args: str) -> dict: + """Run hot-scan in --json mode and return the parsed payload.""" + proc = subprocess.run( + [sys.executable, SCRIPT, vault, '--json', '--today', '2026-07-25', *args], + capture_output=True, text=True, + ) + if proc.returncode != 0: + raise AssertionError(f'exit {proc.returncode}: {proc.stderr}') + return json.loads(proc.stdout) + + +class HotScanTests(unittest.TestCase): + def setUp(self) -> None: + self.tmp = tempfile.TemporaryDirectory() + self.vault = self.tmp.name + self.addCleanup(self.tmp.cleanup) + + def texts(self, payload: dict) -> list[str]: + return [item['text'] for group in payload['groups'] for item in group['items']] + + # --- extraction ------------------------------------------------------ + + def test_extracts_open_checkbox_from_pending_section(self): + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'mnemo', + '## Next steps / pending\n\n- [ ] проверить ре-пин интерактивно\n')) + self.assertIn('проверить ре-пин интерактивно', self.texts(run(self.vault))) + + def test_extracts_flat_bullet_without_checkbox(self): + """v1.1.10 class: live work is not always a checkbox.""" + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'bts', + '### ⏳ Pending\n\n- Вернуть TARGET на боевой 62214\n')) + self.assertIn('Вернуть TARGET на боевой 62214', self.texts(run(self.vault))) + + def test_recognises_russian_pending_headings(self): + for heading in ('## Осталось', '## Ждём', '## Хвосты', '## Что осталось'): + with self.subTest(heading=heading): + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'p', f'{heading}\n\n- уникальный хвост\n')) + self.assertIn('уникальный хвост', self.texts(run(self.vault))) + + def test_ignores_closed_checkbox(self): + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'mnemo', + '## Next steps / pending\n\n- [x] уже сделано\n- [ ] ещё нет\n')) + texts = self.texts(run(self.vault)) + self.assertIn('ещё нет', texts) + self.assertNotIn('уже сделано', texts) + + def test_ignores_bullets_outside_pending_sections(self): + """A flat bullet under Findings is narrative, not a tail.""" + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'mnemo', + '## Findings\n\n- это вывод, а не хвост\n\n' + '## Next steps / pending\n\n- [ ] это хвост\n')) + texts = self.texts(run(self.vault)) + self.assertIn('это хвост', texts) + self.assertNotIn('это вывод, а не хвост', texts) + + def test_open_checkbox_outside_pending_still_counts(self): + """An explicit `- [ ]` is unambiguous wherever it sits.""" + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'mnemo', '## Findings\n\n- [ ] явный незакрытый пункт\n')) + self.assertIn('явный незакрытый пункт', self.texts(run(self.vault))) + + def test_links_section_is_not_a_pending_section(self): + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'mnemo', '## Связи\n\n- [[MOC — mnemo]]\n')) + self.assertEqual([], self.texts(run(self.vault))) + + # --- grouping & window ---------------------------------------------- + + def test_groups_by_project_frontmatter(self): + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'mnemo', '## Pending\n\n- хвост mnemo\n')) + write(os.path.join(self.vault, 'b.md'), session_note( + '2026-07-24', 'bts-holding', '## Pending\n\n- хвост bts\n')) + groups = {g['project']: [i['text'] for i in g['items']] for g in run(self.vault)['groups']} + self.assertEqual(['хвост mnemo'], groups['mnemo']) + self.assertEqual(['хвост bts'], groups['bts-holding']) + + def test_window_days_excludes_old_notes(self): + write(os.path.join(self.vault, 'old.md'), session_note( + '2026-06-01', 'mnemo', '## Pending\n\n- старый хвост\n')) + write(os.path.join(self.vault, 'new.md'), session_note( + '2026-07-24', 'mnemo', '## Pending\n\n- свежий хвост\n')) + texts = self.texts(run(self.vault, '--window-days', '7')) + self.assertIn('свежий хвост', texts) + self.assertNotIn('старый хвост', texts) + + def test_project_filter_keeps_only_requested_project(self): + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'mnemo', '## Pending\n\n- хвост mnemo\n')) + write(os.path.join(self.vault, 'b.md'), session_note( + '2026-07-24', 'bts', '## Pending\n\n- хвост bts\n')) + payload = run(self.vault, '--project', 'mnemo') + self.assertEqual(['mnemo'], [g['project'] for g in payload['groups']]) + + def test_non_session_notes_are_ignored(self): + write(os.path.join(self.vault, 'atom.md'), + '---\ntype: atom\ndate: 2026-07-24\n---\n\n## Pending\n\n- не сессия\n') + self.assertEqual([], self.texts(run(self.vault))) + + # --- byte cap (the len() trap) --------------------------------------- + + def test_cap_is_measured_in_bytes_not_characters(self): + """150 Cyrillic chars ≈ 270 bytes. A char-counting cap keeps too much.""" + item = 'я' * 150 + for i in range(6): + write(os.path.join(self.vault, f'n{i}.md'), session_note( + '2026-07-24', 'p', f'## Pending\n\n- {item} {i}\n')) + payload = run(self.vault, '--max-kb', '1') + self.assertLessEqual(len(payload['digest'].encode('utf-8')), 1024) + + def test_cap_drops_oldest_first(self): + """Three equal-cost items, room for two — the oldest one goes.""" + filler = 'с' * 160 + for date, marker in (('2026-07-19', 'СТАРОЕ'), + ('2026-07-22', 'СРЕДНЕЕ'), + ('2026-07-25', 'НОВОЕ')): + write(os.path.join(self.vault, f'{marker}.md'), session_note( + date, 'p', f'## Pending\n\n- {filler} {marker}\n')) + payload = run(self.vault, '--max-kb', '1') + digest = payload['digest'] + self.assertIn('НОВОЕ', digest) + self.assertIn('СРЕДНЕЕ', digest) + self.assertNotIn('СТАРОЕ', digest) + self.assertLessEqual(len(digest.encode('utf-8')), 1024) + self.assertEqual(3, payload['total']) + self.assertEqual(2, payload['shown']) + + # --- digest shape ----------------------------------------------------- + + def test_digest_carries_stale_premise_warning(self): + """The 21.07 dead-premise incident is designed into the output.""" + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'mnemo', '## Pending\n\n- хвост\n')) + self.assertIn('протухн', run(self.vault)['digest']) + + def test_empty_vault_yields_empty_digest_and_exit_zero(self): + payload = run(self.vault) + self.assertEqual([], payload['groups']) + self.assertEqual('', payload['digest']) + + def test_missing_vault_exits_nonzero_without_traceback(self): + proc = subprocess.run( + [sys.executable, SCRIPT, os.path.join(self.vault, 'nope'), '--json'], + capture_output=True, text=True) + self.assertNotEqual(0, proc.returncode) + self.assertNotIn('Traceback', proc.stderr) + + def test_cap_holds_across_many_groups(self): + """Regression: per-item accounting ignored per-group header lines and + overshot the cap (measured 8335 B against 8192 B on the live vault).""" + for i in range(12): + write(os.path.join(self.vault, f'p{i}.md'), session_note( + '2026-07-24', f'проект-номер-{i}', + '## Pending\n\n- ' + 'ю' * 120 + f' хвост {i}\n')) + payload = run(self.vault, '--max-kb', '2') + self.assertLessEqual(len(payload['digest'].encode('utf-8')), 2048) + self.assertGreater(payload['shown'], 0) + + def test_shared_prefix_items_are_not_deduped_away(self): + """Regression: a prefix-truncated dedupe key merged distinct tails.""" + prefix = 'проверить что всё сходится в ' + 'д' * 130 + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'p', f'## Pending\n\n- {prefix} ПЕРВЫЙ\n- {prefix} ВТОРОЙ\n')) + texts = self.texts(run(self.vault)) + self.assertEqual(2, len(texts), texts) + + def test_identical_tail_from_two_sources_is_deduped(self): + for name in ('a.md', 'b.md'): + write(os.path.join(self.vault, name), session_note( + '2026-07-24', 'p', '## Pending\n\n- ровно один и тот же хвост\n')) + self.assertEqual(1, len(self.texts(run(self.vault)))) + + def test_counts_are_reported_per_group(self): + write(os.path.join(self.vault, 'a.md'), session_note( + '2026-07-24', 'mnemo', '## Pending\n\n- один\n- два\n')) + group = run(self.vault)['groups'][0] + self.assertEqual(2, group['count']) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/scripts/test-migrate-handoff.py b/scripts/test-migrate-handoff.py new file mode 100644 index 0000000..d89c766 --- /dev/null +++ b/scripts/test-migrate-handoff.py @@ -0,0 +1,198 @@ +#!/usr/bin/env python3 +"""Tests for the one-shot handoff migration and its restore path. + +An irreversible sweep over a vault with no version control earns more scrutiny +than ordinary code, so these tests pin the safety contract itself, not just the +happy path: + +- dry run is the default and writes nothing, +- blocks move **verbatim and whole** — never checkbox-extracted, because live + state also lives in flat bullets and prose (90 + ~200 occurrences measured), +- nothing is deleted: every migrated block is byte-present in the archive, +- a `.bak` exists before the first write, and restore actually brings the file + back byte-for-byte. + +Run directly: + + python3 scripts/test-migrate-handoff.py +""" +import os +import subprocess +import sys +import tempfile +import unittest + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +MIGRATE = os.path.join(REPO, 'scripts', 'migrate-handoff-to-index.py') +RESTORE = os.path.join(REPO, 'scripts', 'restore-handoff-from-bak.py') + +HEADER = '---\ntype: meta\n---\n\n🛡️ SIZE-GUARD line.\n\n' +BLOCK_A = ( + '## 2026-05-01 — старый блок про деплой\n' + 'Проза, которая держит состояние без чекбокса.\n' + '- [ ] незакрытый пункт\n' + '- плоский буллет без чекбокса\n' + 'Нарратив: [[Session — 2026-05-01 деплой]]\n\n' +) +BLOCK_B = ( + '## 2026-07-24 — свежий блок\n' + '- [x] сделано\n' + '- [ ] ещё нет\n\n' +) + + +def write(path: str, text: str) -> None: + with open(path, 'w', encoding='utf-8') as handle: + handle.write(text) + + +def read(path: str) -> str: + with open(path, encoding='utf-8') as handle: + return handle.read() + + +def run(script: str, *args: str) -> subprocess.CompletedProcess: + return subprocess.run([sys.executable, script, *args], + capture_output=True, text=True) + + +class MigrationTests(unittest.TestCase): + def setUp(self) -> None: + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.handoff = os.path.join(self.tmp.name, 'Handoff.md') + self.archive = os.path.join(self.tmp.name, 'Handoff Archive.md') + write(self.handoff, HEADER + BLOCK_A + BLOCK_B) + + def migrate(self, *args: str) -> subprocess.CompletedProcess: + result = run(MIGRATE, self.handoff, '--archive', self.archive, + '--today', '2026-07-25', *args) + self.assertEqual(0, result.returncode, result.stderr) + return result + + # --- safety ------------------------------------------------------------ + + def test_dry_run_is_the_default_and_writes_nothing(self): + before = read(self.handoff) + result = self.migrate() + self.assertIn('DRY RUN', result.stdout) + self.assertEqual(before, read(self.handoff)) + self.assertFalse(os.path.exists(self.archive)) + + def test_apply_writes_a_backup_first(self): + original = read(self.handoff) + self.migrate('--apply') + backups = [f for f in os.listdir(self.tmp.name) if '.bak-migrate-' in f] + self.assertTrue(backups, os.listdir(self.tmp.name)) + self.assertEqual(original, read(os.path.join(self.tmp.name, backups[0]))) + + def test_blocks_move_verbatim_and_whole(self): + """Not checkbox-extracted: prose and flat bullets must survive.""" + self.migrate('--apply') + archive = read(self.archive) + self.assertIn('Проза, которая держит состояние без чекбокса.', archive) + self.assertIn('- плоский буллет без чекбокса', archive) + self.assertIn(BLOCK_A.rstrip('\n'), archive) + + def test_nothing_is_lost_between_the_two_files(self): + self.migrate('--apply') + combined = read(self.handoff) + read(self.archive) + for line in (BLOCK_A + BLOCK_B).splitlines(): + if line.strip(): + self.assertIn(line, combined, line) + + def test_verification_reports_success(self): + result = self.migrate('--apply') + self.assertIn('дословно присутствуют в архиве', result.stdout) + + def test_second_run_is_a_true_no_op(self): + """A re-run used to append a "moved 0 blocks" note and blank lines.""" + self.migrate('--apply') + handoff_after, archive_after = read(self.handoff), read(self.archive) + result = self.migrate('--apply') + self.assertIn('мигрировать нечего', result.stdout) + self.assertEqual(handoff_after, read(self.handoff)) + self.assertEqual(archive_after, read(self.archive)) + self.assertNotIn('0 блоков', read(self.archive)) + + def test_second_run_writes_no_extra_backup(self): + self.migrate('--apply') + before = len([f for f in os.listdir(self.tmp.name) if '.bak-migrate-' in f]) + self.migrate('--apply') + after = len([f for f in os.listdir(self.tmp.name) if '.bak-migrate-' in f]) + self.assertEqual(before, after) + + # --- the index --------------------------------------------------------- + + def test_handoff_becomes_an_index(self): + self.migrate('--apply') + body = read(self.handoff) + self.assertTrue(body.startswith(HEADER.rstrip('\n'))) + lines = [l for l in body.splitlines() if l.startswith('- 20')] + self.assertEqual(2, len(lines)) + self.assertIn('[[Session — 2026-05-01 деплой]]', lines[1]) + self.assertIn('open 1', lines[1]) + + def test_index_lines_stay_within_the_byte_budget(self): + write(self.handoff, HEADER + '## 2026-05-01 — ' + 'щ' * 300 + '\n- [ ] x\n') + self.migrate('--apply') + for line in read(self.handoff).splitlines(): + if line.startswith('- 20'): + self.assertLessEqual(len(line.encode('utf-8')), 200) + + def test_block_without_session_link_is_still_indexed(self): + write(self.handoff, HEADER + BLOCK_B) + self.migrate('--apply') + line = [l for l in read(self.handoff).splitlines() if l.startswith('- 20')][0] + self.assertIn('без session-заметки', line) + + def test_keep_blocks_leaves_the_newest_in_full(self): + """"Newest" is by date, not by position — a real handoff is unsorted.""" + self.migrate('--apply', '--keep-blocks', '1') + body = read(self.handoff) + self.assertIn('## 2026-07-24 — свежий блок', body) + self.assertNotIn('## 2026-05-01', body) + self.assertIn('## 2026-05-01 — старый блок про деплой', read(self.archive)) + + def test_existing_archive_content_is_preserved(self): + write(self.archive, '---\ntype: meta\n---\n\n## 2026-01-01 — древний\nстарое\n') + self.migrate('--apply') + archive = read(self.archive) + self.assertIn('## 2026-01-01 — древний', archive) + self.assertIn('UPDATE 2026-07-25', archive) + + # --- restore ------------------------------------------------------------ + + def test_restore_brings_the_file_back_byte_for_byte(self): + original = read(self.handoff) + self.migrate('--apply') + self.assertNotEqual(original, read(self.handoff)) + result = run(RESTORE, self.handoff, '--archive', self.archive, '--apply') + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual(original, read(self.handoff)) + + def test_restore_dry_run_writes_nothing(self): + self.migrate('--apply') + migrated = read(self.handoff) + result = run(RESTORE, self.handoff, '--archive', self.archive) + self.assertIn('DRY RUN', result.stdout) + self.assertEqual(migrated, read(self.handoff)) + + def test_restore_keeps_the_pre_restore_copy(self): + self.migrate('--apply') + migrated = read(self.handoff) + run(RESTORE, self.handoff, '--archive', self.archive, '--apply') + saved = [f for f in os.listdir(self.tmp.name) if '.pre-restore-' in f] + self.assertTrue(saved) + self.assertEqual(migrated, read(os.path.join(self.tmp.name, saved[0]))) + + def test_restore_without_backups_fails_closed(self): + lonely = os.path.join(self.tmp.name, 'Nothing.md') + write(lonely, 'x') + result = run(RESTORE, lonely, '--apply') + self.assertNotEqual(0, result.returncode) + self.assertEqual('x', read(lonely)) + + +if __name__ == '__main__': + unittest.main(verbosity=2) diff --git a/scripts/test-split-archive.py b/scripts/test-split-archive.py new file mode 100644 index 0000000..87c729d --- /dev/null +++ b/scripts/test-split-archive.py @@ -0,0 +1,120 @@ +#!/usr/bin/env python3 +"""Tests for split-handoff-archive.py. + +The archive is cold, not harmless: at 717 KiB it is past the 256 KB read limit, +so "it's all still in the archive" is a promise nothing can keep. Splitting must +therefore produce parts that are actually openable — and must not lose a byte +doing it. + +Run directly: + + python3 scripts/test-split-archive.py +""" +import os +import subprocess +import sys +import tempfile +import unittest + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +SCRIPT = os.path.join(REPO, 'scripts', 'split-handoff-archive.py') + +PREFIX = '---\ntype: meta\n---\n\n# Meta — Session Handoff Archive\n\n' + + +def block(date: str, filler: int = 0) -> str: + body = ('текст ' * filler) + '\n' if filler else '' + return f'## {date} — блок {date}\n- [ ] хвост {date}\n{body}\n' + + +def write(path: str, text: str) -> None: + with open(path, 'w', encoding='utf-8') as handle: + handle.write(text) + + +def read(path: str) -> str: + with open(path, encoding='utf-8') as handle: + return handle.read() + + +class SplitTests(unittest.TestCase): + def setUp(self) -> None: + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.archive = os.path.join(self.tmp.name, 'Meta — Session Handoff Archive.md') + self.blocks = [block('2026-03-01'), block('2026-04-02'), block('2026-04-20')] + write(self.archive, PREFIX + ''.join(self.blocks)) + + def run_split(self, *args: str) -> subprocess.CompletedProcess: + result = subprocess.run( + [sys.executable, SCRIPT, self.archive, '--today', '2026-07-25', *args], + capture_output=True, text=True) + self.assertEqual(0, result.returncode, result.stderr) + return result + + def parts(self) -> list[str]: + return sorted(f for f in os.listdir(self.tmp.name) + if f.startswith('Meta — Session Handoff Archive 2026')) + + def test_dry_run_writes_nothing(self): + before = read(self.archive) + result = self.run_split() + self.assertIn('DRY RUN', result.stdout) + self.assertEqual(before, read(self.archive)) + self.assertEqual([], self.parts()) + + def test_creates_one_part_per_month(self): + self.run_split('--apply') + self.assertEqual(2, len(self.parts()), self.parts()) + + def test_blocks_land_verbatim_in_their_month(self): + self.run_split('--apply') + april = read(os.path.join(self.tmp.name, 'Meta — Session Handoff Archive 2026-04.md')) + self.assertIn(self.blocks[1].rstrip('\n'), april) + self.assertIn(self.blocks[2].rstrip('\n'), april) + self.assertNotIn('2026-03-01', april) + + def test_hub_shrinks_and_links_every_part(self): + self.run_split('--apply') + hub = read(self.archive) + self.assertLess(len(hub.encode()), 2048) + for part in self.parts(): + self.assertIn(f'[[{part[:-3]}]]', hub) + + def test_backup_is_written_before_applying(self): + original = read(self.archive) + self.run_split('--apply') + backups = [f for f in os.listdir(self.tmp.name) if '.bak-split-' in f] + self.assertTrue(backups) + self.assertEqual(original, read(os.path.join(self.tmp.name, backups[0]))) + + def test_oversized_month_is_split_into_readable_parts(self): + """A month is not automatically a readable unit — June was 425 KB.""" + big = [block(f'2026-06-{day:02d}', filler=400) for day in range(1, 13)] + write(self.archive, PREFIX + ''.join(big)) + self.run_split('--apply', '--max-part-bytes', '20000') + parts = self.parts() + self.assertGreater(len(parts), 1, parts) + for part in parts: + size = os.path.getsize(os.path.join(self.tmp.name, part)) + self.assertLessEqual(size, 256 * 1024) + + def test_nothing_is_lost_across_the_split(self): + self.run_split('--apply') + combined = read(self.archive) + ''.join( + read(os.path.join(self.tmp.name, p)) for p in self.parts()) + for chunk in self.blocks: + for line in chunk.splitlines(): + if line.strip(): + self.assertIn(line, combined, line) + + def test_archive_without_blocks_is_a_no_op(self): + write(self.archive, PREFIX) + before = read(self.archive) + result = self.run_split('--apply') + self.assertIn('делить нечего', result.stdout) + self.assertEqual(before, read(self.archive)) + + +if __name__ == '__main__': + unittest.main(verbosity=2)