From 2e109eb72b2a850f7eec36b465a001806f7f0955 Mon Sep 17 00:00:00 2001 From: BY571 Date: Wed, 19 Aug 2026 15:30:24 +0200 Subject: [PATCH 1/2] a graph starts from a question, and a hunch is a source MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The documented shape began at `source`, which left two things unsaid and one thing wrong. WHAT A GRAPH IS FOR HAD NOWHERE TO LIVE. `graph.yaml: description` held the research question as prose, so nothing could cite it: a finding could not be traced back to the question it serves, and a graph with a second sub-question had nowhere to put it. `knoten init` now scaffolds `question-.md` beside the gate example, and `question` joins the declared vocabulary. Everything else descends from it. `question` WAS IN THE WRONG PLACE. `viz.FLOW` listed it third — after `source` and `idea`, behind the two things that derive from it — so the column order contradicted the research order it exists to show. It goes first now, in both `viz.py` and the template's own legend. A SOURCE WAS DEFINED TOO NARROWLY. "External material the work starts from" excludes the case a lot of research actually begins in: somebody's own intuition. It is now "where the work came from — a paper, blogpost, dataset, search, or your own intuition", with the convention of writing `source-own-intuition` and citing it like any paper. That keeps one rule rather than two — an idea always names where it came from — and it makes an uncomfortable question answerable: how much of this graph rests on hunches rather than on having read anything? Verified end to end: init writes the question, the fresh graph validates, `question` is the leftmost column, and a hunch recorded as a source passes. --- README.md | 15 ++++++++++--- SKILL.md | 11 +++++++--- SPEC.md | 8 ++++--- src/knoten/cli.py | 27 +++++++++++++++++++++-- src/knoten/mcp_server.py | 9 +++++--- src/knoten/viz.html | 2 +- src/knoten/viz.py | 2 +- tests/test_cli.py | 46 ++++++++++++++++++++++++++++++++++++++++ 8 files changed, 104 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index 3b2e4a1..2aa1733 100644 --- a/README.md +++ b/README.md @@ -188,13 +188,22 @@ your `graph.yaml` says they mean — the core checks `node_types` for membership nothing else. A shape that recurs: ``` -source ──▶ idea ──▶ hypothesis ──▶ experiment ──▶ finding - ▲ │ - └────── findings open new ideas ────────┘ +question ─▶ source ─▶ idea ─▶ hypothesis ─▶ experiment ─▶ finding + ▲ │ + └────── findings open new ideas ──────┘ gate stands outside the loop: the bar every claim must survive ``` +A graph begins with **one question, statement or task**, and `knoten init` scaffolds it as +a node rather than leaving it as prose — so a finding can be traced back to the question it +serves, and a second sub-question has somewhere to live. From there the usual move is to +investigate **sources**: papers, blogposts, datasets, searches. Work that starts from your +own head instead is not an exception — write `source-own-intuition` and cite it the same +way. One rule, every time: an idea names where it came from. It also makes an +uncomfortable question answerable — how much of this graph rests on hunches rather than on +having read anything? + The names above are a convention, not a rule — `knoten viz` lays columns out in that order and puts anything it does not recognise after the ones it does. (`method` is deliberately unused: the rename in #15 freed the word for "the approach derived from diff --git a/SKILL.md b/SKILL.md index 75076f5..7a85246 100644 --- a/SKILL.md +++ b/SKILL.md @@ -15,12 +15,17 @@ A research graph that remembers what did NOT work. Run `knoten` in the graph dir **Read it first** — knoten defines none of these words, so `hypothesis` means whatever that graph says it means. A common shape: - source ──▶ idea ──▶ hypothesis ──▶ experiment ──▶ finding - ▲ │ - └────── findings open new ideas ────────┘ + question ─▶ source ─▶ idea ─▶ hypothesis ─▶ experiment ─▶ finding + ▲ │ + └────── findings open new ideas ──────┘ gate stands outside the loop: the bar every claim must survive +A graph starts from ONE question, statement or task — `knoten init` scaffolds it, and +everything else descends from it. Investigate sources first (papers, posts, datasets, +searches); if the work starts from your own head instead, record that as a source too, so +an idea always names where it came from. + Those names are the order `knoten viz` lays columns out in; a type it does not know lands after the ones it does. Links are a list, so one hypothesis can carry several experiments and several findings. diff --git a/SPEC.md b/SPEC.md index 0e894e5..8b3a3e8 100644 --- a/SPEC.md +++ b/SPEC.md @@ -77,10 +77,12 @@ rather than thinking.** ### Node types (conventions, not hardcoded) -`source` · `idea` · `hypothesis` · `experiment` · `finding` · `retraction` · `gate` +`question` · `source` · `idea` · `hypothesis` · `experiment` · `finding` · `retraction` · `gate` -A convention only — the core checks `node_types` for membership and nothing else. `gate` -stands outside the loop the others form: it is the bar a claim must survive, not a stage +A convention only — the core checks `node_types` for membership and nothing else. A graph +begins with a `question` and everything descends from it; a `source` is where the work came +from, including the author's own intuition. `gate` stands outside the loop the others +form: it is the bar a claim must survive, not a stage it passes through. `method` is deliberately unclaimed; the rename that freed it reserved it for "the approach derived from findings that survived", and nothing implements that yet. diff --git a/src/knoten/cli.py b/src/knoten/cli.py index ce5659f..d6612dd 100644 --- a/src/knoten/cli.py +++ b/src/knoten/cli.py @@ -350,7 +350,9 @@ def detach(root, nid, name) -> int: # The meanings are not decoration: knoten defines none of these words, so this is the only # place they ARE defined, and `knoten viz` shows them beside each column. node_types: - source: external material the work starts from — a paper, dataset or search + question: what this graph exists to answer — a question, a statement or a task + source: where the work came from — a paper, blogpost, dataset, search, or your + own intuition (write that one down too; a hunch you cite is auditable) idea: what you took from a source; a direction, not yet a testable claim hypothesis: a falsifiable claim derived from an idea experiment: the test built to verify or falsify a hypothesis @@ -383,6 +385,25 @@ def detach(root, nid, name) -> int: message: The post-mortem IS the asset — a dead end must become a standing offer. """ +TEMPLATE_QUESTION = """\ +--- +id: question-{name} +type: question +status: open +--- +# TODO — the question, statement or task this graph exists to answer + +## Why it matters + + +## What would count as an answer + + +Everything else in this graph descends from this node. Investigate sources first +(papers, posts, datasets, searches) — or start from your own intuition and record +that as a source too, so the origin is citable either way. +""" + TEMPLATE_GATE = """\ --- id: gate-example @@ -460,10 +481,12 @@ def init(name) -> int: (root / "graph.yaml").write_text(TEMPLATE_GRAPH.format(name=name), encoding="utf-8") # knoten's own write lock. Nobody should have to see it in `git status`. (root / ".gitignore").write_text(f"{LOCK}\n", encoding="utf-8") + (root / "nodes" / f"question-{name}.md").write_text( + TEMPLATE_QUESTION.format(name=name), encoding="utf-8") (root / "nodes" / "gate-example.md").write_text(TEMPLATE_GATE, encoding="utf-8") print(f"created graph '{name}'\n") print(f" {name}/graph.yaml <- edit the rules for THIS topic") - print(f" {name}/nodes/ <- one markdown file per hypothesis / gate / source\n") + print(f" {name}/nodes/ <- one markdown file per question / source / idea / claim\n") print(f" next: cd {name} && git init && knoten hook") print(" (the hook makes `git commit` refuse a graph that breaks its own rules)") return 0 diff --git a/src/knoten/mcp_server.py b/src/knoten/mcp_server.py index 685af02..9938048 100644 --- a/src/knoten/mcp_server.py +++ b/src/knoten/mcp_server.py @@ -59,9 +59,12 @@ claim carries why it died and what would bring it back. The graph's own `graph.yaml` declares the node kinds and, where the author wrote them, -what each word means — read it first, because knoten defines none of them. A common shape -is source -> idea -> hypothesis -> experiment -> finding, with findings opening new ideas; -a gate stands outside that loop as the bar every claim must survive. Nothing is +what each word means — read it first, because knoten defines none of them. A graph starts +from ONE question, statement or task and everything descends from it: question -> source +-> idea -> hypothesis -> experiment -> finding, with findings opening new ideas. A gate +stands outside that loop as the bar every claim must survive. Sources are papers, posts, +datasets and searches — and your own intuition, which is worth recording as a source too +so that an idea always names where it came from. Nothing is one-to-one: one hypothesis may have several experiments and several findings. An edge always points from the NEW node to the one it depends on — claim -> the gate it diff --git a/src/knoten/viz.html b/src/knoten/viz.html index cce8fe1..2a190ba 100644 --- a/src/knoten/viz.html +++ b/src/knoten/viz.html @@ -217,7 +217,7 @@ // A claim nothing can reopen. Everything else still has somewhere to go. const CLOSED = ["dead", "retracted", "superseded"]; const GATE_RELS = ["kn:survivedGate", "kn:killedByGate"]; -const LOOP = ["source","idea","question","hypothesis","experiment","finding"]; +const LOOP = ["question","source","idea","hypothesis","experiment","finding"]; const CARDH = 74, CARDW = 250, COLW = 300; document.title = "knoten · " + DATA.root; diff --git a/src/knoten/viz.py b/src/knoten/viz.py index 3460ee2..a293d16 100644 --- a/src/knoten/viz.py +++ b/src/knoten/viz.py @@ -39,7 +39,7 @@ # Only an ordering hint. A type this does not name is not dropped — it lands after the # ones that are, in the order the graph first used it. knoten declares no vocabulary. -FLOW = ["source", "idea", "question", "hypothesis", "experiment", "finding", +FLOW = ["question", "source", "idea", "hypothesis", "experiment", "finding", "blocker", "retraction"] diff --git a/tests/test_cli.py b/tests/test_cli.py index 6208b61..f7d4b2f 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -338,3 +338,49 @@ def test_init_ignores_the_lock_file(tmp_path, monkeypatch): main(["init", "my-topic"]) assert ".knoten.lock" in (tmp_path / "my-topic" / ".gitignore").read_text() + + +def test_init_scaffolds_the_question_the_graph_exists_to_answer(tmp_path, monkeypatch): + """A graph starts from a question, a statement or a task — everything else is + downstream of it. As prose in `graph.yaml: description` nothing could cite it, so a + finding could not be traced back to the question it serves and a second sub-question + had nowhere to live.""" + monkeypatch.chdir(tmp_path) + + assert main(["init", "demo"]) == 0 + monkeypatch.chdir(tmp_path / "demo") + + assert (tmp_path / "demo" / "nodes" / "question-demo.md").exists() + assert main(["validate"]) == 0 + + +def test_the_question_comes_before_everything_else(tmp_path, monkeypatch): + """Column order is the research order. `question` used to sit third in it, after + `source` and `idea` — behind the two things that derive from it.""" + from knoten import viz + monkeypatch.chdir(tmp_path) + main(["init", "demo"]) + root = tmp_path / "demo" + for nid, ntype in [("source-a-paper", "source"), ("idea-a", "idea")]: + (root / "nodes" / f"{nid}.md").write_text( + f"---\nid: {nid}\ntype: {ntype}\nstatus: open\n---\n\n# x\n", encoding="utf-8") + + cols, _, _ = viz.roles(load(root)) + + assert cols[0] == "question" + assert cols.index("source") < cols.index("idea") + + +def test_a_hunch_is_a_source_like_any_other(tmp_path, monkeypatch): + """Own intuition is where a lot of research actually starts. Recording it as a source + keeps one rule — every idea names where it came from — and makes the question + answerable: how much of this graph rests on hunches rather than on reading?""" + monkeypatch.chdir(tmp_path) + main(["init", "demo"]) + root = tmp_path / "demo" + (root / "nodes" / "source-own-intuition.md").write_text( + "---\nid: source-own-intuition\ntype: source\nstatus: open\n---\n\n# A hunch\n", + encoding="utf-8") + monkeypatch.chdir(root) + + assert main(["validate"]) == 0 From f4e93fc75bcde08bfe4951f1cf070afa4e9a6c32 Mon Sep 17 00:00:00 2001 From: BY571 Date: Wed, 19 Aug 2026 15:36:57 +0200 Subject: [PATCH 2/2] put the question first for real, and stop shipping a stale copy of the package MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A simplifier pass found that the headline claim of this branch was not true. THE REORDER DID NOT DELIVER QUESTION-FIRST. `roles()` sorted SHELVES ahead of everything, and a shelf is any type that is cited and never cites. A `source` an idea derives from, on a graph where nothing happens to cite the question back, is a shelf — so `source` overtook `question` and the column order contradicted the loop it exists to show. Reproduced both ways before fixing. A type FLOW names is now placed by FLOW whatever its edges look like; shelf/flow detection only decides where the types FLOW does NOT name go. AND THE TEST COULD NOT HAVE CAUGHT IT. Its fixture had no edges, so there were no shelves, so the rule that actually decides ordering never ran and `cols[0] == "question"` passed vacuously. It now builds the real chain — an idea deriving from a source, nothing citing the question — which is exactly the arrangement that failed. Also from the review: - TEMPLATE_QUESTION's closing paragraph was three lines of documentation — a fourth copy of a sentence already in SKILL, INSTRUCTIONS, README and SPEC — committed verbatim into every graph anyone creates, where it would outlive its own accuracy. One line now, in the register TEMPLATE_GATE already uses. - `knoten init` never named the file it just wrote, so the single most important next action was invisible on the onboarding surface. - README's `node_types` example was a stale copy of the template: no `question`, and the exact "external material" wording this branch set out to broaden. `test_docs` extracts that block but only checks it loads, so the drift sailed through. - The `source:` meaning was 141 chars, twice its longest sibling, and wrapped to four lines in a legend panel that is a definition list. The argument for writing hunches down belongs in the README paragraph, not in every graph's config. - `examples/llm-research` had no question node, so the shipped example contradicted the shipped doctrine. It has one now, and `knoten path` traces question -> hypothesis -> the gate that killed it, which is the whole reason the question is a node. - `build/lib/knoten/` was TRACKED: twelve files, a stale copy of the package still carrying the old FLOW order and no TEMPLATE_QUESTION. Pre-existing, but this branch widened the divergence, and grep in a codebase whose value is reading it in one sitting was returning two contradictory answers. --- .gitignore | 1 + README.md | 3 +- SPEC.md | 5 +- build/lib/knoten/__init__.py | 10 - build/lib/knoten/attachments.py | 181 ------ build/lib/knoten/cli.py | 598 ------------------ build/lib/knoten/commit.py | 96 --- build/lib/knoten/core.py | 483 -------------- build/lib/knoten/hook.py | 75 --- build/lib/knoten/mcp_server.py | 312 --------- build/lib/knoten/ops.py | 190 ------ build/lib/knoten/update.py | 123 ---- build/lib/knoten/validate.py | 272 -------- build/lib/knoten/viz.html | 581 ----------------- build/lib/knoten/viz.py | 255 -------- examples/llm-research/graph.yaml | 2 +- .../nodes/hyp-self-consistency.md | 1 + .../nodes/question-what-improves-reasoning.md | 16 + src/knoten/cli.py | 8 +- src/knoten/mcp_server.py | 6 +- src/knoten/viz.py | 14 +- tests/test_cli.py | 11 +- 22 files changed, 46 insertions(+), 3197 deletions(-) delete mode 100644 build/lib/knoten/__init__.py delete mode 100644 build/lib/knoten/attachments.py delete mode 100644 build/lib/knoten/cli.py delete mode 100644 build/lib/knoten/commit.py delete mode 100644 build/lib/knoten/core.py delete mode 100644 build/lib/knoten/hook.py delete mode 100644 build/lib/knoten/mcp_server.py delete mode 100644 build/lib/knoten/ops.py delete mode 100644 build/lib/knoten/update.py delete mode 100644 build/lib/knoten/validate.py delete mode 100644 build/lib/knoten/viz.html delete mode 100644 build/lib/knoten/viz.py create mode 100644 examples/llm-research/nodes/question-what-improves-reasoning.md diff --git a/.gitignore b/.gitignore index 9f5c036..8aa940e 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,4 @@ dist/ .pytest_cache/ .coverage .knoten.lock +build/ diff --git a/README.md b/README.md index 2aa1733..e2ce7ea 100644 --- a/README.md +++ b/README.md @@ -248,7 +248,8 @@ Because it invents none, the only place your words can be *defined* is your own ```yaml node_types: - source: external material the work starts from — a paper, dataset or search + question: what this graph exists to answer — a question, a statement or a task + source: where the work came from — a paper, dataset, search, or your own intuition idea: what you took from a source; a direction, not yet a testable claim hypothesis: a falsifiable claim derived from an idea experiment: the test built to verify or falsify a hypothesis diff --git a/SPEC.md b/SPEC.md index 8b3a3e8..3b64bd3 100644 --- a/SPEC.md +++ b/SPEC.md @@ -81,9 +81,8 @@ rather than thinking.** A convention only — the core checks `node_types` for membership and nothing else. A graph begins with a `question` and everything descends from it; a `source` is where the work came -from, including the author's own intuition. `gate` stands outside the loop the others -form: it is the bar a claim must survive, not a stage -it passes through. `method` is deliberately unclaimed; the rename that freed it reserved +from, including the author's own intuition. `gate` stands outside the loop the others form: +it is the bar a claim must survive, not a stage it passes through. `method` is deliberately unclaimed; the rename that freed it reserved it for "the approach derived from findings that survived", and nothing implements that yet. diff --git a/build/lib/knoten/__init__.py b/build/lib/knoten/__init__.py deleted file mode 100644 index 6c294cf..0000000 --- a/build/lib/knoten/__init__.py +++ /dev/null @@ -1,10 +0,0 @@ -"""knoten — falsification-first research graphs in git. - -Nodes are markdown. Edges are typed. Claims must cite the gates they survived. -Git does versioning, branching, PRs and hosting; knoten does the two things git -cannot: enforce the graph's own rules, and traverse the falsification structure. -""" -from .core import GraphError, Node, load, parse_text # noqa: F401 -from .validate import Violation, check, load_rules # noqa: F401 - -__version__ = "0.2.0" diff --git a/build/lib/knoten/attachments.py b/build/lib/knoten/attachments.py deleted file mode 100644 index 4f5d6c3..0000000 --- a/build/lib/knoten/attachments.py +++ /dev/null @@ -1,181 +0,0 @@ -"""Attaching files to a node — the code that killed it, the plot that shows why. - -Print-free on purpose: the MCP server speaks JSON-RPC over STDOUT, so a stray `print` -here would corrupt every response. This returns data; the CLI does the printing. -""" -from __future__ import annotations - -import json -import re -import shutil -from dataclasses import dataclass, field -from pathlib import Path - -import yaml - -from .core import (FM_RE, GraphError, _Loader, graph_lock, node_path, - read_frontmatter, split, write_atomic) - -IMG = {".png", ".jpg", ".jpeg", ".gif", ".svg", ".webp"} -BIG = 1_000_000 # git is for code and plots, not datasets - - -@dataclass -class Attached: - added: list[str] = field(default_factory=list) # what this call copied in - embedded: list[str] = field(default_factory=list) # images written into the body - warnings: list[str] = field(default_factory=list) - - -def _node_file(root: Path, nid: str) -> Path: - nf = node_path(root, nid) # rejects a traversal before it becomes a path - if not nf.exists(): - raise GraphError(f"no node '{nid}'") - return nf - - -def _scalar(name: str) -> str: - """Emit a filename as YAML, quoting it unless it round-trips as the identical string. - - Filenames were written raw. `plot #1.png` became the YAML comment `plot` (and then a - false missing-attachment violation, with the real file sitting on disk); `run: final - .png` became a dict; `123` became an int. All are legal filenames. - """ - try: - if yaml.load(name, Loader=_Loader) == name: - return name - except yaml.YAMLError: - pass - return json.dumps(name, ensure_ascii=False) # a JSON string is valid YAML - - -def set_list(node_file: Path, names: list[str]) -> None: - """Rewrite ONLY the `attachments:` block, line by line. - - We do not re-dump the frontmatter through yaml — that would reformat it and strip the - comments a human wrote. - - The items we must drop may be indented OR NOT: `yaml.dump` emits list items at zero - indent by default. Requiring leading whitespace orphaned them into the preceding block - and left the node unparseable — which takes the whole graph down with it, since load() - raises rather than skips. - """ - text = node_file.read_text(encoding="utf-8") - read_frontmatter(node_file) # refuse to touch a node we cannot parse - m = FM_RE.match(text) - - kept, dropping = [], False - for line in m.group(1).splitlines(): - if re.match(r"^attachments:\s*(\[.*\])?\s*$", line): - dropping = True - continue - if dropping: - if re.match(r"^\s*-\s", line): # indented or not - continue - dropping = False - kept.append(line) - - if names: - kept.append("attachments:") - kept += [f" - {_scalar(n)}" for n in sorted(names)] - fm = "\n".join(kept).strip("\n") - out = f"---\n{fm}\n---\n{m.group(2)}" - - split(out, node_file.name) # never write a node we cannot read back - write_atomic(node_file, out) - - -def _embed(node_file: Path, nid: str, images: list[str]) -> list[str]: - """Images render on GitHub only if they are in the body, not just the frontmatter.""" - text = node_file.read_text(encoding="utf-8") - embedded = [] - for name in images: - ref = f"![{name}](../attachments/{nid}/{name})" - if ref in text: - continue - if "## Attachments" not in text: - text = text.rstrip() + "\n\n## Attachments\n" - text = text.rstrip() + f"\n\n{ref}\n" - embedded.append(name) - if embedded: - write_atomic(node_file, text) - return embedded - - -def _preflight(files: list[str]) -> list[Path]: - """Vet every file BEFORE copying any, so a bad path halfway through the list cannot - leave the node half-attached. - - `exists()` is true for a directory, so the old pre-check passed one and copy2 then - died mid-way, orphaning the files already copied. - """ - srcs = [Path(f) for f in files] - for src in srcs: - if not src.exists(): - raise GraphError(f"no such file: {src}") - if not src.is_file(): - raise GraphError(f"{src} is not a file") - - names = [s.name for s in srcs] - if dupes := {n for n in names if names.count(n) > 1}: - raise GraphError( - f"two files share the basename {', '.join(sorted(dupes))} — they would land on " - f"top of each other in attachments/. Rename one.") - return srcs - - -def attach(root: Path, nid: str, files: list[str]) -> Attached: - with graph_lock(root): - return _attach(root, nid, files) - - -def _attach(root: Path, nid: str, files: list[str]) -> Attached: - nf = _node_file(root, nid) - fm, _ = read_frontmatter(nf) - names = [str(a) for a in (fm.get("attachments") or [])] - out = Attached() - srcs = _preflight(files) - - dest = root / "attachments" / nid - dest.mkdir(parents=True, exist_ok=True) - images = [] - for src in srcs: - if src.is_symlink(): - out.warnings.append( - f"{src.name} is a symlink to {src.resolve()} — its CONTENTS were copied " - f"into the graph.") - if (size := src.stat().st_size) > BIG: - out.warnings.append( - f"{src.name} is {size / 1e6:.1f} MB — git is for code and plots, not " - f"datasets. Consider linking to it instead.") - shutil.copy2(src, dest / src.name) - if src.name not in names: - names.append(src.name) - out.added.append(src.name) - if src.suffix.lower() in IMG: - images.append(src.name) - - set_list(nf, names) - out.embedded = _embed(nf, nid, images) - return out - - -def detach(root: Path, nid: str, name: str) -> None: - with graph_lock(root): - _detach(root, nid, name) - - -def _detach(root: Path, nid: str, name: str) -> None: - nf = _node_file(root, nid) - fm, _ = read_frontmatter(nf) - have = [str(a) for a in (fm.get("attachments") or [])] - if name not in have: - raise GraphError(f"'{name}' is not attached to {nid}") - - (root / "attachments" / nid / name).unlink(missing_ok=True) - set_list(nf, [a for a in have if a != name]) - - text = nf.read_text(encoding="utf-8") - text = re.sub(rf"\n*!\[{re.escape(name)}\]\([^)]*\)\n*", "\n", text) - text = re.sub(r"\n*## Attachments\s*\n+(?=(##|\Z))", "\n", text) - write_atomic(nf, text.rstrip() + "\n") diff --git a/build/lib/knoten/cli.py b/build/lib/knoten/cli.py deleted file mode 100644 index f15ab12..0000000 --- a/build/lib/knoten/cli.py +++ /dev/null @@ -1,598 +0,0 @@ -"""knoten — a falsification-first research graph. - -A graph is a FOLDER: graph.yaml (the rules) + nodes/ (the knowledge). One graph per -research topic. Each declares its own rules; the core knows nothing about any domain. -""" -from __future__ import annotations - -import argparse -import json -import sys -import webbrowser -from pathlib import Path - -from . import attachments, ops, viz -from .commit import commit -from .core import GraphError, ID_RE, LOCK, find_root, node_path, today -from .hook import install as install_hook -from .validate import _csv, applies, load_config - -# Keyed by the uppercase word `ops` puts in `verdict` — not by raw status, which is -# lowercase and includes values (open, active, …) this table has no symbol for. -MARK = {"ALIVE": "✓ ALIVE", "DEAD": "✗ DEAD", "RETRACTED": "⊘ RETRACTED"} - - -# ---------------------------------------------------------------- read commands - -def _emit(payload: dict, as_json: bool, render) -> None: - if as_json: - print(json.dumps(payload, indent=2, default=str)) - else: - render(payload) - - -def _fail(payload: dict, reason, as_json: bool) -> int: - """Every failure on this surface, one contract. --json keeps the structured payload on - stdout even on failure, so a machine reader never has to check a second stream for the - error; prose puts it on stderr, where every other command's GraphError goes.""" - if as_json: - print(json.dumps(payload, indent=2, default=str)) - else: - print(f"knoten: {reason}", file=sys.stderr) - return 1 - - -def render_validate(payload: dict) -> None: - print(f"{payload['nodes']} nodes\n") - if payload["valid"]: - print(" ✓ all rules pass") - return - for v in payload["violations"]: - print(f" ✗ {v['node']}\n [{v['rule']}] {v['message']}") - print(f"\n {len(payload['violations'])} violation(s) — commit REJECTED") - - -def validate(root, as_json=False) -> int: - payload = ops.validate(root) - _emit(payload, as_json, render_validate) - return 0 if payload["valid"] else 1 - - -def render_query(payload: dict) -> None: - print(f'"{payload["query"]}" → {payload["total"]} claim(s)\n') - # Relevance order, NOT status order: the closest match must be first, because an - # agent reads the top of this list and stops. - for c in payload["claims"]: - print(f" [{MARK[c['verdict']]}] {c['id']}") - for key, label in [("killed_by", "killed by"), ("survived_gates", "survived "), - ("retracted_by", "RETRACTED by"), ("superseded_by", "superseded by")]: - if ts := c.get(key): - print(f" {label} : {', '.join(ts)}") - if reopen := c.get("what_would_reopen_this"): - print(f" reopen if : {reopen[:140]}…") - print() - if payload["related"]: - print(" also: " + ", ".join(payload["related"])) - if note := payload.get("note"): - # The guard against the one failure knoten exists to prevent (a false - # "untested") lives in `note`. Dropping it in prose left an agent reading the - # surface SKILL.md tells it to prefer with no caveat at all. - print(f"\n {note}") - - -def query(root, term, as_json=False) -> int: - payload = ops.query(root, term) - _emit(payload, as_json, render_query) - return 0 - - -def _pairs(pairs, msg): - """Yield (key, raw value) for each `KEY=VALUE` string in `pairs` — only the key is - stripped here, since `_kv` deliberately leaves its value alone while `_where` and - `_links` strip theirs. `msg` is the caller's own error text, with `{}` for the - offending item.""" - for p in pairs or []: - if "=" not in p: - raise GraphError(msg.format(p)) - k, v = p.split("=", 1) - yield k.strip(), v - - -def _where(pairs) -> dict: - """`--where cause=weak_baseline`, repeatable. Values for one field accumulate as - alternatives, so `--where cause=a --where cause=b` reads as "a or b".""" - out = {} - for k, v in _pairs(pairs, "--where takes key=value, got '{}'"): - out.setdefault(k, []).append(v.strip()) - return out - - -def render_index(payload: dict) -> None: - shown = payload["nodes"] - width = max((len(n["id"]) for n in shown), default=0) - for n in shown: - mark = MARK.get(n["verdict"], n["verdict"]) - tag = f"[{','.join(n['tags'])}]" if n["tags"] else "" - print(f" {n['id']:{width}} {mark:12} {tag:24} {n['title']}") - print(f"\n {len(shown)} of {payload['total']} node(s)") - if payload["truncated"]: - # Never a silent cap: a truncated list reads as the whole graph. - print(" (truncated — narrow with --tag/--status/--type, or raise --limit)") - if note := payload.get("note"): - print(f"\n {note}") - - -def index(root, tags, status, ntype, where, since, limit, query=None, as_json=False) -> int: - """The whole graph, one line per node. The answer to "have we done anything LIKE - this?" that keyword search cannot give: a reader — human or agent — judges - relatedness from the claims themselves.""" - payload = ops.index(root, query=query, tags=tags, status=status, type=ntype, - where=_where(where), since=since, limit=limit) - _emit(payload, as_json, render_index) - return 0 - - -def render_gates(payload: dict) -> None: - for g in payload["gates"]: - killed, survived = g["killed"], g["survived"] - record = f"killed {len(killed)}, survived by {len(survived)}" if killed or survived \ - else "never applied" - print(f" {g['id']} ({record})") - print(f" {g['title']}") - if rule := g.get("rule"): - print(f" the rule : {rule[:160]}") - print() - if note := payload.get("note"): - print(f" {note}") - - -def gates_cmd(root, as_json=False) -> int: - """What every claim in this graph has to survive. Read it before you design the - experiment, not after the commit is refused.""" - payload = ops.gates(root) - _emit(payload, as_json, render_gates) - return 0 - - -def render_frontier(payload: dict) -> None: - if payload["open"]: - print(" OPEN — started, never settled") - for n in payload["open"]: - print(f" {n['id']:24} {n['title']}") - if payload["reopenable"]: - print("\n REOPENABLE — died, but said what would bring them back") - for n in payload["reopenable"]: - print(f" {n['id']:24} {n['title']}") - print(f" reopen if : {n['reopen_if'][:120]}…") - if payload["untested_gates"]: - print("\n UNTESTED GATES — no claim has been through them") - for n in payload["untested_gates"]: - print(f" {n['id']:24} {n['title']}") - if not (payload["open"] or payload["reopenable"] or payload["untested_gates"]): - print(" nothing open, nothing reopenable, every gate has fired.") - if note := payload.get("note"): - print(f"\n {note}") - - -def frontier_cmd(root, as_json=False) -> int: - """The one screen that answers "what now?". Kept short on purpose — a frontier you - have to scroll is a frontier nobody reads.""" - payload = ops.frontier(root) - _emit(payload, as_json, render_frontier) - return 0 - - -def render_path(payload: dict) -> None: - p = payload["path"] - if p is None: - print(payload["note"]) - return - print(f"research path {p[0]['node']} → {p[-1]['node']}:\n") - for i, hop in enumerate(p): - rel = hop.get("via") - print(" " * i + (f"└─ {rel} → " if rel else "") + hop["node"]) - - -def path(root, a, b, as_json=False) -> int: - payload = ops.path(root, a, b) - _emit(payload, as_json, render_path) - return 0 - - -def viz_cmd(root, out, show) -> int: - """One HTML file. Read-only, self-contained, no server.""" - dest = viz.write(root, Path(out)) - print(f"wrote {dest} ({dest.stat().st_size // 1024} KB)") - if show: - webbrowser.open(dest.resolve().as_uri()) - return 0 - - -def hook(root, force) -> int: - h = install_hook(root, force=force) - print(f" ✓ installed {h}") - print(" `git commit` now runs `knoten validate` and refuses a broken graph.") - return 0 - - -def render_get(payload: dict) -> None: - print(f"{payload['id']} [{MARK.get(payload['verdict'], payload['verdict'])}] " - f"type={payload['type']}\n") - for l in payload["links"]: - print(f" {l['rel']:22} -> {l['to']}") - for b in payload["backlinks"]: - print(f" {b['rel']:22} <- {b['to']}") - for label, d in [("repro", payload.get("repro")), ("results", payload.get("results"))]: - if d: - print(f"\n {label}:") - for k, v in d.items(): - print(f" {k}: {v}") - if atts := payload.get("attachment_files"): - print("\n attachments:") - for a in atts: - sz = f"{a['size_kb']:.1f} KB" if "size_kb" in a else "MISSING" - print(f" {a['path']} ({sz})") - - -def show(root, nid, as_json=False) -> int: - payload = ops.get(root, nid) - if err := payload.get("error"): - return _fail(payload, err, as_json) - _emit(payload, as_json, render_get) - return 0 - - -# ---------------------------------------------------------------- write commands - -def _read(arg: str) -> str: - """A file path, or `-` for stdin. Frontmatter and bodies are multi-line YAML and - markdown; passing them as shell arguments is how quoting bugs get into a research - record.""" - return sys.stdin.read() if arg == "-" else Path(arg).read_text(encoding="utf-8") - - -def _kv(pairs) -> dict: - """`--result acc=0.7`, repeatable. Typed rather than left as strings, because - `require_result_min` compares numerically.""" - out = {} - for k, v in _pairs(pairs, "--result takes key=value, got '{}'"): - # Deliberately NOT stripped — alone among the four parsers. `--result "note= fine "` - # writes ' fine ' to disk as-is. That asymmetry is existing behaviour, kept. - try: - v = float(v) - except ValueError: - pass - out[k] = v - return out - - -def _fields(pairs) -> dict: - """`--field cause=weak_baseline`, repeatable. Left as STRINGS, unlike `_kv`. - - `_kv` coerces because `require_result_min` compares numerically. `require_field_one_of` - and `--where` both compare with `str()`, so coercing `--field seed=2` to 2.0 made it - match nothing the graph declared — and the refusal quoted `seed=2.0`, a value the user - never typed. Stripped, because this is the write side of `--where`, which strips: the - two must round-trip. - """ - return {k: v.strip() for k, v in _pairs(pairs, "--field takes key=value, got '{}'")} - - -def _links(pairs) -> list[dict]: - """`--link kn:killedByGate=method-x`, repeatable.""" - out = [] - for rel, to in _pairs(pairs, "--link takes rel=to, got '{}'"): - out.append({"rel": rel, "to": to.strip()}) - return out - - -def render_commit(payload: dict) -> None: - print(f" + {payload['path']} ({payload['graph_size']} nodes)") - if warning := payload.get("warning"): - print(f"\n ! {warning}") - for s in payload["similar"]: - print(f" {s['id']} [{MARK.get(s['verdict'], s['verdict'])}] {s['title']}") - - -def commit_cmd(root, nid, frontmatter, body, as_json) -> int: - res = commit(root, nid, _read(frontmatter), _read(body)) - if res["status"] == "REJECTED": - return _fail(res, res.get("reason") or "; ".join( - f"[{v['rule']}] {v['message']}" for v in res["violations"]), as_json) - _emit(res, as_json, render_commit) - return 0 - - -def render_update(payload: dict) -> None: - print(f" {payload['node']} -> {payload['node_status']}") - - -def update_cmd(root, nid, status, append, results, links, fields, as_json) -> int: - # ops.update() is the ONE shape for both outcomes — this used to build its own - # dict here, and a different one in mcp_server.py, and the two shapes drifted. - payload = ops.update(root, nid, status=status, append=_read(append) if append else None, - results=_kv(results), links=_links(links), fields=_fields(fields)) - if payload["status"] == "REJECTED": - return _fail(payload, payload["reason"], as_json) - _emit(payload, as_json, render_update) - return 0 - - -def attach(root, nid, files) -> int: - res = attachments.attach(root, nid, files) - for w in res.warnings: - print(f" ! {w}") - for name in res.added: - print(f" + attachments/{nid}/{name}") - if res.embedded: - print(f" embedded {len(res.embedded)} image(s) in the node body") - return 0 - - -def detach(root, nid, name) -> int: - attachments.detach(root, nid, name) - print(f" - detached {name} from {nid}") - return 0 - - -TEMPLATE_GRAPH = """\ -# {name} — a knoten research graph. -# -# The core knows NOTHING about this domain. Every rule below is declared HERE, as -# data. Write a rule only when you have a corpse: a rule without a body behind it is -# just friction. -name: {name} -description: TODO — what question is this graph about? - -# Enforced. A node whose type or status is not on these lists is a typo — and a claim -# with a typo'd status silently drops out of every query. Edit them for YOUR topic. -node_types: [hypothesis, experiment, finding, method, source, retraction] -statuses: [open, alive, dead, retracted, superseded, active] - -# The axis `knoten index --tag` filters on. Declare them and a typo is a violation; -# declare none and tagging is free. Add tags as the topic tells you what they are. -# tags: [decoding, evaluation] - -# Reused standards: mp:supports / mp:challenges (Micropublications), -# npx:retracts / npx:supersedes (Nanopublications), -# prov:wasDerivedFrom / prov:used (PROV-O) -# knoten adds: kn:survivedGate (claim -> the method it PASSED) -# kn:killedByGate (claim -> the method that KILLED it) -# kn:blockedBy (claim -> a structural wall, not a result) - -rules: - - id: live-claims-must-cite-their-gates - when_status: alive - when_type: hypothesis, finding - require_edge: kn:survivedGate - message: An unchallenged claim is not a finding, it is a hope. - - - id: dead-claims-must-say-why - when_status: dead, retracted - require_sections: Why it died, What would reopen this - message: The post-mortem IS the asset — a dead end must become a standing offer. -""" - -TEMPLATE_METHOD = """\ ---- -id: method-example-gate -type: method -status: active ---- -# Gate: - -## The rule - - -## Why it exists - - -Replace this with a real gate. Delete it if you have none yet — but you will. -""" - - -def new(root, ntype, nid, status) -> int: - """Scaffold a node carrying every section and field THIS graph's rules demand. - - Nothing here is knoten's opinion — it reads the graph's own declarations. The values - are TODO on purpose: `knoten validate` then names the ones you still owe it, so `new` - + `validate` is a checklist rather than a guessing game. - """ - nf = node_path(root, nid) - if nf.exists(): - raise GraphError(f"'{nid}' already exists. Supersede or retract it — corrections " - f"are nodes, not edits.") - - cfg = load_config(root) - # `new` used to skip this while knoten_commit enforced it: the same node was accepted - # by one entry point and rejected by the other. - for field, declared in [("type", cfg.get("node_types")), ("status", cfg.get("statuses"))]: - value = ntype if field == "type" else status - if declared and value not in declared: - raise GraphError(f"{field} '{value}' is not declared in graph.yaml " - f"({field}s: {', '.join(map(str, declared))})") - - sections, results, fields = [], [], {} - for r in cfg.get("rules", []): - if not applies(status, ntype, r): - continue - sections += _csv(r.get("require_sections")) - results += [k for k in [r.get("require_result")] if k] - results += list(r.get("require_result_min") or {}) - fields.update(r.get("require_field_one_of") or {}) - - fm = [f"id: {nid}", f"type: {ntype}", f"status: {status}", f"created: {today()}"] - # The allowed values go in the scaffold as a comment: a closed vocabulary the author - # has to go and look up is a closed vocabulary they will guess at. - fm += [f"{k}: TODO # one of: {', '.join(map(str, v))}" for k, v in fields.items()] - if results: - fm.append("results:") - fm += [f" {k}: TODO" for k in dict.fromkeys(results)] - - body = ["# TODO — state the claim in one line\n"] - body += [f"## {s}\nTODO\n" for s in dict.fromkeys(sections)] - - nf.write_text("---\n" + "\n".join(fm) + "\n---\n\n" + "\n".join(body), encoding="utf-8") - print(f" + nodes/{nid}.md ({ntype}, {status})") - if wanted := ([f"## {s}" for s in dict.fromkeys(sections)] - + list(dict.fromkeys(results)) + list(fields)): - print(f" pre-filled what THIS graph's rules require: {', '.join(wanted)}") - return 0 - - -def init(name) -> int: - if not ID_RE.match(name): - raise GraphError(f"'{name}' is not a valid graph name (use kebab-case: my-topic)") - root = Path.cwd() / name - if root.exists(): - raise GraphError(f"{root} already exists") - (root / "nodes").mkdir(parents=True) - (root / "graph.yaml").write_text(TEMPLATE_GRAPH.format(name=name), encoding="utf-8") - # knoten's own write lock. Nobody should have to see it in `git status`. - (root / ".gitignore").write_text(f"{LOCK}\n", encoding="utf-8") - (root / "nodes" / "method-example-gate.md").write_text(TEMPLATE_METHOD, encoding="utf-8") - print(f"created graph '{name}'\n") - print(f" {name}/graph.yaml <- edit the rules for THIS topic") - print(f" {name}/nodes/ <- one markdown file per hypothesis / method / source\n") - print(f" next: cd {name} && git init && knoten hook") - print(" (the hook makes `git commit` refuse a graph that breaks its own rules)") - return 0 - - -# ---------------------------------------------------------------- entry point - -def _parser() -> argparse.ArgumentParser: - p = argparse.ArgumentParser( - prog="knoten", description=__doc__, - formatter_class=argparse.RawDescriptionHelpFormatter) - sub = p.add_subparsers(dest="cmd") - - s = sub.add_parser("init", help="start a NEW graph for a topic") - s.add_argument("name") - - s = sub.add_parser("validate", help="enforce THIS graph's declared rules") - s.add_argument("--json", action="store_true", help="emit the raw payload") - - s = sub.add_parser("new", help="scaffold a node with whatever the rules demand") - s.add_argument("type") - s.add_argument("id") - s.add_argument("--status", default="open") - - s = sub.add_parser("query", help='"has this been tried?" -> verdicts + causes of death') - s.add_argument("term") - s.add_argument("--json", action="store_true", help="emit the raw payload") - - s = sub.add_parser("index", help="the whole graph, one line per node") - s.add_argument("--query", help="rank the rows by relevance to this, instead of by id") - s.add_argument("--tag", action="append", help="keep nodes carrying this tag (repeatable)") - s.add_argument("--status", action="append") - s.add_argument("--type", action="append") - s.add_argument("--where", action="append", metavar="KEY=VALUE", - help="keep nodes whose frontmatter KEY is VALUE (repeatable)") - s.add_argument("--since", metavar="YYYY-MM-DD", - help="only nodes created or updated on/after this day") - s.add_argument("--limit", type=int, - help=f"0 = the default cap ({ops.INDEX_LIMIT}); never uncapped, so a " - f"truncated list can't silently read as the whole graph") - s.add_argument("--json", action="store_true", help="emit the raw payload") - - s = sub.add_parser("frontier", help="what should I work on next?") - s.add_argument("--json", action="store_true", help="emit the raw payload") - - s = sub.add_parser("gates", help="what must a claim survive here?") - s.add_argument("--json", action="store_true", help="emit the raw payload") - - s = sub.add_parser("path", help="how did we get from A to B?") - s.add_argument("a") - s.add_argument("b") - s.add_argument("--json", action="store_true", help="emit the raw payload") - - s = sub.add_parser("viz", help="write the graph as one self-contained HTML file") - s.add_argument("-o", "--out", default="knoten.html", help="where to write it") - s.add_argument("--open", dest="show", action="store_true", help="open it when done") - - s = sub.add_parser("hook", help="install the git pre-commit gate") - s.add_argument("--force", action="store_true", - help="overwrite a pre-commit hook knoten did not write") - - s = sub.add_parser("show", help="the node, its edges and its attachments") - s.add_argument("node") - s.add_argument("--json", action="store_true", help="emit the raw payload") - - s = sub.add_parser("commit", help="file a new claim — gate-checked before it touches disk") - s.add_argument("id") - s.add_argument("--frontmatter", required=True, metavar="FILE", - help="YAML frontmatter (no --- fences) — file path, or - for stdin") - s.add_argument("--body", required=True, metavar="FILE", - help="markdown body — file path, or - for stdin") - s.add_argument("--json", action="store_true", help="emit the raw payload") - - s = sub.add_parser("update", help="move a node through its lifecycle and append to it") - s.add_argument("id") - s.add_argument("--status", help="the new status, e.g. dead") - s.add_argument("--append", metavar="FILE", - help="markdown to append — file path, or - for stdin") - s.add_argument("--result", action="append", metavar="KEY=VALUE", - help="result to record (repeatable)") - s.add_argument("--field", action="append", metavar="KEY=VALUE", - help="set any top-level frontmatter field, including one already " - "recorded (repeatable). The graph's rules decide what is " - "accepted.") - s.add_argument("--link", action="append", metavar="REL=TO", - help="edge to add, e.g. kn:killedByGate=method-x (repeatable)") - s.add_argument("--json", action="store_true", help="emit the raw payload") - - s = sub.add_parser("attach", help="attach a script / plot / notebook to a node") - s.add_argument("node") - s.add_argument("files", nargs="+") - - s = sub.add_parser("detach", help="remove one") - s.add_argument("node") - s.add_argument("file") - - return p - - -def main(argv=None) -> int: - args = _parser().parse_args(argv if argv is not None else sys.argv[1:]) - try: - if args.cmd is None or args.cmd == "validate": - # No subcommand never parsed a `validate` subparser, so it has no --json. - return validate(find_root(), getattr(args, "json", False)) - if args.cmd == "init": - return init(args.name) - - root = find_root() - return { - "query": lambda: query(root, args.term, args.json), - "path": lambda: path(root, args.a, args.b, args.json), - "frontier": lambda: frontier_cmd(root, args.json), - "gates": lambda: gates_cmd(root, args.json), - "index": lambda: index(root, tags=args.tag, status=args.status, ntype=args.type, - where=args.where, since=args.since, limit=args.limit, - query=args.query, as_json=args.json), - "new": lambda: new(root, args.type, args.id, args.status), - "show": lambda: show(root, args.node, args.json), - "commit": lambda: commit_cmd(root, nid=args.id, frontmatter=args.frontmatter, - body=args.body, as_json=args.json), - "update": lambda: update_cmd(root, nid=args.id, status=args.status, - append=args.append, results=args.result, - links=args.link, fields=args.field, - as_json=args.json), - "viz": lambda: viz_cmd(root, args.out, args.show), - "hook": lambda: hook(root, args.force), - "attach": lambda: attach(root, args.node, args.files), - "detach": lambda: detach(root, args.node, args.file), - }[args.cmd]() - except (GraphError, OSError) as e: - # OSError: a typo'd --frontmatter/--body/--append path is ordinary user error, - # not a traceback — mcp_server.tool already guards this for the same reason. - return _fail({"error": str(e)}, e, getattr(args, "json", False)) - - -def cli() -> None: - sys.exit(main()) - - -if __name__ == "__main__": - cli() diff --git a/build/lib/knoten/commit.py b/build/lib/knoten/commit.py deleted file mode 100644 index 6e1b84c..0000000 --- a/build/lib/knoten/commit.py +++ /dev/null @@ -1,96 +0,0 @@ -"""Filing a new claim. - -The gate is the point: the candidate is parsed and rule-checked IN MEMORY, and nothing -reaches the filesystem until it is clean. An agent cannot record a shiny result that cites -no test it survived. - -This lived inside the MCP server, which put the domain logic in the transport layer and — -worse — made writing a node from Python require the `mcp` SDK, an optional dependency for -a transport you may not be using. `attach` and `update` never had that problem. -""" -from __future__ import annotations - -import re -from pathlib import Path - -from .core import (VERDICT, GraphError, Node, backlink, fields, graph_lock, load, - node_path, parse_text, retrieve, section, today, write_atomic) -from .validate import check - - -def _similar(nodes: dict[str, Node], candidate: Node, keep: int = 3) -> list[dict]: - """Settled claims that look like the same question, worded differently. - - A warning and never a block: two claims can be genuinely close and genuinely - different — a compute-matched rerun of a dead idea IS a new claim, and that is the - whole point of a gate. Refusing would make the tool wrong in the interesting case and - push the agent to route around it. - - Settled claims only — `open` is not an answer, and reporting one would tell the agent - the question is closed when it is exactly what is still being asked. Plus at least two - shared title words, so a single shared "accuracy" does not fire. - - Two is a loose bar on purpose. A false positive costs one line of JSON the agent can - dismiss; a false negative costs a duplicated experiment, which is the failure this - whole tool exists to prevent. The asymmetry says lean permissive. - """ - mine = fields(candidate)[0] - out = [] - for n in retrieve(nodes, candidate.title or candidate.id): - if n.status not in VERDICT or len(mine & fields(n)[0]) < 2: - continue - row = {"id": n.id, "verdict": VERDICT[n.status], "title": n.title} - if why := section(n.body, "Why it died"): - row["why_it_died"] = why[:200] - out.append(row) - return out[:keep] - - -def commit(root: Path, nid: str, frontmatter: str, body: str) -> dict: - """Write a new node, or report why it cannot be written. Never raises for a bad - candidate — the caller is usually an agent, and a refusal it can read and act on beats - a traceback it can only give up on.""" - with graph_lock(root): - # Loaded INSIDE the lock. Read outside it, the snapshot goes stale the moment a - # peer commits, and a claim citing the gate that peer just created is rejected for - # a dangling edge to a node already on disk. - nodes = load(root) - try: - path = node_path(root, nid) - except GraphError as e: - return {"status": "REJECTED", "node": nid, "reason": str(e)} - if path.exists(): - return {"status": "REJECTED", "node": nid, - "reason": f"'{nid}' already exists. Supersede or retract it instead of " - "overwriting — corrections are nodes, not edits."} - - fm = frontmatter.strip() - # Stamped unless the author said otherwise. A graph with no time axis cannot - # answer "what did we learn this week" or spot a hypothesis open since March. - if not re.search(r"^created:", fm, re.M): - fm += f"\ncreated: {today()}" - text = f"---\n{fm}\n---\n\n{body.strip()}\n" - - try: - candidate = parse_text(text, nid) - except GraphError as e: - return {"status": "REJECTED", "node": nid, "reason": str(e)} - - if errs := [e for e in check(backlink({**nodes, nid: candidate}), root) - if e.node == nid]: - return {"status": "REJECTED", "node": nid, - "violations": [{"rule": e.rule, "message": e.message} for e in errs], - "hint": "Fix the violations and commit again. The gate is the point."} - - write_atomic(path, text) - - out = {"status": "COMMITTED", "node": nid, "path": f"nodes/{nid}.md", - "graph_size": len(nodes) + 1, - "next": "git add + commit to version this."} - if similar := _similar(nodes, candidate): - out["similar"] = similar - out["warning"] = ( - f"This resembles {len(similar)} settled claim(s). If it is the same question, " - f"supersede or retract that node (npx:supersedes / npx:retracts) rather than " - f"leaving two answers in the graph.") - return out diff --git a/build/lib/knoten/core.py b/build/lib/knoten/core.py deleted file mode 100644 index 7ad3ced..0000000 --- a/build/lib/knoten/core.py +++ /dev/null @@ -1,483 +0,0 @@ -"""Parse markdown nodes, build the graph, generate back-links. - -A node is a markdown file; an edge is a typed link in its frontmatter. Git does -versioning, branching, PRs and hosting — we do not. - -The parser's first duty is to REFUSE: a node it cannot read raises, never gets skipped. -A node that silently vanishes leaves the graph reporting itself healthy. -""" -from __future__ import annotations - -import math -import os -import re -import tempfile -from contextlib import contextmanager -from datetime import date -from collections import Counter, defaultdict, deque -from dataclasses import dataclass, field -from pathlib import Path - -import yaml - -try: # POSIX only; Windows falls back to no lock - import fcntl -except ImportError: # pragma: no cover - fcntl = None - -LOCK = ".knoten.lock" - - -class GraphError(Exception): - """The graph on disk is malformed. Always name the file.""" - - -# Edges are declared ONCE, on the subject. Back-links are generated, never authored. -INVERSE = { - "kn:survivedGate": "kn:gateSurvivedBy", - "kn:killedByGate": "kn:gateKilled", - "kn:blockedBy": "kn:blocks", - "kn:tests": "kn:testedBy", - "mp:supports": "mp:supportedBy", - "mp:challenges": "mp:challengedBy", - "npx:retracts": "npx:retractedBy", - "npx:supersedes": "npx:supersededBy", - "prov:wasDerivedFrom": "prov:hadDerivation", - "prov:used": "prov:wasUsedBy", -} -GENERATED = set(INVERSE.values()) - -# The claim lifecycle. A node with any other status is not a claim (a method, a source). -VERDICT = {"alive": "ALIVE", "dead": "DEAD", "retracted": "RETRACTED"} - -# The three conventions `frontier` reads. They are conventions, not core vocabulary — a -# graph that names things differently gets an empty bucket, not a wrong answer. They are -# named here rather than buried so that stays obvious. -OPEN = "open" -REOPEN_SECTION = "What would reopen this" -GATE_TYPE = "method" -GATE_SECTIONS = ("The rule", "Why it exists") - -FM_RE = re.compile(r"^---\n(.*?)\n---\n?(.*)$", re.S) - -# An id becomes a filename, so anything else is a path traversal. Go through node_path() -# for EVERY id -> file conversion: `knoten detach ../../x f` used to delete a file outside -# the graph, because only the MCP surface (where the id comes from an LLM) was guarded. -ID_RE = re.compile(r"^[a-z0-9][a-z0-9_-]*$") - - -@contextmanager -def graph_lock(root: Path): - """One writer at a time, for the read-modify-write windows. - - `attach` reads a node's frontmatter, copies files, then rewrites the list. Two agents - doing that at once lost one of the two lists — and the file stayed parseable, so - `validate` passed while the frontmatter no longer mentioned a file on disk. Parallel - agents are the obvious way to scale a research loop, and this is the step most likely - to happen at the same moment. - """ - if fcntl is None: # pragma: no cover - yield - return - with open(root / LOCK, "w") as fh: - fcntl.flock(fh, fcntl.LOCK_EX) - try: - yield - finally: - fcntl.flock(fh, fcntl.LOCK_UN) - - -def write_atomic(path: Path, text: str) -> None: - """Write via a temp file in the same directory, then rename. A reader either sees the - old node or the new one — never the half-written one, which does not parse and takes - the whole graph down with it, since load() raises rather than skips.""" - fd, tmp = tempfile.mkstemp(dir=path.parent, prefix=f".{path.name}.", suffix=".tmp") - try: - with os.fdopen(fd, "w", encoding="utf-8") as fh: - fh.write(text) - os.replace(tmp, path) - except BaseException: - Path(tmp).unlink(missing_ok=True) - raise - - -def today() -> str: - """The stamp knoten writes. A plain ISO date: it sorts lexicographically, survives - the YAML 1.2 loader as a string, and diffs cleanly in git.""" - return date.today().isoformat() - - -def node_path(root: Path, nid: str) -> Path: - if not ID_RE.match(nid or ""): - raise GraphError(f"'{nid}' is not a valid node id (kebab-case: hyp-my-idea)") - return root / "nodes" / f"{nid}.md" - - -class _Loader(yaml.SafeLoader): - """YAML 1.2 scalars, not YAML 1.1. - - PyYAML is YAML 1.1, whose implicit typing silently rewrites `results:` — the block - holding the numbers this tool exists to protect: - - tags: [no, off] -> [False, False] the "Norway problem" - wallclock: 12:30 -> 750 sexagesimal - seed: 042 -> 34 octal (but `08` stays a string!) - n: 1_000 -> 1000 digit separators - run: 2024-01-15 -> datetime.date implicit timestamps - - Rather than patch these one at a time, swap the implicit resolvers for the 1.2 core - schema, which has none of them. Anything we cannot confidently type stays a string — - the safe direction, since a string fails a numeric rule loudly while a silently - rewritten number does not. - """ - - -# YAML 1.2 core schema (https://yaml.org/spec/1.2.2/#103-core-schema). -_CORE = [ - ("tag:yaml.org,2002:bool", r"^(?:true|True|TRUE|false|False|FALSE)$", list("tTfF")), - ("tag:yaml.org,2002:int", r"^[-+]?(?:0|[1-9][0-9]*)$" - r"|^0o[0-7]+$|^0x[0-9a-fA-F]+$", list("-+0123456789")), - # A float needs a `.` or an exponent. Without that guard a bare `042` — which the int - # resolver correctly refuses — falls through to float and becomes 42.0. - ("tag:yaml.org,2002:float", r"^[-+]?(?:[0-9]*\.[0-9]+|[0-9]+\.[0-9]*)(?:[eE][-+]?[0-9]+)?$" - r"|^[-+]?[0-9]+[eE][-+]?[0-9]+$" - r"|^[-+]?\.(?:inf|Inf|INF)$|^\.(?:nan|NaN|NAN)$", - list("-+0123456789.")), -] -_DROP = {t for t, _, _ in _CORE} | {"tag:yaml.org,2002:timestamp"} - -_Loader.yaml_implicit_resolvers = { - ch: [(tag, rx) for tag, rx in rs if tag not in _DROP] - for ch, rs in yaml.SafeLoader.yaml_implicit_resolvers.items() -} -for _tag, _pattern, _first in _CORE: - _Loader.add_implicit_resolver(_tag, re.compile(_pattern), _first) - - -@dataclass -class Node: - id: str - body: str - frontmatter: dict = field(default_factory=dict) - links: list = field(default_factory=list) - results: dict = field(default_factory=dict) - repro: dict = field(default_factory=dict) - sections: list = field(default_factory=list) - backlinks: list = field(default_factory=list) - attachments: list = field(default_factory=list) - tokens: tuple | None = field(default=None, repr=False, compare=False) - - @property - def status(self) -> str: - return str(self.frontmatter.get("status", "")) - - @property - def type(self) -> str: - return str(self.frontmatter.get("type", "")) - - @property - def title(self) -> str: - """The H1 — the claim itself, in one line. This is what makes an index row - judgeable: an id alone cannot be compared against a new idea.""" - return _title(self.body) - - @property - def tags(self) -> list: - """A bare `tags: decoding` is legal YAML that iterates as characters. `validate` - reports it as malformed-tags; here it reads as untagged rather than as five - one-letter tags.""" - raw = self.frontmatter.get("tags") - return [str(x) for x in raw] if isinstance(raw, list) else [] - - def rels(self) -> set: - return {l["rel"] for l in self.links} - - -def _yaml(text: str, label: str) -> dict: - try: - fm = yaml.load(text, Loader=_Loader) - except yaml.YAMLError as e: - where = f" line {e.problem_mark.line + 1}:" if getattr(e, "problem_mark", None) else "" - raise GraphError(f"{label}:{where} invalid YAML — {getattr(e, 'problem', e)}") from e - if fm is None: - return {} - if not isinstance(fm, dict): - raise GraphError(f"{label}: frontmatter must be a mapping, got {type(fm).__name__}") - return fm - - -def split(text: str, label: str) -> tuple[dict, str]: - """(frontmatter, body). Raises GraphError — never returns a half-read node.""" - m = FM_RE.match(text) - if not m: - raise GraphError(f"{label}: no YAML frontmatter (expected a leading `---` block)") - return _yaml(m.group(1), label), m.group(2) - - -def read_frontmatter(path: Path) -> tuple[dict, str]: - return split(path.read_text(encoding="utf-8"), path.name) - - -def parse_text(text: str, nid: str, label: str | None = None) -> Node: - """Build a Node from a string. Used by `knoten_commit` to validate a candidate - node in memory, so an invalid node never reaches the filesystem at all.""" - label = label or f"{nid}.md" - fm, body = split(text, label) - - links = [] - for l in fm.get("links") or []: - if not isinstance(l, dict) or "rel" not in l or "to" not in l: - raise GraphError(f"{label}: every link needs `rel` and `to`, got {l!r}") - links.append({**l, "rel": str(l["rel"]), "to": str(l["to"])}) - - return Node( - id=nid, - body=body, - frontmatter=fm, - links=links, - results=fm.get("results") or {}, - repro=fm.get("repro") or {}, - attachments=[str(a) for a in (fm.get("attachments") or [])], - sections=re.findall(r"^##+ (.+)$", body, re.M), - ) - - -def load(root: Path) -> dict[str, Node]: - nodes = {} - for p in sorted((root / "nodes").glob("*.md")): - if p.stem.upper() == "README": - continue - n = parse_text(p.read_text(encoding="utf-8"), p.stem, p.name) - nodes[n.id] = n - return backlink(nodes) - - -def backlink(nodes: dict[str, Node]) -> dict[str, Node]: - back = defaultdict(list) - for nid, n in nodes.items(): - for l in n.links: - if inv := INVERSE.get(l["rel"]): - back[l["to"]].append({"rel": inv, "to": nid}) - for nid, n in nodes.items(): - n.backlinks = back.get(nid, []) - return nodes - - -# --------------------------------------------------------------------------------- -# Shared by BOTH surfaces. These lived twice — once in cli.py, once in mcp_server.py — -# and drifted: the CLI's path printed relation labels while the MCP's did not, and a -# search fix landed in one copy and not the other. - -def find_root(start: Path | None = None) -> Path: - """The graph containing `start` (default: cwd). A graph is the folder with graph.yaml.""" - p = start or Path.cwd() - for c in [p, *p.parents]: - if (c / "graph.yaml").exists(): - return c - raise GraphError("no graph.yaml found (run `knoten init` or cd into a graph)") - - -def section(body: str, title: str) -> str | None: - """The prose under a `## ` heading, whitespace-collapsed.""" - m = re.search(rf"^##+ {re.escape(title)}\s*\n+(.+?)(?=\n##|\Z)", body, re.S | re.M) - return " ".join(m.group(1).split()) if m else None - - -def _tokens(s: str) -> list[str]: - return [t for t in re.split(r"[^a-z0-9]+", s.lower()) if t] - - -# Function words an agent's question carries and a node never means. Domain words are -# NEVER listed here — a corpus-common word like "accuracy" is handled by idf below, -# which is adaptive; a hardcoded one would be a permanent blind spot. -_STOP = { - "a", "about", "again", "all", "an", "and", "any", "anybody", "anyone", "anything", - "are", "as", "at", "be", "been", "before", "being", "but", "by", "did", "do", "does", - "done", "ever", "for", "from", "had", "has", "have", "how", "i", "if", "in", "into", - "is", "it", "its", "of", "on", "or", "over", "should", "so", "some", "someone", - "something", "than", "that", "the", "then", "these", "this", "those", "to", "was", - "we", "were", "what", "when", "where", "which", "who", "whom", "whose", "why", - "will", "with", "would", "you", -} - -# knoten's own schema words. Their VALUES are searchable; the key names are not, since -# they appear in every node and would make `query "status"` return the whole graph. -_STRUCTURAL = {"id", "type", "status", "tags", "links", "rel", "to", "note", - "results", "repro", "attachments"} - -# id and title carry the claim; tags are a curated label; everything else is context. -_STRONG, _MEDIUM, _WEAK = 3.0, 2.0, 1.0 - - -def _flatten(obj, out: list) -> list: - """Every scalar in the frontmatter, plus the keys a human chose (`tokens_per_question`, - `acc_greedy`). The haystack was id + body + tags, so `repro.model: Qwen3-8B` was - unsearchable and two nodes that ran on the same benchmark answered as one.""" - if isinstance(obj, dict): - for k, v in obj.items(): - if str(k) not in _STRUCTURAL: - out.append(str(k)) - _flatten(v, out) - elif isinstance(obj, (list, tuple)): - for v in obj: - _flatten(v, out) - elif obj is not None: - out.append(str(obj)) - return out - - -def _title(body: str) -> str: - m = re.search(r"^#\s+(.+)$", body, re.M) - return m.group(1) if m else "" - - -def fields(n: Node) -> tuple[set, set, set]: - """(id+title, tags, everything). - - Cached on the node because `retrieve` walks the pool twice — once to count document - frequencies, once to score — and tokenising is the bulk of the work: 33ms vs 150ms - for one query over 500 nodes. The cache does NOT survive between calls; `load` builds - fresh Nodes, so every tool call pays the first pass. - """ - if n.tokens is None: - strong = set(_tokens(f"{n.id} {_title(n.body)}")) - medium = set(_tokens(" ".join(str(t) for t in (n.frontmatter.get("tags") or [])))) - weak = set(_tokens(" ".join(_flatten(n.frontmatter, []) + [n.body]))) | strong | medium - n.tokens = (strong, medium, weak) - return n.tokens - - -def moved(n: Node) -> str: - """When this claim last moved — updated if it has, else created. Not when the FILE - changed: a typo fix and a status flip are the same event to git.""" - fm = n.frontmatter - return max(str(fm.get("updated") or ""), str(fm.get("created") or "")) - - -def _passes(n: Node, tags, status, type, where, since) -> bool: - # An unstamped node predates stamping and cannot answer a question about time. Better - # absent from a `since` view than silently assumed recent. - if since and moved(n) < str(since): - return False - if status and n.status not in status: - return False - if type and n.type not in type: - return False - if tags and not set(n.tags) & set(tags): - return False - # Generic, because the core knows no domain: "everything that died of a weak - # baseline" is one graph's question, and `cause` is one graph's field name. - for key, allowed in (where or {}).items(): - got = n.frontmatter.get(key) - if got is None or str(got) not in {str(a) for a in allowed}: - return False - return True - - -# Keep a hit only if it scores within this fraction of the best hit. Adaptive, so there -# is no absolute threshold to tune per graph: one strong match suppresses the long tail -# of nodes that merely share a common word with the query. -RELATIVE_FLOOR = 0.35 - - -def retrieve(nodes: dict[str, Node], query: str | None = None, tags=None, - status=None, type=None, where=None, since=None) -> list[Node]: - """Rank nodes by relevance to `query`, narrowed by the filters. Ranked, not filtered: - - ANDing every token meant one unmatched word silenced the whole query, so - `"has anyone tried self-consistency?"` — the README's own example — answered "no - prior work found" about a hypothesis the graph was holding, dead and documented. - A false "untested" is the only failure of this tool that costs real work. - - Tokens are weighted by idf, so a word in every node counts for nothing without - anybody having to list it, and a rare one dominates. - - This is the ONE retrieval seam: `query`, `index` and the MCP tools all come through - here, so a semantic backend replaces this body and nothing above it changes. - """ - pool = [n for n in nodes.values() if _passes(n, tags, status, type, where, since)] - if query is None: - return sorted(pool, key=lambda n: n.id) - - qt = [t for t in dict.fromkeys(_tokens(query)) if t not in _STOP] - if not qt or not pool: - return [] - - df = Counter(t for n in pool for t in qt if t in fields(n)[2]) - total = len(pool) - - scored = [] - for n in pool: - strong, medium, weak = fields(n) - s = sum( - (_STRONG if t in strong else _MEDIUM if t in medium else _WEAK) - * math.log(1 + total / (1 + df[t])) - for t in qt if t in weak - ) - if s > 0: - scored.append((s, n)) - if not scored: - return [] - - floor = RELATIVE_FLOOR * max(s for s, _ in scored) - return [n for s, n in sorted(scored, key=lambda p: (-p[0], p[1].id)) if s >= floor] - - -def shortest_path(nodes: dict[str, Node], a: str, b: str) -> list[tuple[str, str]] | None: - """BFS over the graph read as undirected — "how did we get from A to B?" doesn't care - which way an edge points. Returns [(node_id, relation_taken_to_reach_it), ...], the - first with an empty relation. `←rel` means the edge was traversed backwards.""" - for nid in (a, b): - if nid not in nodes: - raise GraphError(f"no node '{nid}'") - - adj = defaultdict(list) - for nid, n in nodes.items(): - for l in n.links: - adj[nid].append((l["to"], l["rel"])) - adj[l["to"]].append((nid, "←" + l["rel"])) - - q, seen = deque([[(a, "")]]), {a} - while q: - p = q.popleft() - if p[-1][0] == b: - return p - for nxt, rel in adj[p[-1][0]]: - if nxt not in seen: - seen.add(nxt) - q.append(p + [(nxt, rel)]) - return None - - -def gates(nodes: dict[str, Node]) -> list[tuple[Node, list, list]]: - """Every gate, with what it killed and what survived it: [(node, killed, survived)]. - - An agent used to meet a gate by being REJECTED by it, once the experiment had already - run. The gates are the reusable asset (SPEC §1), so they belong in front of the work - as a specification. The record is free — the back-links are already generated. - """ - return [(n, - [b["to"] for b in n.backlinks if b["rel"] == "kn:gateKilled"], - [b["to"] for b in n.backlinks if b["rel"] == "kn:gateSurvivedBy"]) - for n in sorted(nodes.values(), key=lambda n: n.id) if n.type == GATE_TYPE] - - -def frontier(nodes: dict[str, Node]) -> dict: - """What is worth doing next, in three buckets. - - `## What would reopen this` is the standing offer SPEC §5 insists on, and until now - the only way to act on one was to re-read every post-mortem and notice the world had - changed. This does not try to decide whether a condition is *met* — that is a - judgement, and encoding it as a predicate would be either trivially wrong or an - ontology project. It presents the offers cheaply and lets the reader judge, the same - bargain `retrieve` makes with an index. - """ - ordered = sorted(nodes.values(), key=lambda n: n.id) - return { - "open": [n for n in ordered if n.status == OPEN], - "reopenable": [(n, offer) for n in ordered - if n.status in ("dead", "retracted") - and (offer := section(n.body, REOPEN_SECTION))], - "untested_gates": [n for n, killed, survived in gates(nodes) - if not killed and not survived], - } diff --git a/build/lib/knoten/hook.py b/build/lib/knoten/hook.py deleted file mode 100644 index 91551e6..0000000 --- a/build/lib/knoten/hook.py +++ /dev/null @@ -1,75 +0,0 @@ -"""The git pre-commit gate. - -`knoten validate` has always printed "commit REJECTED". Nothing rejected a commit — -git wrote the invalid graph to history without complaint, and the phrase was a bluff. - -A rule that only fires when you remember to ask is the same rule that let the previous -attempt (`knowledge_graph.jsonl`, still zero bytes) rot. The gate has to sit in the one -place you cannot forget to walk through. - -We ask GIT where its hooks live rather than assuming `.git/hooks`. That assumption is -wrong in three common cases — `core.hooksPath` (husky, the pre-commit framework, most -monorepos), worktrees and submodules (where `.git` is a FILE, not a directory) — and -being wrong here means writing the hook somewhere git never reads, reporting success, -and silently not gating. Which is precisely the failure this module exists to prevent. -""" -from __future__ import annotations - -import stat -import subprocess -from pathlib import Path - -from .core import GraphError - -MARKER = "# knoten pre-commit gate" - -HOOK = f"""\ -#!/bin/sh -{MARKER} — installed by `knoten hook`. Delete this file to remove it. -# -# A graph you can commit broken is a wiki with extra steps. -# To bypass once (you should have a reason): git commit --no-verify - -if ! command -v knoten >/dev/null 2>&1; then - echo "knoten: not on PATH — activate the environment knoten is installed in," >&2 - echo " or bypass with: git commit --no-verify" >&2 - exit 1 -fi - -cd "$(git rev-parse --show-toplevel)/{{graph}}" || exit 1 -exec knoten validate -""" - - -def _git(root: Path, *args: str) -> str: - try: - r = subprocess.run(["git", "-C", str(root), *args], capture_output=True, text=True) - except FileNotFoundError as e: - raise GraphError("git is not installed") from e - if r.returncode != 0: - raise GraphError(f"{root} is not inside a git repository — run `git init` first") - return r.stdout.strip() - - -def hooks_dir(root: Path) -> Path: - """Where git ACTUALLY reads hooks from — not where we guess it does.""" - p = Path(_git(root, "rev-parse", "--git-path", "hooks")) - return p if p.is_absolute() else (root / p).resolve() - - -def install(root: Path, force: bool = False) -> Path: - repo = Path(_git(root, "rev-parse", "--show-toplevel")) - hooks = hooks_dir(root) - hooks.mkdir(parents=True, exist_ok=True) - hook = hooks / "pre-commit" - - if hook.exists() and MARKER not in hook.read_text(encoding="utf-8") and not force: - raise GraphError( - f"{hook} already exists and knoten did not write it. Refusing to clobber a " - f"hook you wrote. Re-run with --force, or add `knoten validate` to it yourself." - ) - - graph = root.resolve().relative_to(repo.resolve()) - hook.write_text(HOOK.format(graph=graph.as_posix() or "."), encoding="utf-8") - hook.chmod(hook.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH) - return hook diff --git a/build/lib/knoten/mcp_server.py b/build/lib/knoten/mcp_server.py deleted file mode 100644 index fe60fac..0000000 --- a/build/lib/knoten/mcp_server.py +++ /dev/null @@ -1,312 +0,0 @@ -"""MCP server — the surface for clients without a shell. The CLI is the primary agent -surface (see SKILL.md); every tool here delegates to `ops`, `commit` or `update`. - -That used to be reversed: this docstring once called MCP "the reason this project -exists." It loads ~2,340 tokens of schema and instructions into every session whether the -agent touches the graph or not, versus ~304 for `knoten --help` — and only when asked. A -client with a shell should use the CLI. This server exists for the one that can't. - -A knowledge base that depends on someone REMEMBERING to write to it will be empty in six -months. So make the graph reachable from inside the work, not alongside it — that holds -for either surface: - - knoten_index() BEFORE starting -> don't redo dead work - knoten_commit(node) AFTER finishing -> the graph writes itself - -Every tool here is a plain function: its name is the tool name, its docstring is the -description the agent reads, and its annotated signature IS the input schema. There is no -second copy of any of that to drift out of sync, and returning a dict is enough — the SDK -serialises it. - -Run: - knoten-mcp # serves the graph found from $PWD - KNOTEN_GRAPH=/path knoten-mcp # or point it explicitly -""" -from __future__ import annotations - -import functools -import inspect -import os -from pathlib import Path -from typing import Annotated - -try: - from mcp.server import MCPServer - from pydantic import Field -except ImportError as e: # pragma: no cover - try: - from importlib.metadata import version - have = version("mcp") - except Exception: - have = None - # ImportError, NOT SystemExit. SystemExit is a BaseException, so pytest cannot demote - # it to a collection error: on a machine with mcp 1.x, `pytest` aborted the ENTIRE run - # with INTERNALERROR and ran none of the tests that never touch MCP. The friendly exit - # belongs in main(), where a human is reading stderr. - raise ImportError( - f"knoten-mcp needs mcp>=2 (you have {have}). Upgrade with:\n\n" - f" pip install -U 'knoten[mcp]'\n" - if have else - "knoten-mcp needs the `mcp` extra:\n\n pip install 'knoten[mcp]'\n" - ) from e - -from . import attachments, ops -from .commit import commit as commit_node -from .core import ID_RE, GraphError, find_root - -INSTRUCTIONS = """\ -knoten is a research graph that remembers what did NOT work. Each node is a claim; a dead -claim carries why it died and what would bring it back. - -The loop, in order: - - 1. knoten_frontier — what is worth doing next: work left open, dead ends whose stated - reopen condition may now hold, and gates nothing has been through. A dead end with a - standing offer is a cheaper experiment than a new idea; the design is already written. - 2. knoten_index / knoten_query — has this been tried? `index` lists the whole graph one - line per node so you can spot work done in DIFFERENT WORDS; `query` is keyword search, - faster when your idea has a distinctive name and blind to paraphrase. - 3. knoten_get — the full node for anything that looks close: post-mortem, results, and - the path of the script that produced them. - 4. knoten_gates — what a result must survive here. Read it BEFORE designing the - experiment: a claim cannot be filed as alive without citing a gate it survived, so - meeting the gate at commit time means the compute is already spent. - 5. knoten_commit — file the claim when the work concludes, INCLUDING when it fails. Use - knoten_update instead if you opened the node earlier and are now closing it. - 6. knoten_attach — the script that ran it and the plot that shows it. A claim nobody can - re-run is a claim nobody trusts in six months. - -knoten_path answers "how did we get from A to B?"; knoten_validate runs the graph's rules. - -Writes are gated: commit and update validate against the graph's own declared rules and -refuse on violation. The refusal is the feature. Fix the node and call again. -""" - -app = MCPServer("knoten", instructions=INSTRUCTIONS) - - -def tool(fn): - """Register a function as a knoten tool. - - An MCP server must ANSWER, never explode: a broken graph.yaml, or a directory passed - to knoten_attach, has to come back as JSON the agent can act on rather than a - traceback it can only give up on. - - `cleandoc` because the docstring becomes the description verbatim, and an agent should - not have to read our source indentation. - """ - @functools.wraps(fn) - def guarded(*args, **kwargs): - try: - return fn(*args, **kwargs) - except GraphError as e: - return {"error": str(e)} - except OSError as e: - return {"error": f"{type(e).__name__}: {e}"} - - guarded.__doc__ = inspect.cleandoc(fn.__doc__ or "") - app.tool()(guarded) - return guarded - - -# Argument types shared by more than one tool. Declared once, described once. -NodeId = Annotated[str, Field(description="node id, kebab-case, e.g. hyp-self-consistency")] -Tags = Annotated[list[str] | None, Field(description="keep nodes carrying any of these tags")] -Statuses = Annotated[list[str] | None, Field(description="keep nodes with any of these statuses")] -Types = Annotated[list[str] | None, Field(description="keep nodes of any of these types")] -Where = Annotated[dict | None, Field( - description='keep nodes whose frontmatter field is one of these values, ' - 'e.g. {"cause": ["weak_baseline"]}')] -Since = Annotated[str | None, Field( - description="YYYY-MM-DD — only nodes created or updated on/after this day")] - - -def _root() -> Path: - if env := os.environ.get("KNOTEN_GRAPH"): - return Path(env).expanduser() - return find_root() - - -# ------------------------------------------------------------ 1. what to work on - - -@tool -def knoten_frontier() -> dict: - """Use when choosing what to work on next. Three buckets: claims left open, dead - claims that stated what would reopen them, and gates no claim has ever been through. - A dead end with a standing offer is a cheaper experiment than a new idea, because the - design is already written down. - """ - return ops.frontier(_root()) - - -# ------------------------------------------------------------ 2. has this been tried - - -@tool -def knoten_index( - query: Annotated[str | None, - Field(description="optional — rank the rows by relevance to this")] = None, - tags: Tags = None, - status: Statuses = None, - type: Types = None, - where: Where = None, - since: Since = None, - limit: Annotated[int | None, Field(description=f"default {ops.INDEX_LIMIT}")] = None, -) -> dict: - """START HERE when you have an idea and need to know whether it is new. The whole - graph, one line per node: id, type, verdict, tags, claim. Read the lines and judge - relatedness yourself — this is the only way to find work already done in DIFFERENT - WORDS, which keyword search cannot do. Also answers 'what is still open?' - (status=['open']). Narrow a large graph with tags / status / type / since before - reading it. - """ - return ops.index(_root(), query=query, tags=tags, status=status, type=type, - where=where, since=since, limit=limit) - - -@tool -def knoten_query(query: Annotated[str, Field(description="term, tag, or topic")]) -> dict: - """Keyword search over the graph, ranked by relevance. Quicker than knoten_index when - your idea has a distinctive name. It matches WORDS, NOT MEANING: a node phrased - differently will not appear, so an empty result means 'no keyword match' and NOT - 'never tried' — confirm with knoten_index before concluding an idea is new. Returns - verdicts (ALIVE/DEAD/RETRACTED), the gates that killed or validated each claim, why it - died, and what would reopen it. - """ - return ops.query(_root(), query) - - -# ------------------------------------------------------------ 3. read one node - - -@tool -def knoten_get(id: NodeId) -> dict: - """Fetch one node in full: post-mortem, reproduction recipe, and the paths of any - attached scripts or plots (read them with your file tools to re-run or inspect the - experiment). - """ - return ops.get(_root(), id) - - -# ------------------------------------------------------------ 4. what must it survive - - -@tool -def knoten_gates() -> dict: - """Read this BEFORE you design an experiment. The methodological gates every claim - here is held to: what to run, why the check exists, and what each has killed or - validated. A claim cannot be filed as alive without citing a gate it survived, so - meeting a gate at commit time means the compute is already spent on a result that - cannot be recorded. - """ - return ops.gates(_root()) - - -# ------------------------------------------------------------ 5. record the outcome - - -@tool -def knoten_commit( - id: NodeId, - frontmatter: Annotated[str, Field( - description="YAML frontmatter body (no --- fences). Must include type and status. " - "Alive claims need a kn:survivedGate link.")], - body: Annotated[str, Field( - description="Markdown. Dead/retracted nodes MUST contain '## Why it died' and " - "'## What would reopen this'.")], -) -> dict: - """Add a node to the graph. VALIDATES FIRST and REFUSES on rule violation — e.g. a - claim marked 'alive' that cites no gate it survived. Call this when an investigation - concludes, INCLUDING when it fails. A dead hypothesis with a documented cause of death - is the most valuable node in the graph. Also reports settled claims the new node - resembles, so the same question is not filed twice. - """ - return commit_node(_root(), id, frontmatter, body) - - -@tool -def knoten_update( - id: NodeId, - status: Annotated[str | None, Field( - description="the new status, e.g. dead. Must be one the graph declares.")] = None, - append: Annotated[str | None, Field( - description="markdown appended to the body — this is where '## Why it died' and " - "'## What would reopen this' go.")] = None, - results: Annotated[dict | None, Field( - description="result keys to add. An existing key cannot be changed to a different " - "value.")] = None, - links: Annotated[list[dict] | None, Field( - description="edges to add, e.g. [{rel: kn:killedByGate, to: method-x}]")] = None, - fields: Annotated[dict | None, Field( - description="top-level frontmatter fields to set, e.g. {\"cause\": " - "\"weak_baseline\"}. Sets any key, including one already " - "recorded; the graph's own rules decide what is accepted.")] = None, -) -> dict: - """Move a node through its lifecycle and append to it: open -> alive / dead / - retracted. CALL THIS WHEN AN EXPERIMENT YOU OPENED FINISHES, especially when it fails - — a hypothesis left 'open' forever is a ghost on every future frontier. It cannot - rewrite prose, nor overwrite a result already recorded (retract or supersede the node for that), but - `fields` sets any top-level key. VALIDATES FIRST and REFUSES on rule violation, same as - knoten_commit. - """ - return ops.update(_root(), id, status=status, results=results, - links=links, append=append, fields=fields) - - -# ------------------------------------------------------------ 6. leave the evidence - - -@tool -def knoten_attach( - id: NodeId, - files: Annotated[list[str], Field(description="paths to the files to attach")], -) -> dict: - """Attach files you produced to a node — the script that ran the experiment, the plot - that shows the result. Call this after knoten_commit. A claim you cannot re-run is a - claim nobody will trust in six months; the attachment IS the reproduction. Images are - embedded in the node body so they render on GitHub. Write the file to disk first, then - pass its path. - """ - if not ID_RE.match(id or ""): - return {"status": "REJECTED", "node": id, - "reason": "invalid id — use kebab-case, e.g. hyp-self-consistency."} - try: - res = attachments.attach(_root(), id, files) - except (GraphError, OSError) as e: - return {"status": "REJECTED", "node": id, "reason": str(e)} - return {"status": "ATTACHED", "node": id, - "attached": [f"attachments/{id}/{n}" for n in res.added], - "embedded": res.embedded, "warnings": res.warnings, - "next": "git add + commit to version this."} - - -# ------------------------------------------------------------ anytime - - -@tool -def knoten_path( - start: Annotated[str, Field(description="node id to start from")], - end: Annotated[str, Field(description="node id to reach")], -) -> dict: - """Show the research path between two nodes — how did we get from A to B?""" - return ops.path(_root(), start, end) - - -@tool -def knoten_validate() -> dict: - """Run the graph's own declared rules over every node.""" - return ops.validate(_root()) - - -def main() -> None: - """The console-script entry point. Import errors reach a human here, as a clean - message on stderr rather than a traceback — see the ImportError above for why the - module itself must not exit.""" - import asyncio - - asyncio.run(app.run_stdio_async()) - - -if __name__ == "__main__": - main() diff --git a/build/lib/knoten/ops.py b/build/lib/knoten/ops.py deleted file mode 100644 index 9fd4afa..0000000 --- a/build/lib/knoten/ops.py +++ /dev/null @@ -1,190 +0,0 @@ -"""Every question the graph answers, as a dict. - -One implementation per operation. The CLI renders these dicts as prose, `--json` dumps -them, and the MCP tools serialise them — so the three surfaces cannot drift, which they -did repeatedly when the read paths were written twice. -""" -from __future__ import annotations - -from pathlib import Path - -from .core import (GATE_SECTIONS, VERDICT, GraphError, Node, frontier as _frontier, - gates as _gates, load, retrieve, section, shortest_path) -from .update import update as _update_node -from .validate import check, load_config - -# A row is ~45 tokens once JSON key overhead is counted, so 200 rows is ~9k — readable -# in one go, where the same graph's broad query was 83k. It is a CAP, never a silent one: -# `truncated` and `total` always say what was left out. -INDEX_LIMIT = 200 - -# A query returns FULL summaries (post-mortem, results, repro) — a few hundred tokens -# each. Twenty is a read; sixty is a context flood that buries the top hit. -QUERY_LIMIT = 20 - - -def summarise(n: Node) -> dict: - out = {"id": n.id, "type": n.type, "verdict": VERDICT.get(n.status, n.status or "-")} - for rel, key in [("kn:killedByGate", "killed_by"), ("kn:survivedGate", "survived_gates"), - ("npx:retracts", "retracts"), ("kn:blockedBy", "blocked_by")]: - if ts := [l["to"] for l in n.links if l["rel"] == rel]: - out[key] = ts - # A claim someone later WITHDREW is invisible unless we say so: we reported what a node - # retracts, never that it WAS retracted. - for rel, key in [("npx:retractedBy", "retracted_by"), ("npx:supersededBy", "superseded_by")]: - if ts := [b["to"] for b in n.backlinks if b["rel"] == rel]: - out[key] = ts - out["warning"] = (f"This claim was {key.replace('_', ' ')} {', '.join(ts)}. " - f"Read that node before relying on this one.") - if why := section(n.body, "Why it died"): - out["why_it_died"] = why[:400] - if reopen := section(n.body, "What would reopen this"): - out["what_would_reopen_this"] = reopen[:400] - if n.results: - out["results"] = n.results - if n.repro: - out["repro"] = n.repro - if n.attachments: - out["attachments"] = [f"attachments/{n.id}/{a}" for a in n.attachments] - return out - - -def frontier(root: Path) -> dict: - f = _frontier(load(root)) - return { - "open": [{"id": n.id, "title": n.title} for n in f["open"]], - "reopenable": [{"id": n.id, "title": n.title, "reopen_if": offer} - for n, offer in f["reopenable"]], - "untested_gates": [{"id": n.id, "title": n.title} for n in f["untested_gates"]], - "note": ("A reopenable claim states its own condition. Judge whether it holds now " - "— knoten does not, because that is the research."), - } - - -def index(root: Path, query=None, tags=None, status=None, type=None, - where=None, since=None, limit=None) -> dict: - nodes = load(root) - hits = retrieve(nodes, query, tags=tags, status=status, type=type, - where=where, since=since) - cap = max(1, int(limit or INDEX_LIMIT)) - out = { - "total": len(hits), - "truncated": len(hits) > cap, - "nodes": [{"id": n.id, "type": n.type, - "verdict": VERDICT.get(n.status, n.status or "-"), - "tags": n.tags, "title": n.title} for n in hits[:cap]], - "declared_tags": [str(t) for t in (load_config(root).get("tags") or [])], - } - if out["truncated"]: - # A silent cap reads as "that is the whole graph" — the same false negative the - # AND-query bug produced, arriving by a different route. - out["note"] = (f"Showing {cap} of {len(hits)}. Narrow with tags/status/type, or " - f"pass a query to rank by relevance, before concluding anything " - f"about what is NOT here.") - return out - - -def query(root: Path, term: str) -> dict: - hits = retrieve(load(root), term) - claims = [n for n in hits if n.status in VERDICT] # already relevance-ranked - out = {"query": term, "total": len(claims), - "truncated": len(claims) > QUERY_LIMIT, - "claims": [summarise(n) for n in claims[:QUERY_LIMIT]], - # MCP contract — do not narrow, an agent tool call is keyed on this shape. - "related_methods": [n.id for n in hits if n.type == "method"], - # Everything else that matched but isn't a claim: sources, open work, whatever - # types this graph declares. The CLI's "also:" line used to show these before - # it was rewired onto this dict; losing them was silent narrowing, not a fix. - "related": [n.id for n in hits if n.status not in VERDICT]} - if out["truncated"]: - out["note"] = (f"Showing the {QUERY_LIMIT} closest of {len(claims)} matching " - f"claims. List the whole graph, one line per node, and read it " - f"yourself for the full picture.") - elif claims: - out["note"] = ("Claims marked DEAD or RETRACTED have already been tested. Read " - "'what_would_reopen_this' before re-running them.") - else: - # Keyword search cannot find an idea phrased in words the node never used. Saying - # "untested" here without that caveat is how an agent re-runs a dead experiment — - # the exact failure this tool exists to prevent. - out["note"] = ("No keyword match. This is NOT proof the idea is untested — a " - "differently-worded node will not match. List the whole graph, " - "one line per node, and read the claims yourself before " - "concluding it is new.") - return out - - -def get(root: Path, nid: str) -> dict: - nodes = load(root) - if not (n := nodes.get(nid)): - return {"error": f"no node '{nid}'", "available": sorted(nodes)} - out = {**summarise(n), "frontmatter": n.frontmatter, - "links": n.links, "backlinks": n.backlinks, "body": n.body} - if n.attachments: - # A path string alone can't tell a reader whether the file is still there — - # `show` used to print size / MISSING from the filesystem directly; additive - # here so both surfaces (not just the CLI) can see it. - details = [] - for a in n.attachments: - p = root / "attachments" / nid / a - row = {"path": f"attachments/{nid}/{a}"} - if p.exists(): - row["size_kb"] = round(p.stat().st_size / 1024, 1) - else: - row["missing"] = True - details.append(row) - out["attachment_files"] = details - return out - - -def gates(root: Path) -> dict: - rule, why = GATE_SECTIONS - out = [] - for n, killed, survived in _gates(load(root)): - row = {"id": n.id, "title": n.title, "killed": killed, "survived": survived} - if r := section(n.body, rule): - row["rule"] = r - if w := section(n.body, why): - row["why_it_exists"] = w - out.append(row) - return {"gates": out, - "note": ("Design the experiment to pass these. A gate with nothing in " - "`killed` or `survived` has never been applied.")} - - -def validate(root: Path) -> dict: - nodes = load(root) - errs = check(nodes, root) - return {"nodes": len(nodes), "valid": not errs, - "violations": [{"node": e.node, "rule": e.rule, "message": e.message} - for e in errs]} - - -def path(root: Path, start: str, end: str) -> dict: - p = shortest_path(load(root), start, end) - if p is None: - return {"path": None, "note": f"no path {start} -> {end}"} - # WITH the relation on each hop. Without it an agent learns two nodes are connected - # but not HOW, which is useless for reasoning about falsification. - return {"path": [{"node": nid, "via": rel} if rel else {"node": nid} - for nid, rel in p], - "hops": len(p) - 1} - - -def update(root: Path, nid: str, status: str | None = None, results: dict | None = None, - links: list | None = None, append: str | None = None, - fields: dict | None = None) -> dict: - """Move a node through its lifecycle, or report why it was refused — ONE shape for - both outcomes. Update used to be built twice: the CLI's success omitted `status` - entirely and its refusal had no `hint`, while MCP's success carried `status: - "UPDATED"` and its refusal did. That drift is the exact bug this module exists to - prevent, so this mirrors `commit()`'s convention instead of inventing a third shape. - """ - try: - now = _update_node(root, nid, status=status, results=results, - links=links, append=append, fields=fields) - except GraphError as e: - return {"status": "REJECTED", "node": nid, "reason": str(e), - "hint": "Fix it and update again. The gate is the point."} - return {"status": "UPDATED", "node": nid, "node_status": now, - "next": "git add + commit to version this."} diff --git a/build/lib/knoten/update.py b/build/lib/knoten/update.py deleted file mode 100644 index 654d5f7..0000000 --- a/build/lib/knoten/update.py +++ /dev/null @@ -1,123 +0,0 @@ -"""Moving a node through its own lifecycle: open -> alive / dead / retracted. - -`knoten_commit` refuses to overwrite a node, which is right — a correction to a claim is -a new node, not an edit. But that left the lifecycle SPEC §3 draws with no way to walk it: -an agent could open a hypothesis and never close it. Its only outs were writing the file -directly, which bypasses every gate, or a second node leaving the first `open` forever. - -What bounds an edit is the graph's own declared rules, not a list of things this module -refuses: the amended candidate goes through the same in-memory validation `knoten_commit` -uses, and never reaches disk if it fails. `fields` therefore sets any top-level key, -including one already recorded — `results` is the exception, guarded because a number you -already published is a different kind of claim from a label. Git holds the before and -after; that is what living in git buys. -""" -from __future__ import annotations - -import re -from pathlib import Path - -import yaml - -from .core import (FM_RE, GraphError, backlink, graph_lock, load, node_path, parse_text, - split, today, write_atomic) -from .validate import check - -# A key we re-emit; everything else keeps its original text, comments included. -_BLOCK = re.compile(r"^(\w[\w-]*):", re.M) - - -def _spans(fm: str) -> dict[str, tuple[int, int]]: - """Line span of each top-level key. Anything not re-emitted is copied verbatim, which - is how a hand-written comment survives an update.""" - lines = fm.splitlines() - starts = [(i, m.group(1)) for i, l in enumerate(lines) if (m := _BLOCK.match(l))] - out = {} - for n, (i, key) in enumerate(starts): - out[key] = (i, starts[n + 1][0] if n + 1 < len(starts) else len(lines)) - return out - - -def _dump(key: str, value) -> list[str]: - text = yaml.safe_dump({key: value}, sort_keys=False, default_flow_style=False, - allow_unicode=True).rstrip("\n") - return text.splitlines() - - -def _rewrite(fm: str, changed: dict) -> str: - """Replace only the keys we changed. Order is preserved; a key that is new goes last.""" - lines, spans = fm.splitlines(), _spans(fm) - for key, value in changed.items(): - if key in spans: - i, j = spans[key] - lines[i:j] = _dump(key, value) - spans = _spans("\n".join(lines)) # spans shift under us - else: - lines += _dump(key, value) - return "\n".join(lines) - - -def update(root: Path, nid: str, status: str | None = None, results: dict | None = None, - links: list | None = None, append: str | None = None, - fields: dict | None = None) -> str: - """Append to a node, move its status, set its fields. Raises GraphError, having - written nothing. - - Returns the status the node now carries. - """ - with graph_lock(root): - return _update(root, nid, status=status, results=results, links=links, - append=append, fields=fields) - - -def _update(root: Path, nid: str, status, results, links, append, fields) -> str: - nf = node_path(root, nid) # rejects a traversal before it is a path - if not nf.exists(): - raise GraphError(f"no node '{nid}'") - if not any([status, results, links, append, fields]): - raise GraphError(f"'{nid}': nothing to change — pass status, results, links, " - f"fields or append.") - - text = nf.read_text(encoding="utf-8") - fm_text, body = FM_RE.match(text).groups() - fm, _ = split(text, nf.name) - - changed = {"updated": today()} - if status: - changed["status"] = status - if links: - changed["links"] = (fm.get("links") or []) + list(links) - if results: - have = fm.get("results") or {} - # A number that was already recorded is part of the claim. Changing it is not a - # lifecycle move, it is a rewrite of the record — that is what retraction is for. - if clash := [k for k, v in results.items() if k in have and have[k] != v]: - raise GraphError( - f"'{nid}': {', '.join(sorted(clash))} already recorded with a different " - f"value. Retract or supersede the node rather than rewriting a result.") - changed["results"] = {**have, **results} - - if fields: - # No allow-list and no immutability guard: `fields` sets any top-level key to any - # value, including one already recorded. What stops a broken node is the same - # thing that stops one from `commit` — the whole candidate is parsed and run - # through the graph's own rules below, and never reaches disk if it fails. - # - # Applied LAST, so `fields={"status": ...}` beats the `status` argument and - # `fields={"updated": ...}` beats the stamp this call just computed. Deliberate: - # "sets any top-level key" would be a lie if another argument could quietly win. - changed.update(fields) - - out = f"---\n{_rewrite(fm_text, changed)}\n---\n{body}" - if append: - out = out.rstrip("\n") + "\n\n" + append.strip("\n") + "\n" - - # Validate the candidate in memory, exactly as knoten_commit does: an invalid node - # never reaches the filesystem, and a refused update leaves the file untouched. - candidate = parse_text(out, nid, nf.name) - nodes = backlink({**load(root), nid: candidate}) - if errs := [e for e in check(nodes, root) if e.node == nid]: - raise GraphError("; ".join(f"[{e.rule}] {e.message}" for e in errs)) - - write_atomic(nf, out) - return candidate.status diff --git a/build/lib/knoten/validate.py b/build/lib/knoten/validate.py deleted file mode 100644 index 29377a6..0000000 --- a/build/lib/knoten/validate.py +++ /dev/null @@ -1,272 +0,0 @@ -"""The rules engine. - -The core knows NOTHING about any domain — every rule comes from the graph's own -`graph.yaml`, so a trading graph and a biology graph share this code unchanged. - -A rule this engine cannot understand is a hard error, never a no-op: a rule that -silently enforces nothing is worse than no rule, because you believe you are covered. -""" -from __future__ import annotations - -from dataclasses import dataclass -from pathlib import Path - -from .core import GENERATED, INVERSE, GraphError, Node, _yaml - -# A rule key that is not in here is a typo. Refuse it. -RULE_KEYS = { - "id", # required - "message", # what the human sees when it fires - "when_status", # only apply to these statuses - "when_type", # only apply to these node types - "require_edge", # node must declare this relation - "require_sections", # body must contain these `## ` headings - "require_result", # results must carry this key - "require_result_min", # {key: minimum} — numeric floor - "require_field_one_of", # {field: [allowed]} — closed vocabulary -} - -# Same for the top level. `node_type:` (singular) would be the next silent no-op. -GRAPH_KEYS = {"name", "description", "node_types", "statuses", "tags", "rules"} - - -@dataclass -class Violation: - node: str - rule: str - message: str - - -def load_config(root: Path) -> dict: - """The graph's own declaration. Every key here is enforced; a key knoten does not - understand is a hard error, because config that enforces nothing is decoration.""" - f = root / "graph.yaml" - if not f.exists(): - return {} - cfg = _yaml(f.read_text(encoding="utf-8"), "graph.yaml") - - if unknown := set(cfg) - GRAPH_KEYS: - raise GraphError( - f"graph.yaml: unknown key(s) {', '.join(sorted(unknown))}. " - f"Known keys: {', '.join(sorted(GRAPH_KEYS))}" - ) - for key in ("node_types", "statuses", "tags"): - if key in cfg and not isinstance(cfg[key], list): - raise GraphError(f"graph.yaml: `{key}` must be a list, got {cfg[key]!r}") - - rules = cfg.get("rules") or [] - if not isinstance(rules, list): - raise GraphError("graph.yaml: `rules` must be a list") - for r in rules: - if not isinstance(r, dict): - raise GraphError(f"graph.yaml: each rule must be a mapping, got {r!r}") - if "id" not in r: - raise GraphError(f"graph.yaml: rule is missing `id`: {r!r}") - if unknown := set(r) - RULE_KEYS: - raise GraphError( - f"graph.yaml: rule '{r['id']}' has unknown key(s) " - f"{', '.join(sorted(unknown))}. A rule key knoten does not understand " - f"would enforce nothing. Known keys: {', '.join(sorted(RULE_KEYS))}" - ) - _check_values(r) - cfg["rules"] = rules - return cfg - - -def load_rules(root: Path) -> list[dict]: - return load_config(root).get("rules", []) - - -def _check_values(r: dict) -> None: - """Key names are not enough: a wrong-SHAPED value must also be a hard error, not a - TypeError. `require_edge: [x]` is the natural mistake, since `when_status` takes a list.""" - rid = r["id"] - for key in ("require_edge", "require_result"): - if key in r and not isinstance(r[key], str): - raise GraphError( - f"graph.yaml: rule '{rid}': `{key}` must be a single string, " - f"got {r[key]!r}") - - if "require_field_one_of" in r: - fields = r["require_field_one_of"] - if not isinstance(fields, dict): - raise GraphError( - f"graph.yaml: rule '{rid}': `require_field_one_of` must be a mapping of " - f"{{field: [allowed, values]}}, got {fields!r}") - for k, v in fields.items(): - if not isinstance(v, list) or not v: - raise GraphError( - f"graph.yaml: rule '{rid}': `require_field_one_of` for '{k}' must be " - f"a non-empty list of allowed values, got {v!r}") - - if "require_result_min" in r: - floors = r["require_result_min"] - if not isinstance(floors, dict): - raise GraphError( - f"graph.yaml: rule '{rid}': `require_result_min` must be a mapping of " - f"{{result_key: number}}, got {floors!r}") - for k, v in floors.items(): - if isinstance(v, bool) or not isinstance(v, (int, float)): - raise GraphError( - f"graph.yaml: rule '{rid}': `require_result_min` floor for '{k}' must " - f"be a number, got {v!r}") - - -def _tags(n: Node, cfg: dict) -> list[Violation]: - """Tags are the filter axis: they narrow a graph too big to read into a slice an - agent can take in one call. A typo'd tag is therefore not cosmetic — the node is - still in the graph but outside every filtered view of it, which is the same silent - disappearance as a typo'd status.""" - raw = n.frontmatter.get("tags") - if raw is None: - return [] - if not isinstance(raw, list): - return [Violation(n.id, "malformed-tags", - f"`tags` must be a list, got {type(raw).__name__} " - f"({raw!r}). Write `tags: [{raw}]`.")] - if not (declared := cfg.get("tags")): - return [] - known = {str(t) for t in declared} - return [Violation(n.id, "unknown-tag", - f"tag '{t}' is not declared in graph.yaml " - f"(tags: {', '.join(map(str, declared))})") - for t in map(str, raw) if t not in known] - - -def _blocks(n: Node) -> list[Violation]: - """`results:` and `repro:` must be mappings. - - `results: 5` used to reach `n.results.get(key)` in the require_result_min loop and come - back as `AttributeError: 'int' object has no attribute 'get'` — a traceback out of the - validator whose whole job is refusing bad nodes politely. `parse_text` keeps whatever - the YAML held, so the check belongs here rather than in the parser: a scalar is a - malformed node, not an unparseable one. - """ - return [Violation(n.id, f"malformed-{name}", - f"`{name}` must be a mapping of key: value, got " - f"{type(raw).__name__} ({raw!r})") - for name in ("results", "repro") - if (raw := n.frontmatter.get(name)) is not None and not isinstance(raw, dict)] - - -def _vocabulary(n: Node, cfg: dict) -> list[Violation]: - """A node's `type` and `status` must be words THIS graph declared. - - The core invents no vocabulary: declare no `node_types` and none is checked. But a - graph that DOES declare one has said those are the only legal words — and a claim with - a typo'd (or missing) status silently drops out of every query, which filters on the - known set. - """ - out = [] - if not n.type: - out.append(Violation(n.id, "missing-type", "node declares no `type`")) - elif (types := cfg.get("node_types")) and n.type not in types: - out.append(Violation(n.id, "unknown-type", - f"type '{n.type}' is not declared in graph.yaml " - f"(node_types: {', '.join(map(str, types))})")) - - out += _tags(n, cfg) - - if statuses := cfg.get("statuses"): - if not n.status: - out.append(Violation(n.id, "missing-status", - f"node declares no `status`, so it escapes every " - f"when_status rule and never appears in a query " - f"(statuses: {', '.join(map(str, statuses))})")) - elif n.status not in statuses: - out.append(Violation(n.id, "unknown-status", - f"status '{n.status}' is not declared in graph.yaml " - f"(statuses: {', '.join(map(str, statuses))})")) - return out - - -def _structural(nodes: dict[str, Node], root: Path, cfg: dict) -> list[Violation]: - """Checks the core ALWAYS runs. Structural, not domain.""" - out = [] - ids = set(nodes) - for nid, n in nodes.items(): - # The real id is the filename; `id:` in the frontmatter is decorative. A node - # whose `id:` says something else lies about itself to every human reading it - # while every query still resolves it by its filename. - if (declared := n.frontmatter.get("id")) and str(declared) != nid: - out.append(Violation(nid, "mismatched-id", - f"frontmatter says id '{declared}' but the file is " - f"{nid}.md — the filename is the id")) - out += _blocks(n) + _vocabulary(n, cfg) - for l in n.links: - rel = l["rel"] - if rel in GENERATED: - out.append(Violation(nid, "authored-backlink", - f"'{rel}' is a generated back-link — declare the " - f"forward edge on the other node instead")) - elif rel not in INVERSE: - out.append(Violation(nid, "unknown-relation", - f"'{rel}' is not a known relation. It creates no " - f"back-link, so the node is invisible from the other " - f"side. Known: {', '.join(sorted(INVERSE))}")) - if l["to"] not in ids: - out.append(Violation(nid, "dangling-edge", - f"-> {l['to']} ({rel}) does not exist")) - for a in n.attachments: - if not (root / "attachments" / nid / a).exists(): - out.append(Violation(nid, "missing-attachment", - f"'{a}' is listed but not in attachments/{nid}/")) - return out - - -def _csv(v) -> list[str]: - if not v: - return [] - if isinstance(v, list): - return [str(x).strip() for x in v if str(x).strip()] - return [x.strip() for x in str(v).split(",") if x.strip()] - - -def applies(status: str, ntype: str, r: dict) -> bool: - """Does this rule apply to a node with this status/type? Shared with `knoten new`, so - the scaffold and the validator cannot disagree about which rules are in play.""" - if (st := _csv(r.get("when_status"))) and status not in st: - return False - if (ty := _csv(r.get("when_type"))) and ntype not in ty: - return False - return True - - -def check(nodes: dict[str, Node], root: Path) -> list[Violation]: - cfg = load_config(root) - out = _structural(nodes, root, cfg) - - for n in nodes.values(): - for r in cfg.get("rules", []): - if not applies(n.status, n.type, r): - continue - rid, msg = r["id"], str(r.get("message", r["id"])).strip() - - if (rel := r.get("require_edge")) and rel not in n.rels(): - out.append(Violation(n.id, rid, msg)) - - for sec in _csv(r.get("require_sections")): - if not any(sec.lower() in s.lower() for s in n.sections): - out.append(Violation(n.id, rid, f"{msg} (missing '## {sec}')")) - - if (fld := r.get("require_result")) and fld not in n.results: - out.append(Violation(n.id, rid, msg)) - - for fld, allowed in (r.get("require_field_one_of") or {}).items(): - got = n.frontmatter.get(fld) - if got is None or str(got) not in {str(a) for a in allowed}: - out.append(Violation(n.id, rid, - f"{msg} ({fld}={got!r}, one of: " - f"{', '.join(map(str, allowed))})")) - - # `_blocks` already reported a non-mapping; skip rather than crash on it - # in the same pass. - for key, floor in ((r.get("require_result_min") or {}).items() - if isinstance(n.results, dict) else []): - got = n.results.get(key) - # `bool` is a subclass of `int`: `accuracy: true` would otherwise sail - # through a floor of 0.8 as the number 1. - numeric = isinstance(got, (int, float)) and not isinstance(got, bool) - if not numeric or got < floor: - out.append(Violation(n.id, rid, f"{msg} ({key}={got!r}, need >= {floor})")) - return out diff --git a/build/lib/knoten/viz.html b/build/lib/knoten/viz.html deleted file mode 100644 index cce8fe1..0000000 --- a/build/lib/knoten/viz.html +++ /dev/null @@ -1,581 +0,0 @@ -<!doctype html><meta charset="utf-8"><title>knoten - - - -
-
- -

knoten

-
- - -
-
-
nodes
-
-
-
-
-
-
-
-
-
-
- legend -
- judged — a gate ruled on it
-
- related — everything else
-
-
drag to pan · wheel to zoom · f fit · v switch · esc clear
-
- -
-
- - diff --git a/build/lib/knoten/viz.py b/build/lib/knoten/viz.py deleted file mode 100644 index 4924eae..0000000 --- a/build/lib/knoten/viz.py +++ /dev/null @@ -1,255 +0,0 @@ -"""One HTML file: the graph as columns, and the graph as a map. - -Read-only, self-contained, no server and no build step. The payload is inlined, so the -file opens from `file://`, from a share, from a plane. - -Two views because there are two questions. **Columns** is the inventory — what exists, in -what role, with what verdict — laid out left to right along the loop a graph declares. -**Map** is the traversal — what a claim rests on and what else that touched — laid out -around the busiest nodes, because a graph's landmarks are wherever its edges converge and -not wherever its vocabulary says they should be. - -Layout is a pure function of the graph. Nothing is persisted: positions are derived state, -and a `layout.json` in git would be a merge conflict generator with ten agents appending. -""" -import json -import math -from pathlib import Path - -from .core import GATE_TYPE, GraphError, load, section -from .validate import load_config - -HERE = Path(__file__).parent - -# Columns: no vertical centring. Centring would make every column shift when any one of -# them grows, which is the reflow this whole design exists to avoid. -COLW, CARDH, GAP, TOP = 300, 74, 12, 58 - -# Map: golden-angle sunflower. Uniform density, and index k always lands in the same -# place, so appending a node never disturbs 1..k-1. -GOLDEN = math.pi * (3 - math.sqrt(5)) -SPACING = 33 -# Clusters sit on the same spiral, at a fixed spacing. Dividing a ring by the cluster -# COUNT would rotate every cluster the moment a new one appeared — one isolated node -# appended, and the whole map turns. A constant spacing lets a very large cluster graze -# its neighbour; that is the cheaper failure, and it is local. -CLUSTER = SPACING * 7.5 - -GATE_RELS = ("kn:survivedGate", "kn:killedByGate") - -# Only an ordering hint. A type this does not name is not dropped — it lands after the -# ones that are, in the order the graph first used it. knoten declares no vocabulary. -FLOW = ["source", "idea", "question", "hypothesis", "experiment", "finding", - "blocker", "retraction"] - - -# A node with no `created` is almost always one written by hand or by another tool — -# `knoten new` and `knoten commit` both stamp it. Sorting it EMPTY-STRING-FIRST put it in -# slot 0 and pushed every existing node along, which is the one thing this layout promises -# not to do. Undated work sorts last, with the newest, where an unknown arrival belongs. -UNDATED = "9999" - - -def _order(nodes: dict) -> list: - """Oldest first, id breaking ties. `created` is what makes appending safe: new work - sorts last, so it can only ever be added to the end of a column or the rim of a - cluster.""" - return sorted(nodes.values(), - key=lambda n: (str(n.frontmatter.get("created") or UNDATED), n.id)) - - -def _sunflower(k: int) -> tuple: - r = SPACING * math.sqrt(k) - return r * math.cos(k * GOLDEN), r * math.sin(k * GOLDEN) - - -def roles(nodes: dict) -> tuple: - """Which types are gates, which are shelves, and what order the rest go in. - - Derived from what the edges DO. A type cited via a gate relation is a gate — a bar, - not a stage — and belongs at the end. A type that is only ever cited and never cites - is a shelf, and belongs at the start. Precedence is gate, then shelf, then flow: a - gate is nearly always cited-and-never-citing, so testing for it second would file - every gate as a shelf. - - This is a function of the WHOLE graph, so unlike positions within a column it is not - append-stable. A node introducing a type not yet on screen, or the first edge that - makes a type a gate, reorders the columns once. Both are rare and both are real - changes in what the graph is; a claim appended to a type already present does not - move anything. - """ - gate_types, cited, citing = set(), set(), set() - for n in nodes.values(): - for l in n.links: - if (t := nodes.get(l["to"])) is None: - continue - citing.add(n.type) - cited.add(t.type) - if l["rel"] in GATE_RELS: - gate_types.add(t.type) - gate_types.add(GATE_TYPE) - - seen = list(dict.fromkeys(n.type for n in _order(nodes))) - gates = [t for t in seen if t in gate_types] - shelves = [t for t in seen if t not in gates and t in cited and t not in citing] - flow = [t for t in seen if t not in gates and t not in shelves] - ordered = ([t for t in FLOW if t in shelves] - + [t for t in shelves if t not in FLOW] - + [t for t in FLOW if t in flow] - + [t for t in flow if t not in FLOW] - + gates) - return ordered, set(gates), set(shelves) - - -def _columns(nodes: dict) -> dict: - cols, _, _ = roles(nodes) - at = {c: 0 for c in cols} - pos = {} - for n in _order(nodes): - i = cols.index(n.type) - pos[n.id] = [i * COLW, TOP + at[n.type] * (CARDH + GAP)] - at[n.type] += 1 - return pos - - -def _neighbours(nodes: dict) -> dict: - out = {nid: [] for nid in nodes} - for n in nodes.values(): - for l in n.links: - if l["to"] in nodes: - out[n.id].append(l["to"]) - out[l["to"]].append(n.id) - return out - - -def _map(nodes: dict) -> tuple: - """Cluster around the busiest nodes. - - Degree is the one signal every graph has. Clustering on `type: gate` produced 18 - clusters on one real graph and 2 on another, because how many gates a graph declares - is a property of its rules, not of knoten. - - Honest limit, and it is bigger than "a node that gains edges": the hub COUNT is - `round(sqrt(n))`, so it steps up at n ≈ 7, 13, 21, 31, … and the new hub's rank - inserts mid-list, rotating every later cluster. Measured on a growing graph: 7 of 20 - nodes moved at n=21. Between those thresholds an appended leaf moves nothing. - Ordering clusters by degree instead of arrival was tried and is strictly worse. - """ - if not nodes: - return {}, {} - nbrs = _neighbours(nodes) - degree = {nid: len(v) for nid, v in nbrs.items()} - busiest = lambda nid: (-degree[nid], nid) - - hubs = sorted(nodes, key=busiest)[:max(3, round(math.sqrt(len(nodes))))] - hub_set = set(hubs) - - def home(nid): - if direct := [x for x in nbrs[nid] if x in hub_set]: - return min(direct, key=busiest) - if nbrs[nid]: # one hop further out - best = min(nbrs[nid], key=busiest) - if via := [x for x in nbrs[best] if x in hub_set]: - return min(via, key=busiest) - return "unattached" - - cells = {h: [h] for h in hubs} - for n in _order(nodes): - if n.id not in hub_set: - cells.setdefault(home(n.id), []).append(n.id) - - rank = {n.id: i for i, n in enumerate(_order(nodes))} - # `unattached` is not a node and has no arrival rank. Sorting it FIRST (a -1 default) - # meant the day a graph gained its first orphan, every real cluster shifted one slot - # along the spiral. It sorts last, where a bucket that only ever grows belongs. - names = sorted(cells, key=lambda c: (rank.get(c, math.inf), c)) - - pos, walls = {}, {} - for i, c in enumerate(names): - r = CLUSTER * math.sqrt(i) - cx, cy = r * math.cos(i * GOLDEN), r * math.sin(i * GOLDEN) - orbit = [x for x in cells[c] if x != c] - if c in nodes: - pos[c] = [round(cx, 2), round(cy, 2)] - for j, nid in enumerate(orbit): - dx, dy = _sunflower(j + 1 if c in nodes else j) - pos[nid] = [round(cx + dx, 2), round(cy + dy, 2)] - walls[c] = [round(cx, 2), round(cy, 2), - round(SPACING * math.sqrt(max(len(orbit), 1)) + 26, 2)] - return pos, walls - - -def layout(nodes: dict) -> dict: - """Both views, keyed by name. Pure, deterministic, no persisted state.""" - pos, _ = _map(nodes) - return {"columns": _columns(nodes), "map": pos} - - -SECTION_LIMIT = 1500 - - -def _clip(text: str) -> str: - """Say when the section is cut. The panel folds long prose behind "show more", which - would otherwise present a truncated section as the whole of it.""" - return text if len(text) <= SECTION_LIMIT else text[:SECTION_LIMIT] + "… [truncated]" - - -def payload(root: Path) -> dict: - """Everything the page draws. Structured, never raw markdown: `section()` already - splits the body, and `results`/`repro` are already mappings — so no markdown parser - is needed on either side of the wire.""" - nodes = load(root) - cfg = load_config(root) - cols, gates, shelves = roles(nodes) - gates &= set(cols) - pos, walls = _map(nodes) - columns = _columns(nodes) - - types = cfg.get("node_types") - return { - "root": root.name, - "count": len(nodes), - "columns": cols, - "gate_types": sorted(gates), - "shelf_types": sorted(shelves), - "walls": walls, - "graph": { - "name": cfg.get("name"), - # `node_types` is a list when a graph only declares its vocabulary, and a - # mapping when it also says what the words mean. Both are legal. - "vocab": types if isinstance(types, dict) else {}, - "rules": [{k: r.get(k) for k in - ("id", "when_type", "when_status", "require_edge", - "require_sections", "message")} - for r in (cfg.get("rules") or [])], - }, - "nodes": [{ - "id": n.id, "type": n.type, "status": n.status, - "title": n.title, "tags": n.tags, - "created": str(n.frontmatter.get("created") or ""), - "links": [{"rel": l["rel"], "to": l["to"]} for l in n.links], - "backlinks": [{"rel": b["rel"], "to": b["to"]} for b in n.backlinks], - "sections": [{"title": t, "text": _clip(section(n.body, t) or "")} - for t in n.sections], - "results": n.results, "repro": n.repro, "attachments": n.attachments, - "columns": columns[n.id], "map": pos[n.id], - } for n in _order(nodes)], - } - - -def render(root: Path) -> str: - """The template with the payload inlined. - - `<` is escaped rather than the `` sequence alone: a node body containing that - literal would otherwise close the tag and blank the whole page — every node in the - graph lost to one string in one post-mortem. - """ - blob = json.dumps(payload(root), default=str).replace("<", "\\u003c") - return (HERE / "viz.html").read_text(encoding="utf-8").replace("__KNOTEN_DATA__", blob) - - -def write(root: Path, dest: Path) -> Path: - if not (root / "nodes").is_dir(): - raise GraphError(f"{root} is not a knoten graph (no nodes/ directory)") - dest.write_text(render(root), encoding="utf-8") - return dest diff --git a/examples/llm-research/graph.yaml b/examples/llm-research/graph.yaml index 665eec1..ab65ac3 100644 --- a/examples/llm-research/graph.yaml +++ b/examples/llm-research/graph.yaml @@ -7,7 +7,7 @@ description: What actually improves LLM accuracy on reasoning tasks? # These are enforced. A node whose type or status is not on these lists is a typo, and # a claim with a typo'd status silently drops out of every query. -node_types: [hypothesis, experiment, finding, gate, source] +node_types: [question, hypothesis, experiment, finding, gate, source] statuses: [open, alive, dead, retracted, superseded, active] # The filter axis. `knoten index --tag decoding` narrows a graph too big to read into a diff --git a/examples/llm-research/nodes/hyp-self-consistency.md b/examples/llm-research/nodes/hyp-self-consistency.md index 573889b..51df00b 100644 --- a/examples/llm-research/nodes/hyp-self-consistency.md +++ b/examples/llm-research/nodes/hyp-self-consistency.md @@ -6,6 +6,7 @@ created: 2026-03-02 tags: [decoding, reasoning] cause: weak_baseline # greedy at 1x was the strawman links: + - {rel: prov:wasDerivedFrom, to: question-what-improves-reasoning} - {rel: kn:killedByGate, to: gate-compute-matched-baseline} repro: script: attachments/hyp-self-consistency/self_consistency.py diff --git a/examples/llm-research/nodes/question-what-improves-reasoning.md b/examples/llm-research/nodes/question-what-improves-reasoning.md new file mode 100644 index 0000000..d098ebf --- /dev/null +++ b/examples/llm-research/nodes/question-what-improves-reasoning.md @@ -0,0 +1,16 @@ +--- +id: question-what-improves-reasoning +type: question +status: open +tags: [reasoning, evaluation] +--- +# What actually improves LLM accuracy on reasoning tasks? + +## Why it matters +Every published gain is reported against a baseline the authors chose. The question is +which of them survive a baseline chosen by someone with no stake in the answer. + +## What would count as an answer +A method that beats a compute-matched baseline on a held-out reasoning benchmark, by a +margin larger than the seed-to-seed spread, and that still holds when the baseline is +given the same token budget. diff --git a/src/knoten/cli.py b/src/knoten/cli.py index d6612dd..f539f03 100644 --- a/src/knoten/cli.py +++ b/src/knoten/cli.py @@ -351,8 +351,7 @@ def detach(root, nid, name) -> int: # place they ARE defined, and `knoten viz` shows them beside each column. node_types: question: what this graph exists to answer — a question, a statement or a task - source: where the work came from — a paper, blogpost, dataset, search, or your - own intuition (write that one down too; a hunch you cite is auditable) + source: where the work came from — a paper, dataset, search, or your own intuition idea: what you took from a source; a direction, not yet a testable claim hypothesis: a falsifiable claim derived from an idea experiment: the test built to verify or falsify a hypothesis @@ -399,9 +398,7 @@ def detach(root, nid, name) -> int: ## What would count as an answer -Everything else in this graph descends from this node. Investigate sources first -(papers, posts, datasets, searches) — or start from your own intuition and record -that as a source too, so the origin is citable either way. +Replace this with the real question. Everything else in this graph descends from it. """ TEMPLATE_GATE = """\ @@ -485,6 +482,7 @@ def init(name) -> int: TEMPLATE_QUESTION.format(name=name), encoding="utf-8") (root / "nodes" / "gate-example.md").write_text(TEMPLATE_GATE, encoding="utf-8") print(f"created graph '{name}'\n") + print(f" {name}/nodes/question-{name}.md <- start here: what this graph answers") print(f" {name}/graph.yaml <- edit the rules for THIS topic") print(f" {name}/nodes/ <- one markdown file per question / source / idea / claim\n") print(f" next: cd {name} && git init && knoten hook") diff --git a/src/knoten/mcp_server.py b/src/knoten/mcp_server.py index 9938048..011f6f0 100644 --- a/src/knoten/mcp_server.py +++ b/src/knoten/mcp_server.py @@ -63,9 +63,9 @@ from ONE question, statement or task and everything descends from it: question -> source -> idea -> hypothesis -> experiment -> finding, with findings opening new ideas. A gate stands outside that loop as the bar every claim must survive. Sources are papers, posts, -datasets and searches — and your own intuition, which is worth recording as a source too -so that an idea always names where it came from. Nothing is -one-to-one: one hypothesis may have several experiments and several findings. +datasets and searches — and your own intuition, which is worth recording as a source too, +so that an idea always names where it came from. Nothing is one-to-one: one hypothesis may +have several experiments and several findings. An edge always points from the NEW node to the one it depends on — claim -> the gate it faced, experiment -> the hypothesis it tests, claim -> what it was derived from. Writing diff --git a/src/knoten/viz.py b/src/knoten/viz.py index a293d16..e0a6f85 100644 --- a/src/knoten/viz.py +++ b/src/knoten/viz.py @@ -92,11 +92,15 @@ def roles(nodes: dict) -> tuple: seen = list(dict.fromkeys(n.type for n in _order(nodes))) gates = [t for t in seen if t in gate_types] shelves = [t for t in seen if t not in gates and t in cited and t not in citing] - flow = [t for t in seen if t not in gates and t not in shelves] - ordered = ([t for t in FLOW if t in shelves] - + [t for t in shelves if t not in FLOW] - + [t for t in FLOW if t in flow] - + [t for t in flow if t not in FLOW] + rest = [t for t in seen if t not in gates and t not in FLOW] + + # A type FLOW names is placed by FLOW, whatever its edges look like. Sorting shelves + # ahead of everything meant `source` — cited by an idea, citing nothing — overtook + # `question` on any graph where nothing happened to cite the question back, so the + # column order contradicted the loop it exists to show. + ordered = ([t for t in FLOW if t in seen and t not in gates] + + [t for t in rest if t in shelves] + + [t for t in rest if t not in shelves] + gates) return ordered, set(gates), set(shelves) diff --git a/tests/test_cli.py b/tests/test_cli.py index f7d4b2f..c6b55a4 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -361,9 +361,14 @@ def test_the_question_comes_before_everything_else(tmp_path, monkeypatch): monkeypatch.chdir(tmp_path) main(["init", "demo"]) root = tmp_path / "demo" - for nid, ntype in [("source-a-paper", "source"), ("idea-a", "idea")]: - (root / "nodes" / f"{nid}.md").write_text( - f"---\nid: {nid}\ntype: {ntype}\nstatus: open\n---\n\n# x\n", encoding="utf-8") + # Edges matter: `source` is cited by the idea and cites nothing, which made it a + # SHELF — and shelves used to be sorted ahead of everything, so `source` overtook + # `question`. With no edges at all there are no shelves and this passed vacuously. + (root / "nodes" / "source-a-paper.md").write_text( + "---\nid: source-a-paper\ntype: source\nstatus: open\n---\n\n# x\n", encoding="utf-8") + (root / "nodes" / "idea-a.md").write_text( + "---\nid: idea-a\ntype: idea\nstatus: open\nlinks:\n" + " - {rel: prov:wasDerivedFrom, to: source-a-paper}\n---\n\n# x\n", encoding="utf-8") cols, _, _ = viz.roles(load(root))