feat(garden): Duplikate zusammenführen, Widersprüche als Thread, Regressionsschutz, Herkunftstest (REQ-457)

CHANGELOG.md

@@ -10,6 +10,13 @@ Versioning, with pre-1.0 compatibility rules described in
 - Changed: passwords and other secrets that are typed into a prompt are redacted before a transcript is stored (REQ-526, PAIN-17). Until now only tokens with a fixed prefix (private keys, `sk-ant-`, `sk-`, `ghp_`, `AKIA`, `xox`, JWT) were replaced, so "Passwort: …" stayed in `session_chunks.text` and `.raw`, in both full-text indexes, in exports and in backups. The new rule replaces the value behind a label and keeps the label: `Passwort: Hx7!kQ2pz` becomes `Passwort: [REDACTED:labeled]`. Labels, German and English, any case, also at the end of a compound or after an underscore or camel case hump (`Mailpasswort`, `DB_PASSWORD`, `dbPassword`): password, passwd, pwd, pw, pass, passphrase, Passwort, Kennwort, secret (also `secret key`, `client secret`), api key / api_key / apikey, token, access token, bearer, PIN, Zugangsdaten, credentials. A separator may follow (`:`, `=`, `:=`, `=>`, `->`, a quote mark, `ist` / `is` / `lautet`, markdown bold) or nothing but a blank (`Mailkonto Passwort Hx7!kQ2pz`). A value counts only if it looks like a secret: one token without blanks of 6 to 200 characters that has at least two kinds of characters from lower case, upper case, digit and special character, one of them a digit or special character (a single kind, such as `correcthorsebatterystaple`, only from 12 characters and only behind `:` or in quotes or in `KEY=value`); after a bare label (`Passwort Sommer`) it needs a digit or two special characters; a PIN is 4 to 8 digits. It is not a secret if it is a placeholder (`${DB_PASSWORD}`, `$PW`, `<password>`, `****`, `xxxx`, `changeme`, `your-token`), code (`r.FormValue("password")`, `os.Getenv(…)`, `process.env.X`, a type, a struct tag, a call), a path or a URL, or an identifier, version or number. Prose such as "Passwort zurücksetzen" or "das Passwort ist zu kurz" stays as it is. The password in `scheme://user:password@host` is replaced by `[REDACTED:url]`. The rule also runs on the written form of a JSON line: an escaped quote, line break or `\uXXXX` inside a value is part of it, and the line stays valid JSON.
 - Changed: the server redacts on arrival, in `AppendChunks` and `AppendChunkBatches`, next to the image placeholder, on `text` and on `raw`, so clients that still upload unredacted text are covered. `ctx watch` uses the JSON-aware form for `raw`. The question before sharing a session with guests uses the same detection (`FindSecretsIn(raw, JSON)`) and now finds a password behind a label; sessions stored after this change have nothing left to ask about. Document writes are refused for the same labeled secrets, not only for prefixed tokens. Not changed: knowledge entries, requests, coordination messages and notes are not redacted when written (human tasks, explanations and work notes are refused when they look like a secret, as before); a value an agent copied into one of them stays there.
 - Added: `ctx redact-sessions --db <path> [--dry-run] [--batch 200] [--sleep 20ms] [--after-id N] [--skip-optimize]` applies the same redaction to the chunks already stored, and to every copy of their text: `session_chunks.text` and `.raw`, `chunks_fts` (the full-text index over `text`: the old entry is deleted with its old text and the new one inserted), `chunk_index` and `sess_fts` (the search index of the session view, rebuilt from the redacted `raw`), `sessions.title` where it was cut from the prompt, and the quotes in `knowledge_evidence` and `request_sightings`. Deleting an entry from an FTS5 index only writes a deletion mark, the words stay in the old segments, so both indexes are merged with `optimize` at the end, and the write-ahead log is truncated; the run uses `secure_delete`. It runs beside the server (own connection without migrations, short transactions, waits for locks, walks `session_chunks` by id), is idempotent, can be stopped and continued, and reports counts per table and per kind (`raw/labeled:password`), never a value, plus the knowledge entries and requests whose evidence comes from a changed chunk (their text is not touched). Pages freed before the run and older backups still hold the old text until `VACUUM` (with the server stopped) and until the backups are replaced; see `deploy/README.md`. Context snapshots are immutable and are not touched.
+- Added: the gardener consolidates and asks about regression protection (REQ-457, jobs 3 and 5, part A). It still only proposes: what it files is marked as coming from the gardener (harness `gardener`, confidence `staged`), and every change to another entry is a button a person presses on the review page.
+  - Duplicates: a duplicate note on `/ui/review` now has **Merge**. The card first shows which entry stays (the newest) and which are replaced; pressing it supersedes the older entries by the newer one with the same store call and the same right as `context_supersede` (the right to change every entry involved), the reason is "Gärtner-Vorschlag #note, bestätigt von person", and the note is closed as confirmed. It is refused (409) when an entry is no longer current, the scopes differ or the entry that stays is not trusted yet. The duplicate note no longer has plain approval, and "Approve all" skips it.
+  - Contradictions: `ctx garden check-knowledge --propose` opens a thread per validated contradiction (instead of a staged note), linked to every entry it is about, titled as a question and marked "[Gärtner]", with the reason and the verbatim quotes as its first post, written by the gardener's own account. The gardener registers as the agent `gardener:<project>` in the project room for that. For the same entries there is never a second open thread, a thread a person closed (or archived, or parked) is not reopened and not answered with a new one while the entries are unchanged (the thread's question records the wording it was about), and if the existing threads cannot be looked up nothing is opened. A contradiction whose thread cannot be opened is filed as a staged note. The gardener has no call that sets a thread's state.
+  - Regression protection: in the same sessions, without extra ones, the gardener asks about open pitfalls (nobody has judged them with `context_regression_cover`, and it has no proposal standing for them), at most eight per session. A session names a test that exists (file and name; `func TestX` has to be in a tracked `_test.go` file the session read, other languages: the name as a word), sketches one (file, name, 3 to 6 lines of steps) or says "not testable" with a reason; `ctx` drops every answer the checkout does not bear out. Each answer is a staged gardener note. On the review page **Record as covered** (existing test) and **Record as not testable** call the regression verdict (`covered` with the test, or `not_applicable`) with the right to verify; a sketch only has the plain approve and reject. Sketches also get a section of their own ("Testskizzen") in the morning report written with `--out`.
+  - The answer format grows by an optional `regression` field, so older answers and prompts still work; the question sits in the task of a package that has such pitfalls (about 1,000 characters once per session, about 1.4 % of a package of 30 pitfalls with 2,000 characters each) and not in the standing instruction. No schema change.
+  - Operations: set the gardener's account on the server host with `ctx account set-gardener <name|person:N|N> --db <database>` (it works on the database like the other `ctx account` commands, looks the account up once and stores its number in `org_state`, no schema change; the running server uses it at once; `--clear` takes it away; `ctx account list` marks it `gardener=true`; `ctx serve` logs `gardener_account` at start). Only that account may write an entry with harness `gardener` or a session reference starting with `gardener:` (also through `POST /api/knowledge` with `supersedes` and `POST /api/migrated-knowledge`); everyone else gets 403 `gardener_reserved`, and so does everyone while no account is set, so run the command on every server that runs the gardener before the next run. The role is the account number and never a name: renaming the account changes nothing, and an account that takes its old name gets nothing. The review actions Merge, Record as covered and Record as not testable act only on a note that this account wrote (by the stored account number, not by the label), that nobody has edited since, and whose title names the entries its reference names; otherwise the card shows "Not vouched for as the gardener's: no action", the request is refused with 403 and nothing changes, and without a gardener account the gardener notes offer no action at all. The card shows the entries from the reference with title and link. The test of a regression note must have the shape `path/to/x_test.go:TestName` and is recorded as the note holds it (the request carries no test of its own); the server cannot check it against a checkout, `ctx` did before filing the note.
+  - New tests record every call that is not a read of a gardener run in agent mode with a stand-in for Claude Code against the real server: only staged gardener notes, the registration in the room, threads, their links and first post; nothing that patches, supersedes, retires, judges, sets a thread's state, sets a criterion or closes a request; threads and posts belong to the gardener's account; entries and requests are unchanged; a second run adds nothing; and the gardener's own account (a member) is refused for every one of those steps. The review actions are tested with a guest, a stranger, a member, a reviewer and a lead.
 - Changed: stored session transcripts no longer keep base64 images and files (REQ-517, PAIN-12). Screenshots in tool results made up two thirds of the 1.7 GB the database grew by per day, yet search and the session view only ever used text. The server now replaces every base64 payload of 1 KB or more in a chunk's `raw` line with a placeholder such as `[ghosttree: Bild entfernt, image/png, 1234567 Bytes, sha256:0123456789abcdef]` (type, size of the decoded file, first 16 hex digits of its SHA-256). It covers Claude's `{"type":"base64","media_type":…,"data":…}` (also inside `tool_result`, `message.content` and `toolUseResult`), Claude Code's `toolUseResult.file.base64`, MCP `{"type":"image","data":…,"mimeType":…}` and `blob`, and `data:<mime>;base64,…` strings (Codex `input_image`), for images, PDFs and audio alike. The line stays valid JSON with the same structure and every other byte untouched; a line without a payload is stored byte for byte. Old `ctx` versions are covered because the server does it, and `ctx watch` now strips before uploading, which also saves the transfer. `ctx export` therefore writes placeholders instead of images. Base64 inside a text string (JSON in JSON, the content of a `.b64` file) is not touched.
 - Added: `ctx strip-images --db <path> [--dry-run] [--batch 200] [--sleep 50ms]` does the same for the chunks already stored. It runs beside the server (short transactions, waits for locks, walks `session_chunks` by id), is idempotent, can be stopped and resumed, and reports the chunks checked and changed and the bytes saved. The file shrinks only after `VACUUM` with the server stopped; see `deploy/README.md`. The image data is gone for good, so take a backup first.
 - Changed: knowledge that arrives with a prompt (`UserPromptSubmit`) now arrives once per session and only for words that are unusual in the reader's own language (REQ-521). The hook sends the harness session id (`session_id`, the same field in Claude Code and Codex) and the server remembers per account and session which entries it delivered, in the new table `context_deliveries` (created at open, no schema upgrade; rows older than seven days are removed whenever a delivery is recorded). The start-up briefing records what it actually carried, so a pitfall that was already in the bootstrap is not sent again when a prompt names it, while an entry the budget cut off can still arrive; without a session id nothing changes. A word is distinctive only if it also passes a second gate besides being rare among the entries: it must be unusual in the reader's own session text, so "dinge" or "gärtner" no longer fetch unrelated entries while a codename such as `hlpvpngw` does. The gate counts only chunks of sessions of the caller's own account (never anyone else's, so a prompt cannot probe whether a word is frequent in other people's sessions), in the caller's most recent sessions (at most 24 sessions or 40,000 chunks), counted in `chunks_fts` per session up to a ceiling of one chunk in two thousand of that sample (at least 8). An account with fewer than 5,000 chunks of its own has no gate, as before; a word that few chunks carry anywhere is settled with one short look at the index without counting. A word that at most two entries name now also matches the text of an entry, not only its title. Staged entries and the gardener's notes are never delivered this way. Every delivered entry ends with `(triggered by "word")`; the heading `## Possibly relevant to what you just said (ghosttree)` is unchanged. Measured on a synthetic archive of 1.14 million chunks (four accounts, sessions uploaded interleaved) and 3000 entries: 1.4 to 2.2 ms per prompt in the store for ordinary prompts, at most 55 ms for a prompt of 24 words that other accounts use all the time, whether the caller has many sessions or few. `ctx usage --relevance` shows the word behind each replayed hit and judges each prompt by the account that typed it.

cmd/ctx/account.go

@@ -17,7 +17,17 @@ const accountUsage = `usage: ctx account add <name> [--email E] [--admin] --db <
        ctx account token create <name> [--label L] [--machine M] [--expires-in 720h] --db <path>
        ctx account token revoke <id> --db <path>
        ctx account claim-code <name> --db <path>
-       ctx account login-link <name> [--url https://host] --db <path>`
+       ctx account login-link <name> [--url https://host] --db <path>
+       ctx account set-gardener <name|person:N|N> --db <path>
+       ctx account set-gardener --clear --db <path>
+
+set-gardener makes one account the gardener's, once and for good: the number of
+the account is stored (name, or person:N, or the bare number is looked up once),
+not its name, so renaming that account or giving its old name to another does not
+change who the gardener is. Only that account may write entries with harness
+"gardener" or a "gardener:" reference, and the review page acts only on notes it
+wrote. The running server picks it up at once. Without one nobody may, and the
+gardener's proposals are refused. Give it the role member in the project, not more.`
 
 // cmdAccount arbeitet wie person direkt auf der Datenbank: Konten und Tokens
 // auszustellen braucht Zugriff auf den Server-Host, nicht ein Token.
@@ -37,6 +47,8 @@ func cmdAccount(args []string, stdout io.Writer) int {
 		return cmdAccountClaimCode(args[1:], stdout)
 	case "login-link":
 		return cmdAccountLoginLink(args[1:], stdout)
+	case "set-gardener":
+		return cmdAccountSetGardener(args[1:], stdout)
 	case "token":
 		if len(args) > 1 && args[1] == "create" {
 			return cmdAccountTokenCreate(args[2:], stdout)
@@ -119,12 +131,86 @@ func cmdAccountList(args []string, stdout io.Writer) int {
 		fmt.Fprintf(stdout, "list accounts: %v\n", err)
 		return 1
 	}
+	gardener := st.GardenerAccountID()
 	for _, a := range accounts {
-		fmt.Fprintf(stdout, "%s\t%s\tstate=%s admin=%t email=%s\n", a.ID, a.Name, a.State, a.Admin, a.Email)
+		fmt.Fprintf(stdout, "%s\t%s\tstate=%s admin=%t email=%s", a.ID, a.Name, a.State, a.Admin, a.Email)
+		if id, ok := store.AccountNumericID(a.ID); ok && id == gardener {
+			fmt.Fprint(stdout, " gardener=true")
+		}
+		fmt.Fprintln(stdout)
 	}
 	return 0
 }
 
+// cmdAccountSetGardener names the account the gardener runs as. Like the other
+// account commands it works on the database itself: whoever has the host decides
+// who the gardener is, and nothing a user can do over HTTP changes it.
+func cmdAccountSetGardener(args []string, stdout io.Writer) int {
+	var unset *bool
+	_, who, db, ok := accountFlags("account set-gardener", args, stdout, func(fs *flag.FlagSet) {
+		unset = fs.Bool("clear", false, "take the role away: nobody is the gardener afterwards")
+	})
+	if !ok || (who == "") == !*unset {
+		fmt.Fprintln(stdout, "usage: ctx account set-gardener <name|person:N|N> --db <path>\n       ctx account set-gardener --clear --db <path>")
+		return 2
+	}
+	st, ok := openAccountStore(*db, stdout)
+	if !ok {
+		return 1
+	}
+	defer st.Close()
+	before, wasSet := st.GardenerAccount()
+	if *unset {
+		if err := st.ClearGardenerAccount(); err != nil {
+			fmt.Fprintf(stdout, "clear gardener account: %v\n", err)
+			return 1
+		}
+		fmt.Fprintln(stdout, "no gardener account is set: nobody may write gardener entries, and the gardener's proposals are refused")
+		return 0
+	}
+	acct, err := resolveAccountRef(st, who)
+	if err != nil {
+		fmt.Fprintf(stdout, "%v\n", err)
+		return 1
+	}
+	id, _ := store.AccountNumericID(acct.ID)
+	if err := st.SetGardenerAccount(id); err != nil {
+		fmt.Fprintf(stdout, "set gardener account: %v\n", err)
+		return 1
+	}
+	fmt.Fprintf(stdout, "the gardener's account is %s (%s); the number is what counts, a rename changes nothing. The server picks it up at once.\n", acct.ID, acct.Name)
+	if acct.Admin {
+		fmt.Fprintln(stdout, "note: this account is an administrator; the gardener needs the role member in the project and no more")
+	}
+	if wasSet && before.ID != acct.ID {
+		fmt.Fprintf(stdout, "it was %s (%s); that account may not write gardener entries any more\n", before.ID, before.Name)
+	}
+	return 0
+}
+
+// resolveAccountRef finds the account a person named on the command line: a
+// number ("7" or "person:7") by its number, anything else by its name. The
+// number is what the caller stores.
+func resolveAccountRef(st *store.Store, ref string) (store.Account, error) {
+	ref = strings.TrimSpace(ref)
+	byNumber := ref
+	if _, err := strconv.ParseInt(ref, 10, 64); err == nil {
+		byNumber = "person:" + ref
+	}
+	if strings.HasPrefix(byNumber, "person:") {
+		a, err := st.AccountByPrincipalID(byNumber)
+		if err != nil {
+			return store.Account{}, fmt.Errorf("no account %s", byNumber)
+		}
+		return a, nil
+	}
+	a, err := st.AccountByName(store.NormalizeAccountName(ref))
+	if err != nil {
+		return store.Account{}, fmt.Errorf("no account named %q (ctx account list shows them)", ref)
+	}
+	return a, nil
+}
+
 func cmdAccountTokens(args []string, stdout io.Writer) int {
 	_, name, db, ok := accountFlags("account tokens", args, stdout, nil)
 	if !ok || name == "" {