feat: attention-first memory injection redesign

README.md

@@ -87,7 +87,7 @@ Then add to `~/.claude.json` (user-scope MCP — create the file if it doesn't e
 
 And add hooks to `~/.claude/settings.json`:
 
-A single SessionStart hook ships: `session_bootstrap` opens (or reuses) a background episode for the session and injects the project's curated context — both project-scoped and general-scope semantic memories plus all distilled reflections (`do` / `dont` / `neutral` buckets) — as `additionalContext` so Claude has prior memory available without needing to call any retrieval tool first. The hook does its work in-process against `memory.db`; if it fails for any reason, a fallback directive is injected instructing Claude to call `mcp__better-memory__memory_session_bootstrap` manually.
+Four hooks ship. `session_bootstrap` (SessionStart) opens (or reuses) a background episode for the session and injects the project's curated context — project-scoped and general-scope semantic memories plus distilled reflections (`do` / `dont` / `neutral` buckets) — as `additionalContext` so Claude has prior memory available without needing to call any retrieval tool first. Only the top `BETTER_MEMORY_BOOTSTRAP_TOP_N` project-scoped items (default 5) render in full; the rest collapse into a one-line index plus a retrieve affordance (`BETTER_MEMORY_BOOTSTRAP_TOP_N=0` restores the old full-dump behavior). The hook does its work in-process against `memory.db`; if it fails for any reason, a fallback directive is injected instructing Claude to call `mcp__better-memory__memory_session_bootstrap` manually. `contextual_inject` (UserPromptSubmit + PreToolUse) additionally surfaces memories relevant to the current prompt/tool-input mid-session — see [Configuration](website/configuration.md) and [Architecture](website/architecture.md#injection-strategies) for how it scores, floors, and dedups candidates.
 
 ```json
 {
@@ -124,11 +124,34 @@ A single SessionStart hook ships: `session_bootstrap` opens (or reuses) a backgr
           }
         ]
       }
+    ],
+    "UserPromptSubmit": [
+      {
+        "hooks": [
+          {
+            "type": "command",
+            "command": "/absolute/path/to/.venv/bin/python -m better_memory.hooks.contextual_inject"
+          }
+        ]
+      }
+    ],
+    "PreToolUse": [
+      {
+        "matcher": "Skill|Task|Write",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "/absolute/path/to/.venv/bin/python -m better_memory.hooks.contextual_inject"
+          }
+        ]
+      }
     ]
   }
 }
 ```
 
+`contextual_inject` is gated at runtime by `BETTER_MEMORY_CONTEXT_INJECT_MODE` (default `both`) — set it to `userprompt`, `pretool`, or `off` to narrow or disable which event actually injects.
+
 On Windows, point hooks at `.venv\Scripts\pythonw.exe` (no console flash) instead of `python.exe`.
 
 Restart Claude Code. MCP servers don't hot-reload.
@@ -146,6 +169,13 @@ One env var roots the runtime filesystem layout:
 | `BETTER_MEMORY_EMBEDDINGS_BACKEND` | `ollama` | `ollama` (default) — local Ollama at `OLLAMA_HOST`; `sqlite` — pure-SQL trigram-FTS5 fusion, no model downloads and no in-memory state. |
 | `BETTER_MEMORY_AUTO_PRUNE` | (unset = `false`) | When set to `1`, the auto-retention runner (which fires on `memory.retrieve`, throttled to once per 24h) ALSO hard-deletes archived observations older than 365 days. **Irreversible.** Default is archive-only (status flip, reversible). Opt in only if you actively want disk space reclaimed. |
 | `BETTER_MEMORY_PROJECT` | unset | Force the project name for all calls in this process. Highest-priority project-resolution signal — overrides both the `.better-memory` file and the git-derived name. Designed for subprocess scoping (e.g. ralph's executor sets it per-iteration so subagent observations land in the PBI's target_repo regardless of the worktree's cwd). Empty/whitespace-only values are treated as unset. |
+| `BETTER_MEMORY_CONTEXT_INJECT_MODE` | `both` | Contextual memory-injection hook trigger: `userprompt`, `pretool`, `both` (default), or `off`. |
+| `BETTER_MEMORY_BOOTSTRAP_TOP_N` | `5` | Number of project-scoped semantic memories/reflections the SessionStart bootstrap renders in full; the rest collapse into a one-line index. `0` = legacy full dump. |
+| `BETTER_MEMORY_CONTEXT_MIN_HITS` | `2` | Minimum distinct keyword hits a memory needs to clear the contextual-injection floor. |
+| `BETTER_MEMORY_CONTEXT_MAX_ITEMS` | `3` | Max memories the contextual-injection hook injects per firing. |
+| `BETTER_MEMORY_CONTEXT_REINJECT_TURNS` | `0` | Turns before a contextually-injected memory can be re-injected. `0` = never re-inject. A turn is one firing of the `contextual_inject` hook (each user prompt, plus each matched tool call in mode `both`), not one user prompt-response cycle. |
+
+See [Configuration](website/configuration.md) for the full table and injection-tuning detail.
 
 ### Project-name override
 
@@ -236,7 +266,7 @@ The server registers 22 tools, grouped below. Full schemas are in [`website/mcp-
 
 | Tool | Purpose |
 |---|---|
-| `memory.session_bootstrap(source?, session_id?, cwd?)` | Open or reuse a session episode and inject project + general semantic memories and reflections as `additionalContext` markdown. Reflections capped at 20 per polarity bucket, ranked by usefulness then confidence. Mirrors the SessionStart hook; callable manually for recovery, testing, or post-`/clear` re-injection. |
+| `memory.session_bootstrap(source?, session_id?, cwd?)` | Open or reuse a session episode and inject project + general semantic memories and reflections as `additionalContext` markdown. Reflections retrieved up to 20 per polarity bucket, ranked by usefulness then confidence; only the top `BETTER_MEMORY_BOOTSTRAP_TOP_N` project-scoped items render in full, the rest collapse into a one-line index. Mirrors the SessionStart hook; callable manually for recovery, testing, or post-`/clear` re-injection. |
 | `memory.run_retention(retention_days?, prune?, prune_age_days?, dry_run?)` | Apply spec §9 retention rules; archive or hard-delete. |
 | `memory.start_ui()` | Spawn or reuse the management UI; returns `{url, reused}`. |
 

better_memory/config.py

@@ -12,7 +12,10 @@
 Default home is ``~/.better-memory``. External-service knobs
 (``OLLAMA_HOST``, ``EMBED_MODEL``, ``AUDIT_LOG_RETRIEVED``,
 ``BETTER_MEMORY_EMBEDDINGS_BACKEND``) are separate env vars because they're
-orthogonal to path layout.
+orthogonal to path layout. Injection-tuning knobs
+(``BETTER_MEMORY_BOOTSTRAP_TOP_N``, ``BETTER_MEMORY_CONTEXT_MIN_HITS``,
+``BETTER_MEMORY_CONTEXT_MAX_ITEMS``, ``BETTER_MEMORY_CONTEXT_REINJECT_TURNS``)
+control content injection strategies.
 """
 
 from __future__ import annotations
@@ -193,6 +196,19 @@ def _resolve_bool(env_var: str, default: bool) -> bool:
     return raw.strip().lower() in {"1", "true", "yes", "on"}
 
 
+def _resolve_nonneg_int(env_var: str, default: int) -> int:
+    raw = os.environ.get(env_var)
+    if raw is None:
+        return default
+    try:
+        value = int(raw)
+    except ValueError as exc:
+        raise ValueError(f"{env_var} must be a non-negative integer, got {raw!r}") from exc
+    if value < 0:
+        raise ValueError(f"{env_var} must be a non-negative integer, got {raw!r}")
+    return value
+
+
 @dataclass(frozen=True)
 class Config:
     """Resolved better-memory configuration."""
@@ -213,6 +229,10 @@ class Config:
     agentcore_semantic_memory_id: str | None
     agentcore_episodic_memory_id: str | None
     context_inject_mode: Literal["userprompt", "pretool", "both", "off"]
+    bootstrap_top_n: int
+    context_min_hits: int
+    context_max_items: int
+    context_reinject_turns: int
 
 
 _DEFAULT_CONTEXT_INJECT_MODE = "both"
@@ -297,4 +317,8 @@ def get_config() -> Config:
         agentcore_semantic_memory_id=agentcore_semantic_memory_id,
         agentcore_episodic_memory_id=agentcore_episodic_memory_id,
         context_inject_mode=_resolve_context_inject_mode(),
+        bootstrap_top_n=_resolve_nonneg_int("BETTER_MEMORY_BOOTSTRAP_TOP_N", 5),
+        context_min_hits=_resolve_nonneg_int("BETTER_MEMORY_CONTEXT_MIN_HITS", 2),
+        context_max_items=_resolve_nonneg_int("BETTER_MEMORY_CONTEXT_MAX_ITEMS", 3),
+        context_reinject_turns=_resolve_nonneg_int("BETTER_MEMORY_CONTEXT_REINJECT_TURNS", 0),
     )

better_memory/db/migrations/0012_contextual_exposure.sql

@@ -0,0 +1,41 @@
+-- Migration 0012: contextual exposure source + contextual diagnostics.
+--
+-- The contextual_inject hook (UserPromptSubmit / PreToolUse) now records
+-- exposures so contextually-injected memories are rateable. SQLite cannot
+-- ALTER a CHECK constraint, so session_memory_exposure is recreated to
+-- widen source IN ('bootstrap','retrieve') to include 'contextual'.
+-- No table holds a foreign key into session_memory_exposure.
+
+CREATE TABLE session_memory_exposure_new (
+    session_id     TEXT NOT NULL,
+    memory_kind    TEXT NOT NULL CHECK(memory_kind IN ('reflection', 'semantic')),
+    memory_id      TEXT NOT NULL,
+    exposed_at     TEXT NOT NULL,
+    source         TEXT NOT NULL CHECK(source IN ('bootstrap', 'retrieve', 'contextual')),
+    rated_at       TEXT,
+    classification TEXT CHECK(classification IN
+                     ('cited', 'shaped', 'ignored', 'misled', 'overlooked')),
+    PRIMARY KEY (session_id, memory_kind, memory_id, exposed_at)
+);
+
+INSERT INTO session_memory_exposure_new
+    (session_id, memory_kind, memory_id, exposed_at, source, rated_at, classification)
+SELECT
+    session_id, memory_kind, memory_id, exposed_at, source, rated_at, classification
+FROM session_memory_exposure;
+
+DROP TABLE session_memory_exposure;
+ALTER TABLE session_memory_exposure_new RENAME TO session_memory_exposure;
+
+CREATE INDEX idx_sme_session_unrated
+    ON session_memory_exposure(session_id) WHERE rated_at IS NULL;
+CREATE INDEX idx_sme_memory
+    ON session_memory_exposure(memory_kind, memory_id);
+
+-- Contextual-injection observability counters (R1=A in the spec).
+
+INSERT INTO rating_diagnostics (metric, value) VALUES ('contextual_fired_userprompt', 0);
+INSERT INTO rating_diagnostics (metric, value) VALUES ('contextual_fired_pretool', 0);
+INSERT INTO rating_diagnostics (metric, value) VALUES ('contextual_injected', 0);
+INSERT INTO rating_diagnostics (metric, value) VALUES ('contextual_suppressed_floor', 0);
+INSERT INTO rating_diagnostics (metric, value) VALUES ('contextual_suppressed_dedup', 0);

better_memory/hooks/contextual_inject.py

@@ -2,6 +2,15 @@
 current prompt or tool-input. Gated by BETTER_MEMORY_CONTEXT_INJECT_MODE
 (userprompt | pretool | both | off). Never raises; always exits 0.
 
+Candidates are scored via retrieve_relevant and must clear a min-hits floor
+(cfg.context_min_hits) and fit within a max-items cap (cfg.context_max_items).
+A per-session SeenStore dedups injected memories across turns within a run
+(cfg.context_reinject_turns controls re-injection after N turns). Survivors
+are recorded as 'contextual' exposures (best-effort; a write failure never
+blocks injection) and counted in rating_diagnostics for observability
+(contextual_fired_userprompt/pretool, contextual_injected,
+contextual_suppressed_floor, contextual_suppressed_dedup).
+
 NOTE: whether PreToolUse fires for the built-in Skill/Task tools is
 environment-dependent (see the plan's Task 0 probe); UserPromptSubmit is the
 reliable trigger.
@@ -11,18 +20,35 @@
 import json
 import os
 import sys
-from contextlib import closing
+from contextlib import closing, nullcontext
+from datetime import UTC, datetime
 from pathlib import Path
 
 from better_memory.config import get_config, project_name
 from better_memory.db.connection import connect
 from better_memory.hooks._error_log import record_hook_error
+from better_memory.services.context_seen import SeenStore, prune_stale
 from better_memory.services.relevant import format_relevant, retrieve_relevant
 from better_memory.storage import build_backend
 
 _MAX_STDIN_BYTES = 1_000_000
 
 
+def _bump_diagnostic(conn, cfg, metric: str) -> None:
+    """Best-effort observability counter. Sqlite mode only; never raises."""
+    if cfg.storage_backend != "sqlite" or conn is None:
+        return
+    try:
+        conn.execute(
+            "UPDATE rating_diagnostics SET value = value + 1, updated_at = ? "
+            "WHERE metric = ?",
+            (datetime.now(UTC).isoformat(), metric),
+        )
+        conn.commit()
+    except BaseException:  # noqa: BLE001
+        pass
+
+
 def _enabled(event: str, mode: str) -> bool:
     if mode == "off":
         return False
@@ -63,20 +89,62 @@ def main() -> None:
         cfg = get_config()
         if _enabled(event, cfg.context_inject_mode):
             query = _query_from(payload, event)
+            session_id = str(payload.get("session_id") or "")
             cwd = str(payload.get("cwd") or os.getcwd())
             project = project_name(Path(cwd))
-            with closing(connect(cfg.memory_db)) as conn:
+            state_dir = cfg.home / "state"
+            prune_stale(state_dir, now=datetime.now(UTC))
+            seen = SeenStore(state_dir, session_id)
+            seen.bump_turn()
+            conn_ctx = (
+                closing(connect(cfg.memory_db))
+                if cfg.storage_backend == "sqlite"
+                else nullcontext(None)
+            )
+            with conn_ctx as conn:
+                _bump_diagnostic(
+                    conn, cfg,
+                    "contextual_fired_userprompt" if event == "UserPromptSubmit"
+                    else "contextual_fired_pretool",
+                )
                 backend = build_backend(
                     config=cfg,
                     memory_conn=conn,
                     embedder=None,
-                    session_id=None,
+                    session_id=session_id or None,
                     project=project,
                 )
                 items = retrieve_relevant(
-                    backend, query=query, project=project, limit=5
+                    backend, query=query, project=project,
+                    min_hits=cfg.context_min_hits,
+                    max_items=cfg.context_max_items,
                 )
-            rendered = format_relevant(items)
+                had_candidates = bool(items)
+                pairs = [(m.kind, m.id) for m in items]
+                unseen = set(seen.filter_unseen(
+                    pairs, reinject_turns=cfg.context_reinject_turns,
+                ))
+                items = [m for m in items if (m.kind, m.id) in unseen]
+                if items:
+                    rendered = format_relevant(items)
+                    survivors = [(m.kind, m.id) for m in items]
+                    try:
+                        backend.record_exposures(
+                            session_id=session_id,
+                            items=survivors,
+                            source="contextual",
+                        )
+                    except BaseException as exc:  # noqa: BLE001 - never block injection
+                        try:
+                            record_hook_error(hook_name="contextual_inject_exposure", exc=exc)
+                        except BaseException:  # noqa: BLE001
+                            pass
+                    seen.mark_seen(survivors)
+                    _bump_diagnostic(conn, cfg, "contextual_injected")
+                elif had_candidates:
+                    _bump_diagnostic(conn, cfg, "contextual_suppressed_dedup")
+                else:
+                    _bump_diagnostic(conn, cfg, "contextual_suppressed_floor")
     except BaseException as exc:  # noqa: BLE001
         try:
             record_hook_error(hook_name="contextual_inject", exc=exc)

better_memory/hooks/session_close.py

@@ -139,6 +139,7 @@ def _emit_rating_directive_if_unrated(session_id: str) -> bool:
                 """
                 SELECT e.memory_kind, e.memory_id,
                        MIN(e.exposed_at) AS exposed_at,
+                       MIN(e.source) AS source,
                        COALESCE(r.title, s.content) AS display
                   FROM session_memory_exposure e
                   LEFT JOIN reflections        r ON e.memory_kind='reflection'
@@ -160,17 +161,25 @@ def _emit_rating_directive_if_unrated(session_id: str) -> bool:
         CAP_BYTES = 8 * 1024
         refl_lines = []
         sem_lines = []
+        source_counts: dict[str, int] = {}
         for r in rows:
             display = (r["display"] or "")[:TRUNC]
+            source = r["source"] or "bootstrap"
+            source_counts[source] = source_counts.get(source, 0) + 1
+            line = f"- {r['memory_id']} [{source}]: {display}"
             if r["memory_kind"] == "reflection":
-                refl_lines.append(f"- {r['memory_id']}: {display}")
+                refl_lines.append(line)
             else:
-                sem_lines.append(f"- {r['memory_id']}: {display}")
+                sem_lines.append(line)
+        counts_line = "sources: " + ", ".join(
+            f"{k} {v}" for k, v in sorted(source_counts.items())
+        )
 
         directive = (
             "RATE_MEMORIES — before this session ends, classify the "
             "memories that were exposed during this session and that you "
-            "did NOT already credit via memory.credit.\n\n"
+            "did NOT already credit via memory.credit.\n"
+            f"({counts_line})\n\n"
             f"Reflections ({len(refl_lines)}):\n"
             + ("\n".join(refl_lines) if refl_lines else "  (none)")
             + f"\n\nSemantic memories ({len(sem_lines)}):\n"