Skip to content

feat(memory): deferred task-conditioned injection (flag-gated) + three-leg contextual gate#84

Merged
emp3thy merged 17 commits into
mainfrom
feat/deferred-injection
Jul 24, 2026
Merged

feat(memory): deferred task-conditioned injection (flag-gated) + three-leg contextual gate#84
emp3thy merged 17 commits into
mainfrom
feat/deferred-injection

Conversation

@emp3thy

@emp3thy emp3thy commented Jul 24, 2026

Copy link
Copy Markdown
Owner

Summary

Implements the deferred-injection design (docs/superpowers/specs/2026-07-23-deferred-injection-design.md); absorbs the planned PR-C (CLAUDE.md rewrite + drift sentinel).

  1. BETTER_MEMORY_INJECT_MODElegacy (default; today's bootstrap, byte-identical, test-pinned) | deferred (SessionStart injects only general-scope semantic memories + one index line; ~2 exposures/session instead of 7). Flag misparse fails safe to legacy.
  2. Contextual channel upgraded to primary-grade: three-leg scoring (BM25 over reflection FTS + vec cosine with floor BETTER_MEMORY_CONTEXT_VEC_FLOOR=0.55 + Wilson prior) behind an evidence gate — a memory injects only with BM25 or vec evidence; the prior ranks but can never qualify alone; silent when nothing qualifies. Keyword fallback covers semantics-without-vectors and agentcore mode. CONTEXT_MIN_HITS deprecated.
  3. Cadence: UserPromptSubmit every prompt (per-memory session dedup); PreToolUse widened to all tools with a once-per-session latch (fixed a real latent bug: SeenStore._load dropped unknown keys, so any persisted flag silently reset per hook process).
  4. Ollama outage cost bounded cross-process: file-persisted 60s cooldown on SyncEmbedder (hooks are a fresh process per firing — an in-process breaker resets every prompt).
  5. Exploration serves tagged (via_exploration, migration 0015) — excluded from the headline usefulness metric; conversion rate becomes measurable.
  6. CLAUDE.snippet.md rewritten behavioural-only + drift sentinel: schema-derived (property names ∪ enum values), param-signal-gated, max one warning line, best-effort. Live-checked: flags the one genuine drift in the current user CLAUDE.md, silent on everything else.

Validation

  • Full suite 1591 passed / 22 skipped / 0 failed; pyright 0 errors.
  • Per-task spec+quality reviews (9 tasks, 3 multi-round loops); final whole-branch review: 0 Critical, 1 Important (agentcore semantic fallback) fixed + mutation-verified pre-merge.
  • Vec floor calibrated on 1,593 labelled prompt/memory pairs from the A/B corpus: 0.55 kept (precision 0.672 vs 0.529 base; F1-optimal 0.40 rejected — would pass 96% of negatives).
  • Legacy render byte-identical under test; deferred exposures limited to rendered ids.

Rollout

Merges with legacy as the live default — bootstrap behaviour unchanged on merge day. Post-merge: env-flag A/B (identical code, only the flag differs per arm, 24 sessions/arm) gates the flip to deferred; fail-twice = stay legacy.

🤖 Generated with Claude Code

emp3thy and others added 16 commits July 23, 2026 22:54
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…latch, all-tools matcher

Wires the three-leg evidence-gated scorer (BM25/vector/keyword-fallback)
into contextual_inject.py: builds a SyncEmbedder(down_state_file=...) when
embeddings_backend=ollama and calls retrieve_relevant with the new
conn/sync_embedder/vec_floor signature (no more min_hits).

Adds SeenStore.pretool_fired()/mark_pretool_fired() so PreToolUse does real
retrieval work only once per session; later PreToolUse events short-circuit
via a module-local _SkipInjection sentinel before touching the DB. Widens
install_hooks.py's PreToolUse matcher from "Skill|Task|Write" to None
(unscoped) since the latch makes an all-tools matcher cheap.

Also restores the blank-query short-circuit in retrieve_relevant that the
Task 4 rewrite dropped (a contentless query has no evidence by definition),
and fixes a SeenStore._load() bug found while testing the latch: it silently
dropped the pretool_fired key on every reload because it only carried
forward "turn"/"seen".

Updates the 5 tests in test_contextual_inject.py that assumed the old
keyword min-hits gate (BM25 now matches on token overlap, so
test_below_floor_injects_nothing needed genuinely zero-evidence content),
adds latch coverage, and fixes the golden hook-shape assertions in
test_install_hooks.py (both) and test_setup_sh.py for the matcher change.
Also updates test_agentcore_semantic_parity.py's stale min_hits= call,
collateral from the Task 4 signature change but outside its declared scope.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/hooks/test_contextual_inject.py's bm_home fixture never pinned
BETTER_MEMORY_EMBEDDINGS_BACKEND, so every non-blank-query test relied on
config.py's actual default (ollama): the hook built a real SyncEmbedder and
issued an HTTP call to localhost:11434 on every run. This worked only
because no test seeds embeddings, so a connection-refused/failed embed
degrades silently to the qvec=None fallback path with no observable
difference — a live Ollama on that port would have gone unnoticed too.

Pins BETTER_MEMORY_EMBEDDINGS_BACKEND=sqlite in the fixture, mirroring
tests/ui/conftest.py's pin, and corrects the stale comment on
test_below_floor_injects_nothing that claimed sqlite was the config
default rather than the fixture's pin.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cy fixes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ktick detection

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sync website/architecture.md, configuration.md, mcp-tools.md with the
feat/deferred-injection branch: BETTER_MEMORY_INJECT_MODE (legacy default
| deferred bootstrap), the three-leg evidence-gated contextual scorer
(BM25 / vector cosine against BETTER_MEMORY_CONTEXT_VEC_FLOOR / keyword
fallback), deprecate BETTER_MEMORY_CONTEXT_MIN_HITS, document the
PreToolUse-widened-to-all-tools once-per-session latch, the file-persisted
Ollama outage cooldown shared across hook processes, and via_exploration
exposure tagging excluded from the headline usefulness metric.
…string, import hoist

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Claude BugBot Analysis

Found 1 potential bug in this PR.

medium: 1

Found one confirmed cross-backend logic bug: the new three-leg evidence-gated scorer in services/relevant.py computes a Wilson-score ranking prior assuming times_overlooked/times_ignored fields exist on reflection/semantic records, but the AgentCore backend never populates (or even exposes) those fields for reflections, silently inflating agentcore-backed reflection rankings.

Comment thread better_memory/services/relevant.py
…icts for the Wilson prior

Addresses PR #84 review: missing keys made the contextual prior compute a
falsely-perfect hit rate on the AgentCore backend.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Claude BugBot Analysis

No new bugs found. The previously-reported Wilson-score bug is fixed: AgentCore's reflection dict now carries a real times_overlooked count and semantic memories fall through to the SemanticMemory dataclass's times_ignored=0 default (both intentional, documented, and consistent with the protocol contract in storage/protocol.py).

No bugs were detected in this PR.

@emp3thy
emp3thy merged commit 0c02b60 into main Jul 24, 2026
3 checks passed
@emp3thy
emp3thy deleted the feat/deferred-injection branch July 24, 2026 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant