feat: Opus 4.7 integration (v4.5.0)

CHANGELOG.md

@@ -1,5 +1,38 @@
 # Changelog
 
+## v4.5.0 (2026-04-21)
+
+### Added — Opus 4.7 integration
+
+- **Cache-stable instinct ordering** (`_instinct-activator.sh`): added alphabetical `id.localeCompare` tiebreaker after the priority + occurrences sort. The injected `systemMessage` prefix is now byte-stable across consecutive tool uses with the same match set, which is the prerequisite for prompt-cache hits on Opus 4.7's cached system block (~90% discount on input tokens once the cache warms).
+- **`PreCompact` hook** (`core/_precompact-guard.sh`, new): fires right before Claude Code compacts the context in long-running sessions and re-invokes the session-learner so fresh observations are flushed to proposals before the transcript is rewritten. Uses `timeout 8` and a fire-and-forget pattern to never block the harness; relies on the existing advisory lock inside `_session-learner.sh` for parallel safety.
+- **`settings.template.json`** now declares the new PreCompact hook (hooks 6 → 7). `install.sh` copies and chmods `_precompact-guard.sh`.
+- **RFC `docs/rfc-v5-adaptive-thinking.md`**: design for an opt-in `SINAPSIS_LLM_ANALYZE=1` branch in `/analyze-session` that uses Opus 4.7 adaptive thinking via the Anthropic SDK. Not implemented in this release — ships as a design doc so the core stays fully deterministic until the approach is validated.
+
+### Changed — Caps raised for 1M context
+
+- `_instinct-activator.sh`: `TOKEN_BUDGET` 1500 → 4000, top-N per tool use 3 → 6 (`MAX_INSTINCTS_INJECTED`). With Opus 4.7's 1M window and prompt caching the cost of the extra injection is amortised, so we can surface more instincts per turn.
+- `_session-learner.sh`: observation window 1000 → 5000 lines. Cross-session detectors (repetitions, agent patterns) now see a longer history without paging.
+- `_operator-state.json` d017: Scout/Analyst blueprint switched from Haiku to Sonnet 4.6 per operator preference; Architect stays on Opus (now 4.7).
+
+### Tests
+
+- New suite `tests/test-v45-opus47.sh` — 11 TDD tests covering deterministic ordering (shuffled-index byte-identical output, alphabetical tiebreaker), PreCompact hook (file present, executable, wired in settings and install.sh), and raised caps.
+- All existing suites re-run clean: `test-install-upgrade` 21/21, `test-dashboard` 12/12, `test-dream` 25/25, `test-gstack-separation` 18/18, `test-security` 11/11, `test-v433-hardening` 14/14.
+
+### Rationale
+
+Opus 4.7 brings three things Sinapsis can actually use: a stable 1-hour cache TTL that rewards byte-stable prefixes, a 1M context that removes pressure on per-turn caps, and the PreCompact hook Anthropic now ships in Claude Code. None of the "flashy" features (memory tool, context editing) are a natural fit: Sinapsis already *is* a memory system and the inject happens in a stable systemMessage. The v4.5 changes are purely about making the existing design richer and cheaper to run on top of Opus 4.7, without introducing a new LLM dependency in the hot path.
+
+---
+
+## v4.4.2 (2026-04-18)
+
+### Fixed
+- **`_generate-dashboard.py` crashed on `_catalog.json` dict schema** (regression from v4.4.0): `collect_skills()` iterated `cat` assuming a flat list, but the canonical catalog is `{globalSkills: [...], librarySkills: [...]}`. On any fresh v4.4.0/v4.4.1 install, the very first `/dashboard-sinapsis` run raised `AttributeError: 'str' object has no attribute 'get'`. Fix: detect dict vs list shape, concatenate `globalSkills + librarySkills`, derive real global count from the dict instead of hardcoding 5, and guard all `.get()` calls with `isinstance(s, dict)` so mixed content cannot crash. Reported in [#6](https://github.com/Luispitik/sinapsis/issues/6) by @fvayas, fixed in [#7](https://github.com/Luispitik/sinapsis/pull/7) by @NestorPVsf.
+
+---
+
 ## v4.4.1 (2026-04-17)
 
 ### Fixed

README.md

@@ -1,7 +1,7 @@
-# Sinapsis v4.4.1
+# Sinapsis v4.5.0
 
-[![Version](https://img.shields.io/badge/version-4.4.1-blue.svg)](https://github.com/Luispitik/sinapsis)
-[![Tests](https://img.shields.io/badge/tests-99%20passing-green.svg)](tests/)
+[![Version](https://img.shields.io/badge/version-4.5.0-blue.svg)](https://github.com/Luispitik/sinapsis)
+[![Tests](https://img.shields.io/badge/tests-112%20passing-green.svg)](tests/)
 [![CI](https://github.com/Luispitik/sinapsis/actions/workflows/tests.yml/badge.svg)](https://github.com/Luispitik/sinapsis/actions)
 [![License](https://img.shields.io/badge/license-Source%20Available-orange.svg)](LICENSE)
 
@@ -28,6 +28,16 @@ Think of it as going from a dumb terminal to an assistant that actually knows yo
 
 ---
 
+## What's New in v4.5.0 — Opus 4.7 Integration
+
+> Ride the cache, catch the compaction, raise the ceiling.
+
+- **Cache-stable instinct ordering**: added an `id.localeCompare` tiebreaker after the priority + occurrences sort. The injected `systemMessage` prefix is now byte-stable across consecutive tool uses, which unlocks prompt-cache hits on Opus 4.7's cached system block (~90 % discount on the instinct payload once the cache warms).
+- **New `PreCompact` hook** (`_precompact-guard.sh`): fires right before Claude Code compacts the context in long-running sessions and re-runs the session-learner so fresh observations are flushed to proposals before the transcript is rewritten. Uses a fire-and-forget pattern with an 8 s cap — compaction is never blocked.
+- **Caps raised for 1M context**: `TOKEN_BUDGET` 1500 → 4000, top-N instincts per tool use 3 → 6, observation window for the learner 1000 → 5000 lines. Cross-session detectors (repetitions, agent patterns) see a longer history; per-turn injection can surface more relevant rules.
+- **Opt-in adaptive thinking for `/analyze-session`**: design published in [`docs/rfc-v5-adaptive-thinking.md`](docs/rfc-v5-adaptive-thinking.md). Not shipped enabled — core stays fully deterministic, SDK path is additive and gated by `SINAPSIS_LLM_ANALYZE=1`.
+- **11 new TDD tests** (`tests/test-v45-opus47.sh`): shuffled-index byte-identical output, alphabetical tiebreaker, PreCompact wiring end-to-end, and cap assertions. All existing suites re-run clean: 112 total passing.
+
 ## What's New in v4.4.0 — Observability Dashboard
 
 > See the system that sees you.

core/_instinct-activator.sh

@@ -123,11 +123,16 @@ process.stdin.on("end", () => {
 
   // Priority sort: permanent first, then confirmed; within same level, highest occurrences wins
   // v4.2.1: occurrences tiebreaker (inspired by fs-cortex confidence granularity)
+  // v4.5: id alphabetical tiebreaker — guarantees deterministic order across runs so the
+  //       injected systemMessage prefix is byte-stable across consecutive tool uses, which
+  //       is the prerequisite for prompt-cache hits on the Opus 4.7+ cached system block.
   const order = { permanent: 0, confirmed: 1 };
   injectableMatches.sort((a, b) => {
     const lvl = (order[a.level] ?? 2) - (order[b.level] ?? 2);
     if (lvl !== 0) return lvl;
-    return (b.occurrences || 0) - (a.occurrences || 0); // higher occurrences = higher priority
+    const occ = (b.occurrences || 0) - (a.occurrences || 0);
+    if (occ !== 0) return occ;
+    return (a.id || "").localeCompare(b.id || ""); // v4.5 cache stability tiebreaker
   });
 
   // Deduplicate by domain — keep only highest priority match per domain
@@ -136,15 +141,19 @@ process.stdin.on("end", () => {
     const d = m.domain || "_default";
     if (!domainMap[d]) domainMap[d] = m; // already sorted, first = highest priority
   }
-  const top = Object.values(domainMap).slice(0, 3);
+  // v4.5: top-N raised from 3 to 6 — with Opus 4.7 prompt caching the cost of extra
+  // instincts is amortised across tool uses, so we can be more generous per turn.
+  const MAX_INSTINCTS_INJECTED = 6;
+  const top = Object.values(domainMap).slice(0, MAX_INSTINCTS_INJECTED);
 
   // v4.3.1: sanitize inject content (#5F — prompt injection prevention)
   const INJECT_MAX_LEN = 500;
   const INJECT_BLOCKED = /ignore\s+(previous|above|all)\s+instructions|system:\s*you\s+are|<\/?system>|<\/?prompt>/i;
   // v4.3.3: path traversal protection (inspired by Cortex v3.10)
   const PATH_TRAVERSAL = /\.\.[\/\\]|~\/|\/etc\/|\/proc\/|%2e%2e/i;
   // v4.3.3: token budget cap — max chars injected per tool use (inspired by Cortex 8000/session)
-  const TOKEN_BUDGET = 1500;
+  // v4.5: raised 1500 → 4000 to match 6-instinct cap; still well below any sane context ceiling.
+  const TOKEN_BUDGET = 4000;
 
   // Only output systemMessage if there are injectable matches
   if (top.length > 0) {

core/_precompact-guard.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+# PreCompact Guard - Sinapsis v4.5
+# PreCompact hook: fires right before Claude Code compacts the context in a long-running
+# session. Without this, any tool events observed after the previous Stop and before the
+# compaction happens can be dropped when compaction rewrites the transcript.
+#
+# Strategy: re-invoke _session-learner.sh (idempotent, gated by .last-learn) so any fresh
+# observations are flushed to proposals before context is lost. Session-learner already
+# handles its own locking and skips re-processing observations older than .last-learn.
+#
+# Runs sync with short timeout so compaction is not noticeably delayed.
+
+if [ "${SINAPSIS_DEBUG:-}" = "1" ]; then
+  exec 2>>"$HOME/.claude/skills/_sinapsis-debug.log"
+fi
+
+LEARNER="$HOME/.claude/skills/_session-learner.sh"
+LOG="$HOME/.claude/skills/_precompact.log"
+
+[ ! -x "$LEARNER" ] && exit 0
+
+now=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
+echo "$now | precompact | invoking learner" >> "$LOG" 2>/dev/null
+
+# Fire-and-forget: run the learner but bound its time so we never block the harness.
+# If learner is already running (Stop fired in parallel), the advisory lock inside it
+# serialises writes to _projects.json.
+timeout 8 bash "$LEARNER" </dev/null >/dev/null 2>&1 &
+
+exit 0

core/_session-learner.sh

@@ -39,11 +39,12 @@ const indexFile = process.argv[2];
 const proposalsFile = process.argv[3];
 const logFile = process.argv[4];
 
-// Read last 1000 lines of observations (v4.2: was 100, covers parallel sessions)
+// Read last 5000 lines of observations (v4.5: was 1000; Opus 4.7 1M context absorbs the
+// extra payload without pressure and lets cross-session detectors see a longer window).
 let lines;
 try {
   const content = fs.readFileSync(obsFile, "utf8").trim().split("\n");
-  lines = content.slice(-1000).map(l => { try { return JSON.parse(l); } catch(e) { return null; } }).filter(Boolean);
+  lines = content.slice(-5000).map(l => { try { return JSON.parse(l); } catch(e) { return null; } }).filter(Boolean);
 } catch(e) { process.exit(0); }
 
 if (lines.length < 3) process.exit(0);

core/settings.template.json

@@ -1,7 +1,7 @@
 {
-  "_comment": "Sinapsis v4.1 — settings.json template. Copy to ~/.claude/settings.json (merge with existing).",
-  "_hooks_total": 6,
-  "_hooks_breakdown": "PreToolUse(4) + PostToolUse(1) + Stop(1)",
+  "_comment": "Sinapsis v4.5 — settings.json template. Copy to ~/.claude/settings.json (merge with existing).",
+  "_hooks_total": 7,
+  "_hooks_breakdown": "PreToolUse(4) + PostToolUse(1) + Stop(1) + PreCompact(1)",
   "hooks": {
     "PreToolUse": [
       {
@@ -63,6 +63,18 @@
           }
         ]
       }
+    ],
+    "PreCompact": [
+      {
+        "hooks": [
+          {
+            "_comment": "precompact-guard (v4.5): flush observations via learner before context compaction — prevents data loss in long sessions",
+            "type": "command",
+            "command": "bash ~/.claude/skills/_precompact-guard.sh",
+            "timeout": 10
+          }
+        ]
+      }
     ]
   }
 }

docs/rfc-v5-adaptive-thinking.md

@@ -0,0 +1,142 @@
+# RFC: Adaptive thinking in `/analyze-session`
+
+- **Status:** Draft
+- **Author:** Luis Salgado
+- **Created:** 2026-04-21
+- **Target release:** Sinapsis v5.0
+- **Supersedes:** nothing (new feature)
+- **Depends on:** Anthropic SDK >= 2026-04, env `ANTHROPIC_API_KEY`
+
+## Context
+
+Sinapsis v4.5 learner is fully deterministic: bash + node regex detectors over
+`observations.jsonl`. This is great for the hot path (Stop hook, PreToolUse inject),
+but the deep semantic pass — `/analyze-session` — is still only regex-backed.
+
+Claude Opus 4.7 ships with **adaptive thinking** (replacing the old
+`budget_tokens` extended-thinking knob) and native interleaved thinking between
+tool calls. For offline, on-demand analysis of a session this is a better tool
+than hand-written regex: clustering, contradictions across days, natural-language
+gotchas that regex can't see.
+
+## Goals
+
+1. `/analyze-session` can optionally call Claude Opus 4.7 with adaptive thinking
+   to produce higher-quality instinct proposals.
+2. **Zero regression** when the SDK path is disabled: the existing regex pipeline
+   remains the default and runs identically.
+3. Cost-bounded: one session = at most one API call, with a hard `effort` cap.
+4. No leakage of user data (observations are already scrubbed by
+   `observe_v3.py` — we rely on that).
+
+## Non-goals
+
+- Moving `_instinct-activator.sh` or `_session-learner.sh` to the SDK. Hot path
+  stays bash/node. LLM only in explicit `/analyze-session` flow.
+- Auto-apply of LLM proposals. Output still requires human `[A]ccept/[E]dit/[X]`.
+
+## Design
+
+### 1. Opt-in gate
+
+A new env var `SINAPSIS_LLM_ANALYZE=1` enables the SDK branch in
+`commands/analyze-session.md`. Default off. Users who do not export it or do not
+have `ANTHROPIC_API_KEY` see the current regex-only behaviour with zero network
+calls.
+
+### 2. Prompt shape
+
+```python
+response = client.messages.create(
+    model="claude-opus-4-7",
+    max_tokens=4096,
+    # Adaptive thinking replaces extended_thinking.budget_tokens (Opus 4.7)
+    thinking={"type": "adaptive", "display": "summarized"},
+    # Cache the fixed system block — docs for the instinct schema rarely change
+    system=[
+        {
+            "type": "text",
+            "text": SINAPSIS_SCHEMA_SYSTEM_PROMPT,  # ~800 tokens, stable
+            "cache_control": {"type": "ephemeral"},
+        }
+    ],
+    messages=[
+        {
+            "role": "user",
+            "content": json.dumps({
+                "observations": scrubbed_observations[-500:],
+                "existing_instincts": index_instincts,
+                "raw_proposals": proposals_from_session_learner,
+            }),
+        }
+    ],
+)
+```
+
+**Cache expectation:** the system block is byte-stable across every session.
+Second call within 5 min TTL = ~90 % input-token discount.
+
+### 3. Output contract
+
+The model must return JSON of the form:
+
+```json
+{
+  "proposals": [
+    {
+      "id": "kebab-case",
+      "domain": "security|git|nextjs|...",
+      "trigger_pattern": "regex",
+      "inject": "<= 500 chars",
+      "confidence": "HIGH|MEDIUM|LOW",
+      "reasoning": "one-liner for the human review"
+    }
+  ]
+}
+```
+
+`/analyze-session` validates the JSON, merges with existing regex proposals,
+deduplicates by `id`, and presents to the user. Any invalid JSON falls back to
+regex-only output.
+
+### 4. Failure handling
+
+- Missing `ANTHROPIC_API_KEY` → warn once, continue regex-only.
+- API error / timeout (30 s cap) → log to `_sinapsis-debug.log`, continue
+  regex-only. Never block the user.
+- Malformed JSON → discard LLM proposals, continue with regex.
+
+### 5. Architectural concern (open question)
+
+Today Sinapsis is *core-deterministic*. Introducing an SDK call — even gated by
+env var — opens a door to drift: future features could start relying on the LLM
+branch and silently degrade without it. Mitigation:
+
+- Keep the SDK branch strictly **additive** (merges with regex output, never
+  replaces it).
+- Add `test-v5-llm-fallback.sh` that exercises the `SINAPSIS_LLM_ANALYZE=0`
+  path and asserts bit-identical output to v4.5.
+- Document in README: "LLM analyze is a convenience layer. Sinapsis stays
+  functional with it turned off."
+
+## Rollout
+
+- Phase 1 (this RFC): ship opt-in `SINAPSIS_LLM_ANALYZE`. Collect telemetry on
+  cache-hit ratio and proposal quality for 2-3 weeks.
+- Phase 2: if cache-hit ratio > 60 % and proposal acceptance rate > 40 %, make
+  it default-on (still overridable).
+- Phase 3 (speculative, multi-agent v5): Managed Agents on Anthropic Cloud
+  fan out Scout(Sonnet 4.6) → Analyst(Sonnet 4.6) → Architect(Opus 4.7). Not
+  covered here.
+
+## Out of scope
+
+- Memory tool (`memory_20250818`) integration. Sinapsis already *is* a memory
+  system — adding the tool adds duplicate plumbing without a clear win. Revisit
+  only if multi-machine sync becomes a real requirement.
+- Context editing. Not applicable to `systemMessage` injection and adds cost.
+
+## Not doing
+
+- Haiku. Per operator preference, the Scout/Analyst/Architect blueprint skips
+  Haiku entirely; Sonnet 4.6 is the lowest tier we use.

install.sh

@@ -164,6 +164,7 @@ cp "$SCRIPT_DIR/core/_session-learner.sh" "$SKILLS_DIR/_session-learner.sh"
 cp "$SCRIPT_DIR/core/_project-context.sh" "$SKILLS_DIR/_project-context.sh"
 cp "$SCRIPT_DIR/core/_eod-gather.sh" "$SKILLS_DIR/_eod-gather.sh"
 cp "$SCRIPT_DIR/core/_dream.sh" "$SKILLS_DIR/_dream.sh"
+cp "$SCRIPT_DIR/core/_precompact-guard.sh" "$SKILLS_DIR/_precompact-guard.sh"
 cp "$SCRIPT_DIR/core/_generate-dashboard.py" "$SKILLS_DIR/_generate-dashboard.py"
 cp "$SCRIPT_DIR/core/_dashboard-template.html" "$SKILLS_DIR/_dashboard-template.html"
 
@@ -173,9 +174,10 @@ chmod +x "$SKILLS_DIR/_session-learner.sh"
 chmod +x "$SKILLS_DIR/_project-context.sh"
 chmod +x "$SKILLS_DIR/_eod-gather.sh"
 chmod +x "$SKILLS_DIR/_dream.sh"
+chmod +x "$SKILLS_DIR/_precompact-guard.sh"
 chmod +x "$SKILLS_DIR/_generate-dashboard.py" 2>/dev/null || true
 
-echo -e "${GREEN}  OK${NC} 5 hook scripts + dream cycle + dashboard generator installed"
+echo -e "${GREEN}  OK${NC} 6 hook scripts + dream cycle + dashboard generator installed"
 
 # ── Step 5b: Legacy file cleanup (v4.3.3) ──
 LEGACY_CLEANED=0