🧠 pi-memory

Persistent memory layer for pi-agent-core

A Write → Manage → Read memory system that gives your AI agent durable, semantic memory across sessions.

---

Agent runs → context grows → compaction → information lost → agent gets dumb
                                                ↑
                                       pi-memory fixes this

✨ Features

• 🔍 5-channel retrieval — FTS · fact-key · vector · HyDE · raw-message
• 🔀 Reciprocal Rank Fusion — intelligently merges results from all channels
• ⏳ Memory decay — half-life model keeps stale memories from polluting context
• 🔁 Supersession — newer facts automatically replace outdated ones
• 🧩 Consolidation — clusters episodic memories into durable facts
• 📦 Zero infrastructure — works out of the box with SQLite + sqlite-vec
• 🔌 pi-plugin — one-line integration via transformContext + subscribe
• 🛠️ Agent tools — remember / recall / forget for in-session control
• 📚 Wiki Knowledge Layer — compile raw documents into a searchable knowledge base

📦 Packages

Package	Description
engram-core	Core Write-Manage-Read engine
engram-store-sqlite	SQLite + sqlite-vec storage (default)
engram-store-postgres	PostgreSQL + pgvector storage (WIP)
engram-pi-plugin	pi-agent-core plugin integration
engram-tools	AgentTool definitions
engram-wiki	[New] Wiki Knowledge Layer — ingest, search, maintain
engram-wiki-store	[New] SQLite + filesystem storage for Wiki

🚀 Quick Start

npm install engram-core engram-store-sqlite

import { Engram } from "engram-core";
import { SqliteStore } from "engram-store-sqlite";

const store = new SqliteStore({ databasePath: "./memory.db" });

const engram = new Engram({
  store,
  scope: { project: "my-app", user: "alice" },
  writer: { llmProvider: myLlm },
  reader: { synthesize: true },
});

// Extract memories from a conversation
await engram.retain([
  { role: "user", content: "We use PostgreSQL 15 in production" },
  { role: "assistant", content: "Got it, I'll remember that." },
]);

// Recall with multi-channel search
const result = await engram.recall("what database are we using?");
console.log(result.answer); // "You use PostgreSQL 15 in production."

🔌 pi-plugin Integration

import { createEngramPlugin } from "engram-pi-plugin";

const plugin = createEngramPlugin({
  engram,
  recall:  { strategy: "every-turn", includeIdentity: true },
  retain:  { trigger: "on-turn-end" },
  consolidation: { trigger: "on-agent-end" },
});

const agent = createAgent({
  transformContext: plugin.transformContext,
  subscribe: plugin.subscriber,
});

🛠️ Agent Tools

import { createMemoryTools } from "engram-tools";

const agent = createAgent({
  tools: [...yourTools, ...createMemoryTools(engram)],
});

The agent can now call:

• engram_remember(content, type?) — store a memory
• engram_recall(query) — search memories
• engram_forget(memoryId) — archive a memory

🗂️ Memory Types

Type	Description	Has topicKey
fact	Stable info (preferences, versions, settings)	✅
instruction	Rules to follow	✅
event	Something that happened	❌
task	Pending to-do	❌

📡 Retrieval Channels

Channel	Method	Weight
fact-key	Exact topic key lookup	2.0
hyde	Hypothetical document embeddings	1.2
fts	Full-text search (Porter stemming)	1.0
vector	Cosine similarity	1.0
raw-message	Original transcript search	0.3

♻️ Memory Lifecycle

extracted → verified → classified → deduplicated → active
                                                      │
                                          ┌───────────┼──────────────┐
                                       decay       supersede    consolidate
                                          │                          │
                                        weak                    new fact /
                                          │                    instruction
                                       archived

🔧 No-LLM Mode

Without a llmProvider, retain() falls back to storing conversations directly as event memories. Recall still works via FTS — useful for testing or resource-constrained environments.

📖 API Reference

Write

Method	Description
retain(messages)	Extract and store memories from a conversation
remember(content, options?)	Manually store a single memory

Read

Method	Description
recall(query)	Multi-channel retrieval with optional synthesis
reflect(query)	Deep recall (higher topK)
identity()	Generate identity summary from top facts
criticalFacts(limit?)	Retrieve strongest active facts

Manage

Method	Description
consolidate()	Run decay + conflict detection + clustering
forget(id)	Archive a specific memory
forgetByTopic(key)	Archive all memories with a topic key
stats()	Memory counts by type/status
export() / import(data)	Backup and restore

🧪 Tests

npm test

All 142 tests pass across 11 test files.

---

📚 Wiki Knowledge Layer

The Wiki Knowledge Layer is a document-centric knowledge base inspired by Karpathy's LLM Wiki. While Memory stores ephemeral conversational context, Wiki compiles raw documents into structured, searchable knowledge pages.

Memory vs Wiki

Dimension	Memory	Wiki
Granularity	One sentence per item	One concept per page
Lifecycle	Decays + superseded	Persistent, accumulative
Write method	Auto-extracted from chat	Explicit ingest()
Retrieval speed	<200ms/turn	Seconds (on-demand)
Purpose	Agent knows you	Agent has domain knowledge

Quick Start

npm install engram-wiki engram-wiki-store

import { Wiki, DEFAULT_WIKI_SCHEMA } from "engram-wiki";
import { WikiSqliteStore } from "engram-wiki-store";

const store = new WikiSqliteStore({ databasePath: "./engram.db" });

const wiki = new Wiki({
  store,
  schema: DEFAULT_WIKI_SCHEMA,
  embeddingProvider: myEmbedder,
  llmProvider: myLlm,
  wikiDir: "./wiki",        // Markdown files for Obsidian / Git
  rawDir: "./raw",          // Source documents
});

// Ingest a document — idempotent, skips if content unchanged
const result = await wiki.ingest("./docs/architecture.md");
console.log(result.pagesCreated); // ["architecture-overview", "sqlite-vec"]

// Search with BM25 + vector + LLM reranking
const pages = await wiki.search({
  query: "how does the caching layer work?",
  topK: 3,
  rerank: true,
});

// Grounded Q&A
const answer = await wiki.ask("What database do we use?");
console.log(answer.answer);
console.log(answer.sources); // [{page: WikiPage, relevantChunk: string}]

Wiki + Memory in pi-plugin

import { createEngramPlugin } from "engram-pi-plugin";

const plugin = createEngramPlugin({
  engram,
  wiki,                        // Optional: adds Wiki injection
  wikiSearch: {
    strategy: "auto",           // Auto-detect when to query Wiki
    maxTokens: 1500,            // Budget for injected wiki context
    triggerKeywords: ["how", "what", "explain"],
  },
});

When strategy: "auto", the plugin detects question-like messages and injects relevant Wiki pages before the LLM call, sandwiched between Memory context and the user message.

Wiki Agent Tools

import { createWikiTools } from "engram-tools";

const agent = createAgent({
  tools: [...createMemoryTools(engram), ...createWikiTools(wiki)],
});

New tools available:

• engram_wiki_search(query, pageTypes?) — search the knowledge base
• engram_wiki_ingest(filePath, force?) — compile a document into the wiki

Wiki Page Types

Type	Purpose	Storage
summary	Compiled overview of one source document	wiki/summaries/
concept	Cross-document technical concept	wiki/concepts/
entity	Person / project / product entity	wiki/entities/
synthesis	Multi-source comparative analysis	wiki/synthesis/
index	Auto-generated directory page	wiki/index.md

Ingest Pipeline

Raw file (.md / .txt)
    │
    ├─ 1. Parse + hash (idempotent — skip if unchanged)
    ├─ 2. Chunk by Markdown heading structure (target 900 tokens)
    ├─ 3. Embed chunks (sqlite-vec)
    ├─ 4. LLM compile → Summary page + entities/concepts
    ├─ 5. Resolve [[wikilinks]] and build link graph
    └─ 6. Write .md files (Obsidian-compatible) + SQLite

Health Check

const issues = await wiki.lint();
console.log(issues.brokenLinks);     // [[wikilinks]] pointing nowhere
console.log(issues.orphanPages);     // Pages with no incoming links
console.log(issues.stalePages);      // Source changed but page not recompiled
console.log(issues.missingCrossRefs); // Suggested links based on content overlap

---

MIT