diff --git a/changelog/unreleased/2026-08-07-memory.md b/changelog/unreleased/2026-08-07-memory.md new file mode 100644 index 00000000..1608cb3e --- /dev/null +++ b/changelog/unreleased/2026-08-07-memory.md @@ -0,0 +1,55 @@ +# Memory: what an agent keeps between Sessions + +An agent now has a long-term store it maintains itself: Markdown notes under `agent_state/memory/`, kept in two scopes — one for the user and one per Workspace — each with its own `MEMORY.md` index that enters the context, topic bodies read on demand. It covers what a later Session cannot re-derive from the Workspace — who the user is and their standing preferences, project decisions with their reasons, conventions, entry points into external systems. + +Memory is not context compaction. Compaction preserves one Session's short-term working state; Memory is what survives the Session ending. + +## Scope and layout + +There are two scopes, both belonging to one agent and never shared with another: + +- **User scope** (`memory/user/`) — what stays true wherever the agent works: who the user is, their standing preferences, reference material not tied to one codebase. Every session reads it. +- **Workspace scope** (`memory//`) — facts about one Workspace: project decisions, feedback received, pointers into its external systems. Sessions of one agent in one Workspace share it; different Workspaces keep their topic files apart. + +Each scope carries its own `MEMORY.md` index — one line per memory, `- [Title](file.md) — hook`, links relative to the scope directory — and different agents never share Memory even in the same Workspace. + +```text +agent_state/memory/ +├── user/ # user scope (created with the agent) +│ ├── MEMORY.md # this scope's index +│ └── prefers-pnpm.md +└── my-app-a81f32c4/ # created on the first Session in that Workspace + ├── .workspace # the Workspace path this key stands for + ├── MEMORY.md + └── testing-conventions.md +``` + +`user` is safe to reserve because every generated workspace memory key is `-<8 hex>` and so always carries a hyphen — a hyphen-free name can never be produced. + +The workspace memory key is `-<8 hex of the real path's sha256>`. Identity is the directory itself, with no dependence on Git: two symlinks to one directory resolve to a single key, and moving or renaming a directory makes it a new Workspace — the old Memory stays on disk under the old key rather than following a path that no longer exists. + +A **temporary** Workspace gets no Workspace scope. One is allocated per session, so no later session would ever run there to read it back — memory keyed off it would be write-only storage. Such a session still gets the user scope, which is where anything it learns belongs anyway. The test is the directory's location — anything under an agent's `workspaces/` — rather than whether the caller passed a Workspace explicitly, because a subagent inherits its parent's Workspace as an explicit argument, temporary ones included. + +A topic file is a semantic subject, not one per Task, Session or date, and declares `name` (kebab-case, matching the file name) / `description` (one line used to decide relevance during recall) / `updated_at` in frontmatter — which scope a memory belongs to is expressed by its directory, so there is no type field. In the body, `[[name]]` links related memories and corrections / decisions carry **Why:** and **How to apply:** lines. Never saved: credentials, task progress, unconfirmed guesses, or facts the code and Git history already state. + +## What reaches the model + +Only the indexes, through the template's `{{MEMORY}}` placeholder: it expands to `memory.prompt` — what Memory is for, the save mechanics in template-example form, then a `## User memory` section with its index (`{{USER_MEMORY_INDEX}}`) — plus `memory.workspace_prompt` (a `## Workspace memory` section with `{{WORKSPACE_MEMORY_INDEX}}`) when the Session runs in a persistent Workspace, so a temporary Workspace is never told about a scope it does not have. Both prompts are per-agent config, editable on the Memory tab, organized by Markdown headings like the template's other sections. The `User Memory Dir` line is the literal pattern `/agents//agent_state/memory/user`, resolvable from the Environment section; the `Workspace Memory Dir` line renders resolved via `{{WORKSPACE_MEMORY_DIR}}`, because its final segment — the workspace memory key — is a path hash the model could never compose itself. + +A blank index injects an explicit "nothing saved yet" note, and injection is capped at 200 lines per scope, then at 25,000 characters total as a backstop for long-line indexes — past a cap a truncation note tells the model to open the full `MEMORY.md` itself. The default Memory prompt declares the line cap and asks for index lines under ~150 characters; the character backstop lives only in code. + +A template without `{{MEMORY}}` injects nothing — an agent created before Memory, for instance. The Memory tab reports this and offers inserting the placeholder (before `# Environment`, the position the default template gives it) as an explicit one-click action; nothing is ever spliced in automatically. The assembled prompt is recorded in `session_meta`. + +Reading, writing and deduplicating are the model's own work through the ordinary file tools — the Harness decides where Memory lives and keeps writes inside it, nothing more. + +## Managing it in the Web App + +Agent settings gain a **Memory** tab between Prompt and Runtime: the switch on top (written immediately, not joining the tab's Save, so turning Memory off never drags an unrelated half-finished edit along), then every memory grouped by scope — user memory first, then one group per Workspace titled by its directory basename with the full `.workspace` path beside it, newest activity first. Groups collapse on header click, the same convention as the skill library's, and the collapse state is remembered in the browser per user × Project × Agent. Rows show the memory's name, description and date. + +The tab also edits the two memory prompts in place, below the groups. Rows are deliberately read-only, with icon-only actions like the skills tab's. **View** (eye) opens the rendered body (frontmatter stripped — the header already shows those fields) in a right drawer on desktop and a bottom sheet on narrow screens (the chat page panels' interaction, half / full snap points); **Delete** (trash) confirms, removes the file and mechanically drops its `]()` lines from that scope's `MEMORY.md`, so the index never lists a file that is gone; **Edit** (pencil) opens a bridging modal first — a requirement field and a live preview of the generated prompt, the same shape as the skill import modal — then jumps to a new chat with this agent through the same draft-cache route, the prompt prefilled and the requirement filled in (or left trailing to complete in the composer). A Workspace memory's edit chat also pins that Workspace, so the editing Session is injected with the very index it is about to change. + +Each group header also carries an **Add** entry, a ghost text action to the collapse arrow's left — the models page's per-group convention — whose modal is the edit modal's shape with a required content-or-source field (pasted text, a file path or a URL; the agent reads sources itself), bridging into a new chat where the agent organizes the content into that scope. Both bridge drafts stay deliberately minimal, naming only the target and the ask — the save mechanics already live in the agent's Memory prompt. And the agents list cards' stat line shows the memory count behind a brain icon next to the other counts, deep-linking to the Memory tab. + +Turning Memory off keeps every file and leaves the tab fully usable; it only stops Memory from entering the context and from preparing directories for new Sessions. + +The API is under `/api/projects/:p/agents/:a/memory` — overview, per-scope file listing, file read, file delete, and the idempotent placeholder insert; deliberately no content write or rename, since content changes go through chat. The memory prompts ride the ordinary config route. It never accepts a path: a file is addressed by agent, scope key and a name inside that scope, each validated and then re-checked for containment after resolution. diff --git a/changelog/unreleased/README.md b/changelog/unreleased/README.md index 1bbef0a9..8faa3ebf 100644 --- a/changelog/unreleased/README.md +++ b/changelog/unreleased/README.md @@ -13,3 +13,5 @@ - [2026-08-06] Models: Thinking Machines Lab's Inkling joins on OpenRouter and Fireworks AI, Fireworks AI gains DeepSeek V4 Flash 0731, and the OpenRouter + SiliconFlow GLM-5.1 gateway listings are delisted (Z.AI direct stays; existing Project configs unaffected); OpenRouter prices refreshed from the models API on 2026-08-07 (Inkling cached input $0.17, four drifted rows corrected); agenthub-models skill v11. ([details](2026-08-06-model-catalog-inkling-dsv4-flash-0731.md)) - [2026-08-06] Release tooling: repo versions realigned with the shipped 0.2.1, and the release workflow now refuses a tag push whose version does not match `package.json` (the drift that made every dev build nag about updates); the bump is documented as a release-prep step. ([details](2026-08-06-release-version-guard.md)) + +- [2026-08-07] Memory: an agent keeps long-term notes between Sessions under `agent_state/memory/` — a user scope (`memory/user/`, read by every Session) plus one directory per Workspace, each with its own `MEMORY.md` index; only the indexes enter the context (capped at 200 lines / 25,000 chars per scope), bodies read on demand. A temporary Workspace gets the user scope only. The template's `{{MEMORY}}` placeholder expands to the agent's own `memory.prompt` (+ `memory.workspace_prompt` in a persistent Workspace), both editable on the Memory tab; a pre-Memory template injects nothing until its one-click placeholder insert. Agent settings gain a Memory tab: memories grouped by scope with view / delete (index lines pruned) / edit-via-chat. ([details](2026-08-07-memory.md)) diff --git a/packages/core/src/agent.ts b/packages/core/src/agent.ts index 1872e82b..99e64ec3 100644 --- a/packages/core/src/agent.ts +++ b/packages/core/src/agent.ts @@ -22,6 +22,7 @@ import { loadProjectConfig, projectDir, goalFilePath, + resolveSessionMemory, resolveModelRef, sessionScratchpadDir, systemConfigPath, @@ -260,12 +261,23 @@ export class Agent { this.state.projectId, this.state.agentId, ); + // Memory for this Session: null when the Agent has Memory off; a temporary Workspace gets + // the user scope only (nothing written against it could ever be read back). Reads the + // current indexes every time, like the vault and Skills above. + const memory = await resolveSessionMemory({ + root: this.state.root, + projectId: this.state.projectId, + agentId: this.state.agentId, + workspaceDir, + enabled: this.state.systemConfig.memory?.enabled !== false, + }); // The assembled system prompt goes both to the LLM and into session_meta (so the // Trace can audit the actual effective value). The vault only injects **key names** // into the prompt (so the model knows which API keys are available); values only // go into the subprocess environment. Skills only inject metadata (name and - // description); the model reads the body on demand via shell. + // description); the model reads the body on demand via shell. Memory likewise injects + // only its index; topic bodies are read on demand. const systemPrompt = assembleSystemPrompt( this.state, sessionEnvironment(workspaceDir, sessionId, { @@ -276,6 +288,7 @@ export class Agent { }), Object.keys(vault), installedSkills, + memory, ); const rt = await this.buildRuntime({ diff --git a/packages/core/src/state/agent-state.ts b/packages/core/src/state/agent-state.ts index 120edba0..6506bd0a 100644 --- a/packages/core/src/state/agent-state.ts +++ b/packages/core/src/state/agent-state.ts @@ -29,6 +29,16 @@ import { PROVIDER_PLACEHOLDER, MODEL_ID_PLACEHOLDER, DATE_PLACEHOLDER, + MEMORY_PLACEHOLDER, + WORKSPACE_MEMORY_DIR_PLACEHOLDER, + WORKSPACE_MEMORY_INDEX_PLACEHOLDER, + USER_MEMORY_INDEX_PLACEHOLDER, + MEMORY_INDEX_EMPTY_NOTE, + MEMORY_INDEX_MAX_LINES, + MEMORY_INDEX_MAX_CHARS, + DEFAULT_MEMORY_PROMPT, + DEFAULT_MEMORY_WORKSPACE_PROMPT, + type MemoryConfig, agentStateVersion, defaultAgentsMd, defaultSystemConfig, @@ -40,6 +50,7 @@ import { type SystemConfig, } from "./default-config.js"; import { builtinProjectAgentPresets, type AgentPreset } from "./builtin-agents.js"; +import { ensureUserMemoryDir, type SessionMemory } from "./memory.js"; import { provisionExampleBenchmark } from "./example-benchmark.js"; import { agentsMdPath, @@ -151,7 +162,9 @@ export async function loadOrInitAgentState(opts?: { await Promise.all([ fs.mkdir(stateDir, { recursive: true }), fs.mkdir(toolsDir(root, projectId, agentId), { recursive: true }), - fs.mkdir(memoryDir(root, projectId, agentId), { recursive: true }), + // Creates memory/user/ (and memory/ above it) with an empty MEMORY.md, so the User scope + // exists from the Agent's first day; Workspace scopes appear at Session creation. + ensureUserMemoryDir(root, projectId, agentId), fs.mkdir(skillsDir(root, projectId, agentId), { recursive: true }), fs.mkdir(scratchpadDir(root, projectId, agentId), { recursive: true }), ]); @@ -269,6 +282,81 @@ function vaultKeysList(keys: string[]): string { return keys.map((key) => `- ${key}`).join("\n"); } +/** + * An index for injection: the trimmed `MEMORY.md` content, or the empty note so the model reads + * "nothing saved" instead of a blank line. Injection is capped at `MEMORY_INDEX_MAX_LINES` + * lines (one memory per line by convention), then at `MEMORY_INDEX_MAX_CHARS` as a backstop + * for indexes whose few lines are enormous — past a cap the rest is replaced by a note telling + * the model to open the full file, and the file itself is never touched. + */ +function indexForInjection(index: string): string { + const trimmed = index.trim(); + if (trimmed.length === 0) return MEMORY_INDEX_EMPTY_NOTE; + const totalLines = trimmed.split("\n").length; + let kept = trimmed.split("\n").slice(0, MEMORY_INDEX_MAX_LINES).join("\n"); + if (kept.length > MEMORY_INDEX_MAX_CHARS) { + // Cut at a line boundary; only a single line exceeding the cap on its own is cut mid-line. + const cut = kept.lastIndexOf("\n", MEMORY_INDEX_MAX_CHARS); + kept = kept.slice(0, cut > 0 ? cut : MEMORY_INDEX_MAX_CHARS); + } + if (kept.length === trimmed.length) return trimmed; + const keptLines = kept.split("\n").length; + const reason = + keptLines < totalLines + ? `showing ${keptLines} of ${totalLines} lines` + : `showing the first ${MEMORY_INDEX_MAX_CHARS} characters`; + return `${kept}\n(index truncated: ${reason} — open MEMORY.md for the rest)`; +} + +/** + * The `{{MEMORY}}` replacement value: the Agent's own `memory.prompt` (the User scope and its + * index, which every Session has), plus `memory.workspace_prompt` when the Session also runs + * in a persistent Workspace. An empty string when this Session has no Memory (disabled) or + * when every half that would render is emptied. Both prompts are per-Agent config, editable + * on the Web App's Memory tab. + * + * The two blocks are separate config keys because substitution has no conditionals: a + * temporary Workspace must never be handed the Workspace scope's section (its directory line + * and the scope-choice rule), so that half is simply not appended there. + * + * Every word of the block comes from `system_config.yaml`; the only text this function can add + * is `MEMORY_INDEX_EMPTY_NOTE` (via `indexForInjection`, which also caps the index). The only + * injection points are the two indexes and the Workspace directory + * (`{{WORKSPACE_MEMORY_DIR}}`, whose key segment is a hash the model could not compose + * itself) — the User directory stays a literal pattern in the prompt. Topic bodies are never + * injected — the indexes say what exists, and the model opens what it needs. + */ +function memorySection( + config: MemoryConfig | undefined, + memory: SessionMemory | null | undefined, +): string { + if (!memory) return ""; + // Missing keys fall back to the built-in defaults — matching compaction and the config DTO — + // so an Agent whose yaml predates Memory injects the very prompts the Memory tab shows it. + // An explicitly emptied half drops that half alone (`??`, not `||`): the two are edited + // independently on the Memory tab, so clearing one must never silence the other. + const promptText = config?.prompt ?? DEFAULT_MEMORY_PROMPT; + const workspacePromptText = config?.workspace_prompt ?? DEFAULT_MEMORY_WORKSPACE_PROMPT; + const substituteUser = (text: string): string => + text.split(USER_MEMORY_INDEX_PLACEHOLDER).join(indexForInjection(memory.userIndex)); + + const userBlock = substituteUser(promptText).trim(); + const workspace = memory.workspace; + const workspaceBlock = + workspace && workspacePromptText ? substituteUser(workspacePromptText).trim() : ""; + const joined = + userBlock && workspaceBlock ? `${userBlock}\n\n${workspaceBlock}` : userBlock || workspaceBlock; + // The Workspace placeholders substitute over the whole joined block, so one written into + // the main prompt resolves too (with a real value in a persistent Workspace, blank + // otherwise) instead of leaking literally. + return joined + .split(WORKSPACE_MEMORY_INDEX_PLACEHOLDER) + .join(workspace ? indexForInjection(workspace.index) : "") + .split(WORKSPACE_MEMORY_DIR_PLACEHOLDER) + .join(workspace?.dir ?? "") + .trim(); +} + /** * Guards a Skill auxiliary-file path (relative to the skill directory) before it's written: * rejects empty, absolute, backslash-bearing, and any `..`-segment path so a file entry can never @@ -462,15 +550,21 @@ function withShellLineFallback( * wrapper text such as `[developer_instructions]` and the # Vault / # Skills statements are * written directly into the system Prompt template itself (the Prompt is fully * transparent and editable via `system_config.yaml`). Other files in Agent State / Workspace are - * never auto-injected. Sole exception: on win32 a template without `{{SHELL}}` gets a `- Shell:` - * line injected at render time (see `withShellLineFallback`). + * never auto-injected. Sole exception: on win32 a template without `{{SHELL}}` gets a + * `- Shell:` line injected at render time (see `withShellLineFallback`). * * `{{VAULT_KEYS}}` is replaced with the vault key-name list (an empty string if empty/not * provided): this lets the model know which APIs requiring a key it can call; values are never * injected. `{{SKILL_METADATA}}` is replaced with the installed Skills' metadata lines (an empty - * string if empty/not provided). A custom template that removes a placeholder gets no - * corresponding content injected. `{{PROJECT_DIR}}` resolves to the Project directory — - * the app data root the default prompt labels "App Data Dir". + * string if empty/not provided). `{{MEMORY}}` expands to the rendered `memory.prompt` block + * (plus `memory.workspace_prompt` in a persistent Workspace), and to an empty string when + * Memory is disabled — only those blocks' own `{{USER_MEMORY_INDEX}}` / + * `{{WORKSPACE_MEMORY_INDEX}}` carry Memory content (indexes capped, topic bodies always read + * on demand), and `{{WORKSPACE_MEMORY_DIR}}` renders the Workspace Memory directory right in + * the workspace half. A custom template that removes a placeholder gets no corresponding + * content injected — a template without `{{MEMORY}}` injects no Memory, and the Web App's + * Memory tab offers inserting the placeholder explicitly. `{{PROJECT_DIR}}` resolves to the + * Project directory — the app data root the default prompt labels "App Data Dir". * Docs: /docs/configuration § "System prompt placeholders". */ export function assembleSystemPrompt( @@ -478,6 +572,7 @@ export function assembleSystemPrompt( sessionEnvironment?: SessionEnvironmentValues, vaultKeys?: string[], skillMetadata?: SkillMetadata[], + memory?: SessionMemory | null, ): string { const template = state.systemConfig.system_prompt; const assembled = template @@ -507,6 +602,11 @@ export function assembleSystemPrompt( .join(sessionEnvironment?.shell ?? "") .split(DATE_PLACEHOLDER) .join(sessionEnvironment?.date ?? "") + // {{MEMORY}} expands last: everything the Memory block carries (index lines the model wrote + // included) lands after the other placeholders were consumed, so index content can never + // smuggle a {{VAULT_KEYS}}-style token into a second expansion. + .split(MEMORY_PLACEHOLDER) + .join(memorySection(state.systemConfig.memory, memory)) .trim(); return withShellLineFallback(assembled, template, sessionEnvironment); } diff --git a/packages/core/src/state/default-config.ts b/packages/core/src/state/default-config.ts index 2766cadf..605b2668 100644 --- a/packages/core/src/state/default-config.ts +++ b/packages/core/src/state/default-config.ts @@ -36,6 +36,25 @@ export const OS_VERSION_PLACEHOLDER = "{{OS_VERSION}}"; /** The shell exec_command runs (`bash` on POSIX; on Windows whatever shell.ts resolved), so the model knows which command syntax to write. */ export const SHELL_PLACEHOLDER = "{{SHELL}}"; export const DATE_PLACEHOLDER = "{{DATE}}"; +/** + * Expands to the rendered `memory.prompt` block, plus `memory.workspace_prompt` when the + * Session runs in a persistent Workspace; an empty string when Memory is off. A template + * without this placeholder injects no Memory at all — the Web App's Memory tab offers + * inserting it as an explicit action, nothing is spliced in automatically. + */ +export const MEMORY_PLACEHOLDER = "{{MEMORY}}"; +/** + * Inside `memory.workspace_prompt` only: the absolute path of the current Workspace's Memory + * directory (`…/memory/`). The key half is a hash the model could never + * derive itself, so the resolved directory is rendered right where the prompt names it — + * the User section's directory stays a literal `/…` pattern, resolvable from + * the Environment section. + */ +export const WORKSPACE_MEMORY_DIR_PLACEHOLDER = "{{WORKSPACE_MEMORY_DIR}}"; +/** Inside `memory.workspace_prompt` only: the content of the current Workspace scope's `MEMORY.md` index (capped, see MEMORY_INDEX_MAX_LINES / MEMORY_INDEX_MAX_CHARS). */ +export const WORKSPACE_MEMORY_INDEX_PLACEHOLDER = "{{WORKSPACE_MEMORY_INDEX}}"; +/** Inside either Memory prompt: the content of the User scope's `MEMORY.md` index (capped, see MEMORY_INDEX_MAX_LINES / MEMORY_INDEX_MAX_CHARS). */ +export const USER_MEMORY_INDEX_PLACEHOLDER = "{{USER_MEMORY_INDEX}}"; /** * Context compaction config (the `compaction` section of `system_config.yaml`). @@ -52,6 +71,100 @@ export interface CompactionConfig { prompt?: string; } +/** + * Memory config (the `memory` section of `system_config.yaml`). Both prompts are editable on + * the Web App's Memory tab and rendered into the template's `{{MEMORY}}` placeholder. + * Docs: /docs/configuration § "Memory". + */ +export interface MemoryConfig { + /** Whether Memory enters the model context and its directories are prepared; defaults to true. */ + enabled?: boolean; + /** + * The always-injected half of the `{{MEMORY}}` block: what Memory is for, the save mechanics, + * and the User scope with its index — carrying the `{{USER_MEMORY_INDEX}}` injection point + * (the User directory is literal text, not a placeholder). Defaults to the built-in value. + */ + prompt?: string; + /** + * Appended to `prompt` only when the Session runs in a persistent Workspace: the Workspace + * scope, its index and the rule for choosing between the two — carrying + * `{{WORKSPACE_MEMORY_INDEX}}` and the `{{WORKSPACE_MEMORY_DIR}}` directory. A separate key + * rather than a conditional inside `prompt` because substitution has no conditionals — a + * temporary Workspace would otherwise be told about a scope it does not have. + */ + workspace_prompt?: string; +} + +/** Stands in for an index placeholder when the `MEMORY.md` does not exist yet or is blank — the model is told the store is empty rather than being handed nothing. */ +export const MEMORY_INDEX_EMPTY_NOTE = "(the index is empty — nothing has been saved yet)"; + +/** + * Cap on injected index lines per scope (one memory per line by convention), so a runaway + * `MEMORY.md` cannot flood the context. Only the injection is capped — the file on disk is + * never touched — and a truncation note tells the model to open the full index itself. + */ +export const MEMORY_INDEX_MAX_LINES = 200; + +/** + * Character backstop on an injected index, applied after the line cap: catches the long-line + * index the line cap alone misses (a file under 200 lines can still be arbitrarily large). + * Deliberately code-only — the default prompt teaches per-line brevity (~150 characters) + * instead of quoting this number; when the backstop does fire, the truncation note says so. + */ +export const MEMORY_INDEX_MAX_CHARS = 25_000; + +/** + * Built-in default Memory Prompt: the always-injected half of the `{{MEMORY}}` block, in + * template-example form — a fenced frontmatter example, what is worth saving, the index + * contract (the line cap and a per-line length hint, so the model keeps the index short + * before ever hitting the code-side caps) and the hygiene rules, then the User scope section + * with its index. Stored + * per-Agent in `system_config.yaml` and editable on the Web App's Memory tab. The User + * directory is literal text in the template's angle-bracket convention (resolved from the + * Environment section by the model, like the Skills paths) — the only injection point is the + * index itself. + */ +export const DEFAULT_MEMORY_PROMPT = `# Memory +Your long-term record across sessions: Markdown files you maintain with the file tools, in the memory directories named below (they already exist). One file per fact, with frontmatter: + +\`\`\`markdown +--- +name: +description: +updated_at: +--- + + +\`\`\` + +Worth saving: who the user is (role, expertise, preferences) and how they want you to work, with the why; ongoing work, goals and constraints not derivable from the code; pointers to external resources. + +Each directory's \`MEMORY.md\` is its index, injected below: one line per memory, under ~150 characters (\`- [Title](file.md) — hook\`), no content, updated in the same round as the file — deletions included. Only the first ${MEMORY_INDEX_MAX_LINES} lines of an index are injected — keep it well under that: merge overlapping entries, drop stale ones, move detail into the topic files. Before saving, check the index and update the file that already covers the subject instead of duplicating; delete memories that prove wrong. Never save what code, config or git history already states, task progress, secrets, unconfirmed guesses, or transcript excerpts — if asked to, save the non-obvious part instead. Memory is readable by everyone who can reach this agent: no sensitive personal data. + +## User memory +What holds wherever you work; every one of your sessions reads it. + +User Memory Dir: \`/agents//agent_state/memory/user\` + +Index: +{{USER_MEMORY_INDEX}}`; + +/** + * Built-in default for the Workspace half of the `{{MEMORY}}` block, appended to + * `memory.prompt` only when the Session runs in a persistent Workspace. The rule for choosing + * between the two scopes lives here on purpose: a Session in a temporary Workspace has one + * scope and no choice to make, so it never sees the rule at all. The directory is rendered in + * place via `{{WORKSPACE_MEMORY_DIR}}` — its final segment is a path hash the model could not + * compose from Environment values the way it can the User directory. + */ +export const DEFAULT_MEMORY_WORKSPACE_PROMPT = `## Workspace memory +Facts about the workspace you are working in now. What would still hold in a different project goes in user memory; when unsure, write here. + +Workspace Memory Dir: \`{{WORKSPACE_MEMORY_DIR}}\` + +Index: +{{WORKSPACE_MEMORY_INDEX}}`; + /** * System-level config for Agent State, serialized as `system_config.yaml`. * Docs: /docs/configuration § "Agent config". @@ -78,6 +191,8 @@ export interface SystemConfig { }; /** Context compaction (enabled by default, max_context_length 128k, mode summarize). */ compaction?: CompactionConfig; + /** Memory (enabled by default; only reaches the prompt through the template's `{{MEMORY}}` placeholder). */ + memory?: MemoryConfig; tools?: { /** Built-in system tool configuration (per-entry fields incl. the `call_description` toggle live on ToolDefinitionConfig). */ builtin?: ToolDefinitionConfig[]; @@ -144,9 +259,11 @@ The vault holds this agent's per-agent secrets (agent_state/.vault.toml). Each e {{VAULT_KEYS}} # Skills -Skills are reusable instruction packages at /agents//agent_state/skills//SKILL.md. When a task matches one below, or the user asks for one (the message may start with a [use_skills] block naming them), read that SKILL.md in full with read_file, then follow it. If a request names a skill without a concrete task, ask the user what they need first. +Skills are reusable instruction packages at \`/agents//agent_state/skills//SKILL.md\`. When a task matches one below, or the user asks for one (the message may start with a [use_skills] block naming them), read that SKILL.md in full with read_file, then follow it. If a request names a skill without a concrete task, ask the user what they need first. {{SKILL_METADATA}} +{{MEMORY}} + # Environment - Platform: {{PLATFORM}} - OS Version: {{OS_VERSION}} @@ -159,6 +276,26 @@ Skills are reusable instruction packages at /agents//age - Model ID: {{MODEL_ID}} - Session ID: {{SESSION_ID}}`; +/** Whether a template carries the `{{MEMORY}}` placeholder — without it no Memory is injected. */ +export function hasMemoryPlaceholder(template: string): boolean { + return template.includes(MEMORY_PLACEHOLDER); +} + +/** + * Inserts the `{{MEMORY}}` placeholder into a template: before the `# Environment` heading + * (the position the default template gives it), else appended at the end. Idempotent — a + * template already carrying it is returned unchanged. This is the explicit adoption path for + * Agents created before Memory shipped (the Web App's Memory tab offers it); nothing ever + * inserts automatically. + */ +export function insertMemoryPlaceholder(template: string): string { + if (template.includes(MEMORY_PLACEHOLDER)) return template; + const heading = /^#+ Environment[ \t]*$/m.exec(template); + return heading + ? `${template.slice(0, heading.index)}${MEMORY_PLACEHOLDER}\n\n${template.slice(heading.index)}` + : `${template.trimEnd()}\n\n${MEMORY_PLACEHOLDER}\n`; +} + /** * Built-in default compaction Prompt (summarize mode): tells the model the summary will * replace the transcript as its only record (so it must include everything needed to @@ -510,6 +647,11 @@ export function defaultSystemConfig(): SystemConfig { mode: "summarize", prompt: DEFAULT_COMPACTION_PROMPT, }, + memory: { + enabled: true, + prompt: DEFAULT_MEMORY_PROMPT, + workspace_prompt: DEFAULT_MEMORY_WORKSPACE_PROMPT, + }, tools: { builtin: defaultBuiltinTools(), mcpServers: [], diff --git a/packages/core/src/state/index.ts b/packages/core/src/state/index.ts index aacd1c3a..64bedf97 100644 --- a/packages/core/src/state/index.ts +++ b/packages/core/src/state/index.ts @@ -10,6 +10,7 @@ export * from "./model-catalog.js"; export * from "./project-config.js"; export * from "./agent-state.js"; export * from "./agent-vault.js"; +export * from "./memory.js"; export * from "./example-benchmark.js"; // Skill library types and frontmatter parser (from the skills package; server reuses the same implementation via core). diff --git a/packages/core/src/state/memory.ts b/packages/core/src/state/memory.ts new file mode 100644 index 00000000..c8db5fa7 --- /dev/null +++ b/packages/core/src/state/memory.ts @@ -0,0 +1,336 @@ +/** + * Memory: the Agent's long-term notes across Sessions. + * + * Memory keeps what cannot be re-derived from the current Workspace or its code history — + * user feedback, project decisions, working conventions, entry points into external systems. + * It is **not** context compaction: compaction preserves one Session's short-term working + * state, Memory preserves what later Sessions need. + * + * There are two scopes, both belonging to one Agent and never shared with another: + * + * - **User scope** (`memory/user/`) — what stays true wherever the Agent works: who the user + * is, their standing preferences, reference material not tied to one codebase. Every + * Session reads it, including one running in a temporary Workspace, which has no other + * place to write. + * - **Workspace scope** (`memory//`) — facts about one Workspace. Sessions of + * the same Agent in the same Workspace share it; different Workspaces keep their topic files + * apart. A Session in a temporary Workspace gets no Workspace scope at all. + * + * Each scope directory carries its own `MEMORY.md` index; only the indexes enter the model + * context, topic bodies are read on demand. Memory lives in Agent State, so it travels with + * export / import / snapshots and is visible to every Project member who can reach the Agent. + * + * On-disk layout (`agent_state/memory/`): + * + * memory/ + * ├── user/ # User scope (no marker: it stands for no path) + * │ ├── MEMORY.md # this scope's index + * │ └── .md + * └── / + * ├── .workspace # the Workspace path this key stands for + * ├── MEMORY.md + * └── .md # frontmatter + body, semantic topics (not per Task/date) + * + * The Harness only decides *where* Memory lives, keeps writes inside that directory, and + * injects the indexes into the prompt; the model owns the semantics — what is worth keeping, + * how topics are split, and how the indexes are maintained — using the ordinary file tools. + * Docs: /docs/configuration § "Memory". + */ +import { createHash } from "node:crypto"; +import fs from "node:fs/promises"; +import path from "node:path"; +import { agentsDir, memoryScopeDir } from "./paths.js"; + +/** Per-scope index file name, also excluded from a scope's topic listing. */ +export const MEMORY_INDEX_FILENAME = "MEMORY.md"; + +/** + * Directory name of the User scope under `memory/`, and its scope key throughout the API — it + * sits alongside the Workspace directories and is read through the same code paths. + * + * The name is safe to reserve because a generated workspace key is always + * `-<8 hex>` and therefore always contains a hyphen: a name without one can + * never be produced by `workspaceMemoryKeyForRealPath`, so no Workspace can ever collide with + * it. (A Workspace directory literally named `user` yields `user-`, not `user`.) + */ +export const USER_SCOPE_KEY = "user"; + +/** + * Marker file inside a Workspace Memory directory recording the Workspace path the key was + * derived from. The key alone is a hash and cannot be read back into a path, so this is what + * lets the Web App label a Workspace; it is a dotfile so it never shows up as a topic. + */ +export const WORKSPACE_MARKER_FILENAME = ".workspace"; + +/** Length of the path hash suffix in a workspace key: 32 bits of a sha256, enough that two Workspaces on one machine practically never collide. */ +const KEY_HASH_LENGTH = 8; + +/** Cap on the readable part of a workspace key, so a deeply named directory can't produce an unwieldy path. */ +const KEY_BASE_MAX_LENGTH = 40; + +/** Frontmatter of one Memory topic file. */ +export interface MemoryTopicMetadata { + /** Display name; falls back to the file name when the frontmatter omits it. */ + name: string; + /** One line telling a reader whether the body is worth opening. */ + description: string; + /** Last-updated date as written in the file (`YYYY-MM-DD` by convention, not parsed). */ + updatedAt?: string; +} + +/** The Workspace scope of one Session: absent when the Session runs in a temporary Workspace. */ +export interface SessionWorkspaceMemory { + /** Workspace memory key — the `memory/` subdirectory name and the API's scope key. */ + key: string; + /** Absolute path of the Workspace's topic directory — used by the server and Web App, and the `{{WORKSPACE_MEMORY_DIR}}` value in the workspace Memory prompt. */ + dir: string; + /** Content of this scope's `MEMORY.md` (the `{{WORKSPACE_MEMORY_INDEX}}` value; empty string when blank). */ + index: string; +} + +/** + * The Memory binding of one Session, as resolved at Session creation: the User scope (always) + * and the Workspace scope (only in a persistent Workspace), each with its own index. + */ +export interface SessionMemory { + /** Absolute path of `memory/user/` — used by the server and Web App; the prompt names this directory as a literal `/…` pattern instead. */ + userDir: string; + /** Content of `memory/user/MEMORY.md` (the `{{USER_MEMORY_INDEX}}` value; empty string when blank). */ + userIndex: string; + /** The Workspace scope; `undefined` in a temporary Workspace, where the User scope is the only place to write. */ + workspace?: SessionWorkspaceMemory; +} + +/** + * Turns a Workspace directory name into the readable half of its key: everything outside + * `[A-Za-z0-9_-]` collapses to a hyphen, and the result is lowercased and truncated. A + * directory whose name survives as empty (a filesystem root, a name made only of separators + * or CJK characters) falls back to `workspace` — the hash half still keeps the key unique. + */ +function safeKeyBase(dirName: string): string { + const cleaned = dirName + .replace(/[^A-Za-z0-9_-]+/g, "-") + .replace(/^-+|-+$/g, "") + .slice(0, KEY_BASE_MAX_LENGTH) + .toLowerCase(); + return cleaned.length > 0 ? cleaned : "workspace"; +} + +/** + * The workspace key for an already-canonical absolute path: `-<8 hex of the + * path's sha256>`. The hyphen before the hash is what makes `USER_SCOPE_KEY` safe to reserve — + * every generated key carries one, so a hyphen-free name is unreachable from here. + * + * Identity is the directory itself and has nothing to do with Git: two symlinks pointing at + * one directory resolve to the same real path and therefore the same key, while moving or + * renaming a directory makes it a new Workspace (its old Memory stays on disk under the old + * key). Prefer `workspaceMemoryKey`, which canonicalizes first. + */ +export function workspaceMemoryKeyForRealPath(realPath: string): string { + const hash = createHash("sha256").update(realPath).digest("hex").slice(0, KEY_HASH_LENGTH); + return `${safeKeyBase(path.basename(realPath))}-${hash}`; +} + +/** + * The workspace key for a Workspace directory: resolves symlinks and `..` first so every + * route to one directory produces one key. A path that cannot be canonicalized (already + * deleted, or not readable) falls back to `path.resolve`, which still yields a stable key + * rather than failing Session creation. + */ +export async function workspaceMemoryKey(workspaceDir: string): Promise { + return workspaceMemoryKeyForRealPath(await realPathOrResolve(workspaceDir)); +} + +async function realPathOrResolve(dir: string): Promise { + try { + return await fs.realpath(dir); + } catch { + return path.resolve(dir); + } +} + +/** + * Whether a Workspace is one PenguinHarness created for a Session itself, i.e. it sits under + * some Agent's `workspaces/` directory (`/agents//workspaces/tmp-xxxxxxxx`). + * + * Temporary Workspaces get no Memory, because one is allocated per Session (see + * `createTempWorkspace`, called once per `createSession`): no later Session ever runs in that + * directory, so anything written to a Memory directory keyed off it could never be read back — + * it would be write-only storage. Note this is *not* because the directory gets cleaned up: + * deleting a Session removes its Traces and scratchpad but leaves `workspaces/tmp-xxxxxxxx` + * behind, and nothing prunes it. + * + * The check is by location rather than by "did the caller pass a workspaceDir", because a + * subagent inherits its parent's Workspace as an explicit argument — including when that + * Workspace is the parent's temporary one. + */ +export async function isTemporaryWorkspace( + root: string, + projectId: string, + workspaceDir: string, +): Promise { + const real = await realPathOrResolve(workspaceDir); + const base = await realPathOrResolve(agentsDir(root, projectId)); + const rel = path.relative(base, real); + if (rel === "" || rel.startsWith("..") || path.isAbsolute(rel)) return false; + const segments = rel.split(path.sep); + // /workspaces/: anything shallower is the Agent directory itself. + return segments.length >= 3 && segments[1] === "workspaces"; +} + +/** + * Creates a scope's empty `MEMORY.md` when it does not exist yet. Existing content — including + * a deliberately emptied file — is never touched: the index is the model's document, the + * Harness only guarantees there is one to open. + */ +async function ensureScopeIndex(dir: string): Promise { + try { + await fs.writeFile(path.join(dir, MEMORY_INDEX_FILENAME), "", { flag: "wx" }); + } catch (err) { + if ((err as NodeJS.ErrnoException).code !== "EEXIST") throw err; + } +} + +/** Reads a scope's `MEMORY.md`; an empty string when it does not exist yet. */ +export async function readScopeIndex(dir: string): Promise { + try { + return await fs.readFile(path.join(dir, MEMORY_INDEX_FILENAME), "utf8"); + } catch { + return ""; + } +} + +/** + * Creates a Workspace's Memory directory (with an empty `MEMORY.md`) if needed and records the + * Workspace path in its `.workspace` marker. + * + * The marker is (re)written whenever it is missing or does not match, which repairs one that + * was hand-edited, truncated or written by an older version. It is deliberately *not* a rename + * path and cannot be one: the key is a hash of the very path the marker records, so a directory + * reached another way — through a symlink — canonicalizes to the same real path and lands on + * this same directory with an identical marker, while a directory that genuinely moved hashes + * to a different key and therefore a different Memory directory. + * + * Never touches topic files or an existing index — those are the model's to maintain. + */ +export async function ensureWorkspaceMemoryDir(args: { + root: string; + projectId: string; + agentId: string; + workspaceKey: string; + workspacePath: string; +}): Promise { + const dir = memoryScopeDir(args.root, args.projectId, args.agentId, args.workspaceKey); + await fs.mkdir(dir, { recursive: true }); + await ensureScopeIndex(dir); + const marker = path.join(dir, WORKSPACE_MARKER_FILENAME); + const line = `${args.workspacePath}\n`; + try { + if ((await fs.readFile(marker, "utf8")) === line) return dir; + } catch { + // No marker yet (or unreadable): fall through and write it. + } + await fs.writeFile(marker, line, "utf8"); + return dir; +} + +/** Absolute path of the User scope's topic directory (`memory/user/`). */ +export function userMemoryDir(root: string, projectId: string, agentId: string): string { + return memoryScopeDir(root, projectId, agentId, USER_SCOPE_KEY); +} + +/** + * Creates the User scope's directory (with an empty `MEMORY.md`) if needed. Unlike a Workspace + * directory it gets no `.workspace` marker: the marker records the path a key was hashed from, + * and this scope stands for no path at all — it belongs to the Agent itself. Called at Agent + * State init and again at every Session creation, so Agents created before Memory shipped + * self-heal. + */ +export async function ensureUserMemoryDir( + root: string, + projectId: string, + agentId: string, +): Promise { + const dir = userMemoryDir(root, projectId, agentId); + await fs.mkdir(dir, { recursive: true }); + await ensureScopeIndex(dir); + return dir; +} + +/** Reads the Workspace path recorded in a Memory directory's marker; `undefined` when the marker is missing or empty. */ +export async function readWorkspaceMarker(dir: string): Promise { + try { + const raw = (await fs.readFile(path.join(dir, WORKSPACE_MARKER_FILENAME), "utf8")).trim(); + return raw.length > 0 ? raw : undefined; + } catch { + return undefined; + } +} + +/** + * Resolves the Memory a Session should run with, creating the scope directories (and their + * empty `MEMORY.md` indexes) as a side effect. The User scope is always prepared; the Workspace + * scope only when the Session runs in a persistent Workspace, so a temporary one never gets a + * directory that no later Session could read back. + * + * Returns `null` — nothing injected, nothing created — when Memory is disabled for the Agent. + * Failures to prepare a directory are also `null`: Memory is an enhancement, and an unwritable + * Agent State should not take down Session creation. + */ +export async function resolveSessionMemory(args: { + root: string; + projectId: string; + agentId: string; + workspaceDir: string; + enabled: boolean; +}): Promise { + if (!args.enabled) return null; + try { + const userDir = await ensureUserMemoryDir(args.root, args.projectId, args.agentId); + const userIndex = await readScopeIndex(userDir); + if (await isTemporaryWorkspace(args.root, args.projectId, args.workspaceDir)) { + return { userDir, userIndex }; + } + const workspacePath = await realPathOrResolve(args.workspaceDir); + const key = workspaceMemoryKeyForRealPath(workspacePath); + const dir = await ensureWorkspaceMemoryDir({ + root: args.root, + projectId: args.projectId, + agentId: args.agentId, + workspaceKey: key, + workspacePath, + }); + return { userDir, userIndex, workspace: { key, dir, index: await readScopeIndex(dir) } }; + } catch { + return null; + } +} + +/** + * Parses a Memory topic file's frontmatter, in the same line-oriented way as Skill + * frontmatter (values are plain scalars, not full YAML). Returns `null` when the file has no + * frontmatter block at all; individual missing fields are simply left out, and unknown fields + * (including the retired `type:` earlier files may carry) are ignored, so a hand-edited file + * never fails to list. `fallbackName` (normally the file name) stands in for a missing `name`. + */ +export function parseMemoryFrontmatter( + content: string, + fallbackName: string, +): MemoryTopicMetadata | null { + // Strip a possible UTF-8 BOM (editors add one when a file is edited by hand); CRLF is handled by \r?\n. + const match = /^---\r?\n([\s\S]*?)\r?\n---/.exec(content.replace(/^/, "")); + if (!match) return null; + const fields: Record = {}; + for (const line of match[1]!.split(/\r?\n/)) { + const idx = line.indexOf(":"); + if (idx <= 0) continue; + const key = line.slice(0, idx).trim(); + if (key) fields[key] = line.slice(idx + 1).trim(); + } + const updatedAt = fields["updated_at"]; + return { + name: fields["name"] || fallbackName, + description: fields["description"] ?? "", + ...(updatedAt ? { updatedAt } : {}), + }; +} diff --git a/packages/core/src/state/paths.ts b/packages/core/src/state/paths.ts index 25bf0099..93923c62 100644 --- a/packages/core/src/state/paths.ts +++ b/packages/core/src/state/paths.ts @@ -124,11 +124,24 @@ export function toolsDir(root: string, projectId: string, agentId: string): stri return path.join(agentStateDir(root, projectId, agentId), "tools"); } -/** `/memory`. */ +/** `/memory`, the Memory root (one subdirectory per scope, see state/memory.ts). */ export function memoryDir(root: string, projectId: string, agentId: string): string { return path.join(agentStateDir(root, projectId, agentId), "memory"); } +/** + * `/memory/`, one Memory scope's topic-file directory — the `user` + * scope or one Workspace's key. Each scope carries its own `MEMORY.md` index inside. + */ +export function memoryScopeDir( + root: string, + projectId: string, + agentId: string, + scopeKey: string, +): string { + return path.join(memoryDir(root, projectId, agentId), scopeKey); +} + /** `/skills`. */ export function skillsDir(root: string, projectId: string, agentId: string): string { return path.join(agentStateDir(root, projectId, agentId), "skills"); diff --git a/packages/core/test/memory.test.ts b/packages/core/test/memory.test.ts new file mode 100644 index 00000000..d7358000 --- /dev/null +++ b/packages/core/test/memory.test.ts @@ -0,0 +1,501 @@ +/** + * Memory: key derivation, temporary-Workspace exclusion, directory preparation, frontmatter + * parsing, and the `{{MEMORY}}` prompt block (heading-led scope sections, the line and char + * caps, the rendered Workspace dir, and the no-placeholder / no-workspace_prompt degradations). + */ +import fs from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { + DEFAULT_AGENT_ID, + DEFAULT_PROJECT_ID, + createAgent, + MEMORY_INDEX_EMPTY_NOTE, + MEMORY_INDEX_FILENAME, + USER_SCOPE_KEY, + WORKSPACE_MARKER_FILENAME, + assembleSystemPrompt, + ensureUserMemoryDir, + ensureWorkspaceMemoryDir, + isTemporaryWorkspace, + loadOrInitAgentState, + memoryDir, + memoryScopeDir, + parseMemoryFrontmatter, + readScopeIndex, + readWorkspaceMarker, + resolveSessionMemory, + userMemoryDir, + workspaceMemoryKey, + workspaceMemoryKeyForRealPath, + type AgentState, + type SessionMemory, +} from "../src/index.js"; +import { stubProviderKeys } from "./provider-keys.js"; + +let root: string; +let workspace: string; +let restoreKeys: () => void; + +beforeEach(async () => { + root = await fs.mkdtemp(path.join(os.tmpdir(), "penguin-memory-")); + workspace = path.join(root, "projects", "my-app"); + await fs.mkdir(workspace, { recursive: true }); + restoreKeys = stubProviderKeys(); +}); + +afterEach(async () => { + restoreKeys(); + await fs.rm(root, { recursive: true, force: true }); +}); + +/** Loads a default Agent State under the temp root (creates it on first call). */ +async function agentState(): Promise { + return loadOrInitAgentState({ root }); +} + +/** The effective system prompt a created Session recorded in its session_meta. */ +function sessionPrompt(session: { metaMessage: { payload: unknown } }): string { + return (session.metaMessage.payload as { system_prompt: string }).system_prompt; +} + +describe("workspace key", () => { + it("is a safe basename plus a path hash, and is stable across calls", () => { + const key = workspaceMemoryKeyForRealPath("/home/u/work/My App!"); + expect(key).toMatch(/^my-app-[0-9a-f]{8}$/); + expect(workspaceMemoryKeyForRealPath("/home/u/work/My App!")).toBe(key); + }); + + it("separates two Workspaces that share a directory name", () => { + expect(workspaceMemoryKeyForRealPath("/a/site")).not.toBe( + workspaceMemoryKeyForRealPath("/b/site"), + ); + }); + + it("falls back to a generic base when the name has nothing key-safe in it", () => { + expect(workspaceMemoryKeyForRealPath("/srv/项目")).toMatch(/^workspace-[0-9a-f]{8}$/); + }); + + it("resolves symlinks, so two routes to one directory share a key", async () => { + const link = path.join(root, "link-to-app"); + await fs.symlink(workspace, link, "dir"); + expect(await workspaceMemoryKey(link)).toBe(await workspaceMemoryKey(workspace)); + }); + + it("keeps a stable key for a directory that no longer exists (realpath fails)", async () => { + const gone = path.join(root, "deleted"); + expect(await workspaceMemoryKey(gone)).toBe(workspaceMemoryKeyForRealPath(path.resolve(gone))); + }); +}); + +describe("temporary Workspace detection", () => { + it("recognizes an Agent's own workspaces/ directory", async () => { + const tmp = path.join( + root, + DEFAULT_PROJECT_ID, + "agents", + DEFAULT_AGENT_ID, + "workspaces", + "tmp-1234abcd", + ); + await fs.mkdir(tmp, { recursive: true }); + expect(await isTemporaryWorkspace(root, DEFAULT_PROJECT_ID, tmp)).toBe(true); + }); + + it("treats a user directory — including one elsewhere under the Agent — as persistent", async () => { + const stateSubdir = path.join(root, DEFAULT_PROJECT_ID, "agents", DEFAULT_AGENT_ID, "traces"); + await fs.mkdir(stateSubdir, { recursive: true }); + expect(await isTemporaryWorkspace(root, DEFAULT_PROJECT_ID, workspace)).toBe(false); + expect(await isTemporaryWorkspace(root, DEFAULT_PROJECT_ID, stateSubdir)).toBe(false); + }); +}); + +describe("directory preparation", () => { + it("creates the Workspace directory with an empty index and records the path in its marker", async () => { + const key = await workspaceMemoryKey(workspace); + const dir = await ensureWorkspaceMemoryDir({ + root, + projectId: DEFAULT_PROJECT_ID, + agentId: DEFAULT_AGENT_ID, + workspaceKey: key, + workspacePath: workspace, + }); + expect(dir).toBe(memoryScopeDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID, key)); + expect(await readWorkspaceMarker(dir)).toBe(workspace); + expect(await readScopeIndex(dir)).toBe(""); + // Idempotent: a second call neither fails nor rewrites a different path. + await ensureWorkspaceMemoryDir({ + root, + projectId: DEFAULT_PROJECT_ID, + agentId: DEFAULT_AGENT_ID, + workspaceKey: key, + workspacePath: workspace, + }); + expect((await fs.readdir(dir)).sort()).toEqual( + [WORKSPACE_MARKER_FILENAME, MEMORY_INDEX_FILENAME].sort(), + ); + }); + + it("initializes Agent State with the User scope and its empty index, and no topic file", async () => { + await agentState(); + expect(await fs.readdir(memoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID))).toEqual([ + USER_SCOPE_KEY, + ]); + const userDir = userMemoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID); + expect(await fs.readdir(userDir)).toEqual([MEMORY_INDEX_FILENAME]); + expect(await readScopeIndex(userDir)).toBe(""); + }); + + it("never overwrites an existing index", async () => { + await agentState(); + const userDir = userMemoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID); + await fs.writeFile(path.join(userDir, MEMORY_INDEX_FILENAME), "- [a](a.md) — hook\n", "utf8"); + await ensureUserMemoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID); + expect(await readScopeIndex(userDir)).toBe("- [a](a.md) — hook\n"); + }); +}); + +describe("resolveSessionMemory", () => { + const resolve = (opts: { workspaceDir: string; enabled: boolean }) => + resolveSessionMemory({ + root, + projectId: DEFAULT_PROJECT_ID, + agentId: DEFAULT_AGENT_ID, + ...opts, + }); + + /** Path of the temporary Workspace the SDK would allocate for a Session given no Workspace. */ + function tempWorkspacePath(): string { + return path.join( + root, + DEFAULT_PROJECT_ID, + "agents", + DEFAULT_AGENT_ID, + "workspaces", + "tmp-cafebabe", + ); + } + + it("prepares both scopes and returns each scope's own index for a persistent Workspace", async () => { + await agentState(); + const userDir = userMemoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID); + await fs.writeFile( + path.join(userDir, MEMORY_INDEX_FILENAME), + "- [pnpm](prefers-pnpm.md) — package manager\n", + "utf8", + ); + const memory = await resolve({ workspaceDir: workspace, enabled: true }); + expect(memory?.userDir).toBe(userDir); + expect(memory?.userIndex).toContain("prefers-pnpm.md"); + expect(memory?.workspace?.key).toBe(await workspaceMemoryKey(workspace)); + expect(memory?.workspace?.index).toBe(""); + await expect(fs.stat(memory!.workspace!.dir)).resolves.toBeTruthy(); + }); + + it("returns null and creates no Workspace scope when Memory is disabled", async () => { + await agentState(); + expect(await resolve({ workspaceDir: workspace, enabled: false })).toBeNull(); + // The User scope from Agent State init stays; the point is nothing new appears. + expect(await fs.readdir(memoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID))).toEqual([ + USER_SCOPE_KEY, + ]); + }); + + it("gives a temporary Workspace the User scope and no Workspace scope", async () => { + await agentState(); + const tmp = tempWorkspacePath(); + await fs.mkdir(tmp, { recursive: true }); + const memory = await resolve({ workspaceDir: tmp, enabled: true }); + // The User scope is the only place such a Session could write and have it read back. + expect(memory?.userDir).toBe(userMemoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID)); + expect(memory?.workspace).toBeUndefined(); + expect(await fs.readdir(memoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID))).toEqual([ + USER_SCOPE_KEY, + ]); + }); +}); + +describe("frontmatter", () => { + it("reads name / description / updated_at", () => { + const parsed = parseMemoryFrontmatter( + "---\nname: testing-conventions\ndescription: how tests run\nupdated_at: 2026-08-07\n---\n\n- body\n", + "testing-conventions.md", + ); + expect(parsed).toEqual({ + name: "testing-conventions", + description: "how tests run", + updatedAt: "2026-08-07", + }); + }); + + it("falls back to the file name and ignores unknown fields, including a legacy type line", () => { + const parsed = parseMemoryFrontmatter( + "---\ntype: feedback\nmood: blue\n---\nbody\n", + "notes.md", + ); + expect(parsed).toEqual({ name: "notes.md", description: "" }); + }); + + it("returns null when the file has no frontmatter", () => { + expect(parseMemoryFrontmatter("just a note\n", "notes.md")).toBeNull(); + }); +}); + +describe("{{MEMORY}} rendering", () => { + /** Heading lines of each half of the block, so a test can assert which scopes were rendered. */ + const USER_LINE = "## User memory"; + const WORKSPACE_LINE = "## Workspace memory"; + + const bothScopes: SessionMemory = { + userDir: "/data/memory/user", + userIndex: "- [pnpm](prefers-pnpm.md) — package manager", + workspace: { + key: "my-app-12345678", + dir: "/data/memory/my-app-12345678", + index: "- [testing](testing-conventions.md) — how tests run", + }, + }; + + it("renders both scopes as heading-led sections; the User dir stays literal, the Workspace dir is injected", async () => { + const state = await agentState(); + const prompt = assembleSystemPrompt(state, undefined, undefined, undefined, bothScopes); + // The User dir is a literal pattern the model resolves from Environment values; the + // Workspace dir's key segment is a hash it never could, so that one renders resolved. + expect(prompt).toContain( + "User Memory Dir: `/agents//agent_state/memory/user`", + ); + expect(prompt).toContain("Workspace Memory Dir: `/data/memory/my-app-12345678`"); + expect(prompt).not.toContain("{{WORKSPACE_MEMORY_DIR}}"); + expect(prompt).toContain(`${USER_LINE}\n`); + expect(prompt).toContain("Index:\n- [pnpm](prefers-pnpm.md) — package manager"); + expect(prompt).toContain(`${WORKSPACE_LINE}\n`); + expect(prompt).toContain("Index:\n- [testing](testing-conventions.md) — how tests run"); + expect(prompt).not.toContain("{{MEMORY}}"); + expect(prompt).not.toContain(MEMORY_INDEX_EMPTY_NOTE); + // The retired marker fences are gone — the headings are the structure. + expect(prompt).not.toContain("[user_memory_index]"); + expect(prompt).not.toContain("[workspace_memory_index]"); + }); + + it("renders the User half alone when the Session has no Workspace scope", async () => { + const state = await agentState(); + const prompt = assembleSystemPrompt(state, undefined, undefined, undefined, { + userDir: "/data/memory/user", + userIndex: "", + }); + expect(prompt).toContain(USER_LINE); + // The Workspace half is a separate config key precisely so it can be left out entirely: + // a temporary Workspace must never be told about a scope it does not have. + expect(prompt).not.toContain(WORKSPACE_LINE); + expect(prompt).not.toContain("Workspace Memory Dir"); + // The scope-choice rule lives in the Workspace half, so a one-scope Session never sees it. + expect(prompt).not.toContain("Facts about the workspace"); + }); + + it("states a scope's store is empty when its index has no content yet", async () => { + const state = await agentState(); + const prompt = assembleSystemPrompt(state, undefined, undefined, undefined, { + ...bothScopes, + workspace: { ...bothScopes.workspace!, index: " \n" }, + }); + // Only the blank Workspace index gets the note; the User index keeps its content. + expect(prompt).toContain(`Index:\n${MEMORY_INDEX_EMPTY_NOTE}`); + expect(prompt).toContain("Index:\n- [pnpm](prefers-pnpm.md) — package manager"); + }); + + it("caps an injected index at 200 lines and notes the truncation", async () => { + const state = await agentState(); + const lines = Array.from({ length: 220 }, (_, i) => `- [m${i}](m${i}.md) — hook`); + const prompt = assembleSystemPrompt(state, undefined, undefined, undefined, { + userDir: "/data/memory/user", + userIndex: lines.join("\n"), + }); + expect(prompt).toContain("- [m199](m199.md) — hook"); + // The file on disk keeps all 220 lines; only the injection is capped. + expect(prompt).not.toContain("- [m200](m200.md) — hook"); + expect(prompt).toContain("showing 200 of 220 lines"); + }); + + it("caps an injected index at 25k characters, cutting at a line boundary", async () => { + const state = await agentState(); + // 10 lines of exactly 5,000 chars slip past the line cap; the char backstop keeps the + // first 4 whole lines (4 × 5,000 + 3 separators = 20,003; a fifth would need 25,004). + const lines = Array.from({ length: 10 }, (_, i) => `- [m${i}](m${i}.md) ${"x".repeat(4986)}`); + const prompt = assembleSystemPrompt(state, undefined, undefined, undefined, { + userDir: "/data/memory/user", + userIndex: lines.join("\n"), + }); + expect(prompt).toContain("- [m3](m3.md) x"); + expect(prompt).not.toContain("- [m4](m4.md) x"); + expect(prompt).toContain("showing 4 of 10 lines"); + }); + + it("cuts mid-line only when a single line alone exceeds the char cap", async () => { + const state = await agentState(); + const prompt = assembleSystemPrompt(state, undefined, undefined, undefined, { + userDir: "/data/memory/user", + userIndex: `- [huge](huge.md) ${"y".repeat(30_000)}`, + }); + expect(prompt).toContain("showing the first 25000 characters"); + }); + + it("injects nothing at all when the Session has no Memory", async () => { + const state = await agentState(); + const prompt = assembleSystemPrompt(state); + expect(prompt).not.toContain("{{MEMORY}}"); + expect(prompt).not.toContain("# Memory"); + expect(prompt).not.toContain(USER_LINE); + expect(prompt).not.toContain(WORKSPACE_LINE); + // Neighboring sections are untouched. + expect(prompt).toContain("# Skills"); + expect(prompt).toContain("# Environment"); + }); + + it("injects no Memory into a template without the placeholder — nothing is spliced in", async () => { + const state = await agentState(); + const bare: AgentState = { + ...state, + systemConfig: { ...state.systemConfig, system_prompt: "# Role\nDo things.\n# Environment" }, + }; + // The Memory tab offers inserting {{MEMORY}} explicitly; rendering never adds it itself. + expect(assembleSystemPrompt(bare, undefined, undefined, undefined, bothScopes)).toBe( + "# Role\nDo things.\n# Environment", + ); + }); + + it("falls back to the built-in prompts for a config that predates Memory", async () => { + const state = await agentState(); + const { memory: _memory, ...withoutMemory } = state.systemConfig; + const legacy: AgentState = { ...state, systemConfig: withoutMemory }; + // The DTO reports the defaults as effective; rendering must agree, or the one-click + // placeholder insert on an old agent would enable a block that expands to nothing. + const prompt = assembleSystemPrompt(legacy, undefined, undefined, undefined, bothScopes); + expect(prompt).toContain("# Memory"); + expect(prompt).toContain(USER_LINE); + expect(prompt).toContain(WORKSPACE_LINE); + }); + + it("resolves the workspace placeholders anywhere in the block instead of leaking them", async () => { + const state = await agentState(); + const custom: AgentState = { + ...state, + systemConfig: { + ...state.systemConfig, + memory: { + enabled: true, + prompt: "P {{WORKSPACE_MEMORY_INDEX}} Q {{WORKSPACE_MEMORY_DIR}} R", + workspace_prompt: "", + }, + }, + }; + // In a persistent Workspace a workspace token written into the main prompt gets the value… + const withWorkspace = assembleSystemPrompt(custom, undefined, undefined, undefined, bothScopes); + expect(withWorkspace).toContain("P - [testing](testing-conventions.md) — how tests run Q"); + expect(withWorkspace).toContain("Q /data/memory/my-app-12345678 R"); + // …and without one it blanks rather than leaking the literal token. + const without = assembleSystemPrompt(custom, undefined, undefined, undefined, { + userDir: "/data/memory/user", + userIndex: "", + }); + expect(without).not.toContain("{{WORKSPACE_MEMORY_INDEX}}"); + expect(without).not.toContain("{{WORKSPACE_MEMORY_DIR}}"); + }); + + it("never re-expands template placeholders smuggled into index content", async () => { + const state = await agentState(); + const prompt = assembleSystemPrompt(state, undefined, ["SOME_KEY"], undefined, { + userDir: "/data/memory/user", + userIndex: "- [x](x.md) — {{VAULT_KEYS}} {{SESSION_ID}}", + }); + // {{MEMORY}} expands last, so the template tokens the model wrote stay literal text. + expect(prompt).toContain("- [x](x.md) — {{VAULT_KEYS}} {{SESSION_ID}}"); + }); + + it("keeps the Workspace half when the main prompt is explicitly emptied", async () => { + const state = await agentState(); + const emptied: AgentState = { + ...state, + systemConfig: { + ...state.systemConfig, + memory: { ...state.systemConfig.memory, prompt: "" }, + }, + }; + // The halves are edited independently on the Memory tab: clearing one never silences the + // other — the Workspace section still renders on its own with its index. + const prompt = assembleSystemPrompt(emptied, undefined, undefined, undefined, bothScopes); + expect(prompt).not.toContain(USER_LINE); + expect(prompt).toContain(WORKSPACE_LINE); + expect(prompt).toContain("Index:\n- [testing](testing-conventions.md) — how tests run"); + }); + + it("drops the Workspace half only for an explicitly emptied workspace_prompt", async () => { + const state = await agentState(); + const emptied: AgentState = { + ...state, + systemConfig: { + ...state.systemConfig, + memory: { ...state.systemConfig.memory, workspace_prompt: "" }, + }, + }; + // A missing key falls back to the built-in default (see the legacy-config test above); + // clearing the field is the deliberate off channel — `??`, not `||`. + const prompt = assembleSystemPrompt(emptied, undefined, undefined, undefined, bothScopes); + expect(prompt).toContain(USER_LINE); + expect(prompt).not.toContain(WORKSPACE_LINE); + }); + + it("reaches the Session prompt with both scopes, or the User scope alone for a temporary Workspace", async () => { + const agent = await createAgent({ root }); + const withWorkspace = await agent.createSession({ workspaceDir: workspace }); + const key = await workspaceMemoryKey(workspace); + expect(sessionPrompt(withWorkspace)).toContain( + `Workspace Memory Dir: \`${memoryScopeDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID, key)}\``, + ); + expect(sessionPrompt(withWorkspace)).toContain(WORKSPACE_LINE); + + // No Workspace given: the SDK allocates a temporary one, which gets the User scope only — + // the User directory renders as the literal pattern, and no Workspace section appears. + const temporary = await agent.createSession(); + expect(sessionPrompt(temporary)).toContain( + "User Memory Dir: `/agents//agent_state/memory/user`", + ); + expect(sessionPrompt(temporary)).not.toContain(WORKSPACE_LINE); + expect( + (await fs.readdir(memoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID))).sort(), + ).toEqual([USER_SCOPE_KEY, key].sort()); + }); + + it("is left out of the Session prompt when the Agent config disables Memory", async () => { + const agent = await createAgent({ root }); + agent.state.systemConfig.memory = { ...agent.state.systemConfig.memory, enabled: false }; + const session = await agent.createSession({ workspaceDir: workspace }); + expect(sessionPrompt(session)).not.toContain(USER_LINE); + expect(sessionPrompt(session)).not.toContain(WORKSPACE_LINE); + // No Workspace scope appears; only the User scope from Agent State init. + expect(await fs.readdir(memoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID))).toEqual([ + USER_SCOPE_KEY, + ]); + }); +}); + +describe("ensureUserMemoryDir", () => { + it("creates the User scope directory with its index and leaves no .workspace marker", async () => { + await agentState(); + const dir = await ensureUserMemoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID); + expect(dir).toBe(userMemoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID)); + // The marker records the path a key was hashed from; this scope stands for no path. + expect(await readWorkspaceMarker(dir)).toBeUndefined(); + // Idempotent: a second Session must not fail on an existing directory. + await expect(ensureUserMemoryDir(root, DEFAULT_PROJECT_ID, DEFAULT_AGENT_ID)).resolves.toBe( + dir, + ); + }); + + it("is a name no generated workspace key can collide with", async () => { + // Every generated key is `-<8 hex>`, so it always carries a hyphen. + const key = workspaceMemoryKeyForRealPath("/home/dev/user"); + expect(key).not.toBe(USER_SCOPE_KEY); + expect(key.startsWith(`${USER_SCOPE_KEY}-`)).toBe(true); + }); +}); diff --git a/packages/docs/content/configuration.en.md b/packages/docs/content/configuration.en.md index 5d7eade4..a173c6c7 100644 --- a/packages/docs/content/configuration.en.md +++ b/packages/docs/content/configuration.en.md @@ -106,6 +106,9 @@ Edit this file via the CLI (`penguin config model …`) or the Web Models page | `compaction.max_session_turns` | `-1` | Cumulative Session turn threshold (`-1` = unlimited) | | `compaction.mode` | `summarize` | `summarize` / `discard` | | `compaction.prompt` | built-in template | Prompt used for summarize compaction | +| `memory.enabled` | `true` | Whether Memory enters the context and Memory directories are prepared | +| `memory.prompt` | built-in template | Always-injected half of the `{{MEMORY}}` block, editable on the Memory tab — carries `{{USER_MEMORY_INDEX}}` | +| `memory.workspace_prompt` | built-in template | Appended only in a persistent Workspace, editable on the Memory tab — carries `{{WORKSPACE_MEMORY_INDEX}}` and `{{WORKSPACE_MEMORY_DIR}}` | | `tools.builtin` | full default toolset when omitted | Tool entries: `name` / `description` / `parameters` / `permission` (`r` or `rw`) / `forModel` / `timeoutMs` / `maxOutputLength` / `call_description` (per-tool toggle for the `description` call argument, required while on; missing = kept); once written it replaces the default list wholesale | | `tools.mcpServers` | `[]` | MCP Server configuration (`name` + `config`); reserved for the MCP adapter layer | @@ -151,6 +154,10 @@ An existing Agent always runs with its on-disk config verbatim — newer code de | `{{AGENTS_MD}}` | Full text of `AGENTS.md` | | `{{VAULT_KEYS}}` | List of Vault key names (names only) | | `{{SKILL_METADATA}}` | Metadata of installed Skills | +| `{{MEMORY}}` | The rendered `memory.prompt` block, plus `memory.workspace_prompt` in a persistent Workspace; empty when Memory is off. A template without it injects no Memory — the Memory tab offers inserting it explicitly | +| `{{USER_MEMORY_INDEX}}` | Inside the Memory prompts: content of the user scope's `MEMORY.md` index (at most 200 lines and 25,000 characters total) | +| `{{WORKSPACE_MEMORY_INDEX}}` | Inside `memory.workspace_prompt` only: content of the Workspace scope's `MEMORY.md` index (at most 200 lines and 25,000 characters total) | +| `{{WORKSPACE_MEMORY_DIR}}` | Inside `memory.workspace_prompt` only: absolute path of the current Workspace's Memory directory | | `{{PLATFORM}}` | Runtime platform | | `{{OS_VERSION}}` | Operating system version | | `{{DATE}}` | Current date | @@ -167,6 +174,58 @@ On Windows, `{{PROJECT_DIR}}` and `{{CWD}}` are injected with forward slashes `agent_state/AGENTS.md` is the developer-editable instruction file, injected via `{{AGENTS_MD}}` and empty by default — it is also the file an optimizer edits most (see [Self-Improvement](/self-improvement)). +## Memory + +`agent_state/memory/` is what the Agent remembers between Sessions: user feedback, project decisions, working conventions and entry points into external systems — the things that cannot be re-derived from the Workspace or its code history. It is not context compaction, which preserves one Session's short-term state. + +Memory has two scopes, both belonging to one Agent and never shared with another: + +- **User scope** (`memory/user/`) — what stays true wherever the Agent works: who the user is, their standing preferences, reference material not tied to one codebase. Every Session reads it, including one running in a temporary Workspace, which has no other place to write. +- **Workspace scope** (`memory//`) — facts about one Workspace. Sessions of one Agent in one Workspace share it; different Workspaces keep their topic files apart. + +Each scope carries its own `MEMORY.md` index, and different Agents never share Memory even in the same Workspace. Because Memory lives in Agent State it travels with export / import and snapshots, and every Project member who can reach the Agent can read it — so credentials and sensitive personal data never belong in it. + +```text +agent_state/memory/ +├── user/ # user scope (no marker: it stands for no path) +│ ├── MEMORY.md # this scope's index +│ └── prefers-pnpm.md +└── my-app-a81f32c4/ # one Workspace + ├── .workspace # the Workspace path this key stands for + ├── MEMORY.md + └── testing-conventions.md +``` + +`user` is a reserved directory name, safe because every generated workspace memory key is `-<8 hex>` and therefore always carries a hyphen — a hyphen-free name can never be produced. + +The workspace memory key is `-<8 hex of the real path's sha256>`. Identity is the directory itself and has nothing to do with Git: two symlinks to one directory resolve to one key, while moving or renaming a directory makes it a new Workspace (the old Memory stays on disk under the old key). A temporary Workspace — one PenguinHarness allocated under `agents//workspaces/` — gets no Workspace scope at all, a subagent inheriting one included: a temporary Workspace is allocated per Session, so no later Session would ever run there to read it back. Such a Session still gets the user scope, which is where anything it learns belongs anyway. + +A topic file is a semantic subject, not one per Task, Session or date, and carries frontmatter: + +```markdown +--- +name: testing-conventions +description: the project's test environment and verification rules +updated_at: 2026-08-07 +--- + +- Integration tests connect to a real database; no mock repositories. +``` + +These three fields are all the frontmatter there is — which layer a memory belongs to is expressed by its directory, so there is no `type` field (a `type:` line left in an earlier file is ignored as an unknown field). Worth saving: who the user is (role, expertise, standing preferences) and how they want the Agent to work, with the why; decisions, constraints and plans not derivable from the code; stable entry points into external systems, documents and services. A topic that turns out to be wrong is deleted together with its index line, and dates are written absolute (`YYYY-MM-DD`) — relative ones mean nothing to a later Session. What must never be saved: facts the code, config or Git history already states; short-lived task progress and debugging notes; credentials, tokens or secrets; unconfirmed guesses; long transcript excerpts. + +Each `MEMORY.md` lists its scope's memories one line each — `- [Title](file.md) — hook`, links relative to the scope directory — and is updated in the same round as the file, so the two never disagree. + +Only the indexes reach the context, through the template's `{{MEMORY}}` placeholder. It expands to `memory.prompt` — what Memory is for, the save mechanics, then a `## User memory` section with its index (`{{USER_MEMORY_INDEX}}`) — plus `memory.workspace_prompt`, a `## Workspace memory` section with `{{WORKSPACE_MEMORY_INDEX}}`, when the Session runs in a persistent Workspace. Both prompts are per-Agent config, editable on the settings page's Memory tab, and organized by Markdown headings like the template's other sections. The `User Memory Dir` line is the literal pattern `/agents//agent_state/memory/user`, resolvable from the Environment section; the `Workspace Memory Dir` line renders resolved via `{{WORKSPACE_MEMORY_DIR}}`, because its final segment — the workspace memory key — is a path hash the model could never compose itself. + +A blank index injects an explicit "nothing saved yet" note. Injection is capped at 200 lines per scope (one memory per line by convention), then at 25,000 characters total as a backstop for indexes whose few lines are enormous — past a cap a truncation note tells the model to open the full `MEMORY.md` itself, and the file on disk is never touched. The default Memory prompt declares the line cap and asks for index lines under ~150 characters, so the model keeps the index short before ever hitting the caps; the character backstop lives only in code. Topic bodies are read on demand by the model. + +The two halves are separate config keys because substitution has no conditionals: a temporary Workspace must never be handed the Workspace section (its directory line and the scope-choice rule), so that half is simply not appended there. The Harness only decides where Memory lives and keeps writes inside it — deciding what is worth keeping, splitting topics and maintaining the indexes is the model's job, done with the ordinary file tools. + +A template without `{{MEMORY}}` injects no Memory — an Agent created before Memory shipped, for instance. The Memory tab reports this and offers inserting the placeholder (before `# Environment`, the position the default template gives it) as an explicit one-click action; nothing is ever spliced in automatically. The assembled prompt is recorded in `session_meta`. + +To read, delete or ask the Agent to edit what it has saved, use the settings page's [Memory tab](/web-app#agent-settings-agents). + ## Vault `agent_state/.vault.toml` is the Agent-level environment-variable vault: a hidden file written with mode 0600. diff --git a/packages/docs/content/configuration.zh.md b/packages/docs/content/configuration.zh.md index dc75fc24..1e45836e 100644 --- a/packages/docs/content/configuration.zh.md +++ b/packages/docs/content/configuration.zh.md @@ -106,6 +106,9 @@ output = 0.857143 | `compaction.max_session_turns` | `-1` | Session 累计轮数阈值（`-1` 不限制） | | `compaction.mode` | `summarize` | `summarize` / `discard` | | `compaction.prompt` | 内置模板 | summarize 压缩使用的 Prompt | +| `memory.enabled` | `true` | 记忆是否进入上下文、是否为持久 Workspace 准备记忆目录 | +| `memory.prompt` | 内置模板 | `{{MEMORY}}` 区块中恒注入的一半，可在记忆标签页编辑——含 `{{USER_MEMORY_INDEX}}` | +| `memory.workspace_prompt` | 内置模板 | 仅持久 Workspace 追加，可在记忆标签页编辑——含 `{{WORKSPACE_MEMORY_INDEX}}` 与 `{{WORKSPACE_MEMORY_DIR}}` | | `tools.builtin` | 缺省时为完整默认工具集 | 工具条目：`name` / `description` / `parameters` / `permission`（`r` 或 `rw`）/ `forModel` / `timeoutMs` / `maxOutputLength` / `call_description`（条目级开关：控制 `description` 调用参数，开启时为必填，缺省保留）；一旦写出即整体替换默认列表 | | `tools.mcpServers` | `[]` | MCP Server 配置（`name` + `config`），预留给 MCP 适配层 | @@ -151,6 +154,10 @@ compaction: | `{{AGENTS_MD}}` | `AGENTS.md` 的全文 | | `{{VAULT_KEYS}}` | Vault 的键名列表（仅键名） | | `{{SKILL_METADATA}}` | 已安装 Skill 的元数据 | +| `{{MEMORY}}` | 渲染后的 `memory.prompt` 区块，持久 Workspace 下再追加 `memory.workspace_prompt`；关闭记忆时为空。模板没有它就不注入记忆——记忆标签页提供显式插入 | +| `{{USER_MEMORY_INDEX}}` | 记忆提示词内：用户作用域 `MEMORY.md` 索引的内容（最多注入 200 行、总计 25000 字符） | +| `{{WORKSPACE_MEMORY_INDEX}}` | 仅 `memory.workspace_prompt` 内：Workspace 作用域 `MEMORY.md` 索引的内容（最多注入 200 行、总计 25000 字符） | +| `{{WORKSPACE_MEMORY_DIR}}` | 仅 `memory.workspace_prompt` 内：当前 Workspace 记忆目录的绝对路径 | | `{{PLATFORM}}` | 运行平台 | | `{{OS_VERSION}}` | 操作系统版本 | | `{{DATE}}` | 当前日期 | @@ -167,6 +174,58 @@ Windows 上注入的 `{{PROJECT_DIR}}` 与 `{{CWD}}` 统一使用正斜杠—— `agent_state/AGENTS.md` 是开发者可编辑的指令文件，经 `{{AGENTS_MD}}` 注入系统提示词，缺省为空——它也是优化器最常改动的文件（见[自我进化](/self-improvement)）。 +## 记忆 + +`agent_state/memory/` 保存 Agent 跨 Session 的长期记忆：用户反馈、项目决策、协作约定与外部系统入口——这些无法从 Workspace 或代码历史可靠重新推导。它不是上下文压缩：压缩保存单个 Session 的短期工作状态。 + +记忆有两个作用域，都归属于单个 Agent，绝不跨 Agent 共享： + +- **用户作用域**（`memory/user/`）——无论在哪工作都成立的内容：用户是谁、其长期偏好、与具体代码库无关的参考。每个 Session 都会读到，包括运行在临时 Workspace 中的会话——那种会话没有别处可写。 +- **Workspace 作用域**（`memory//`）——关于某一个 Workspace 的事实。同一 Agent、同一 Workspace 的多个 Session 共享；不同 Workspace 的主题文件相互隔离。 + +每个作用域各带一份 `MEMORY.md` 索引；不同 Agent 即使使用同一 Workspace 也各自维护。记忆位于 Agent State，因此随导出、导入与快照一同流转，Project 内有权访问该 Agent 的成员都能读到——所以凭据与敏感个人信息绝不应写入。 + +```text +agent_state/memory/ +├── user/ # 用户作用域（无 marker：它不对应任何路径） +│ ├── MEMORY.md # 本作用域索引 +│ └── prefers-pnpm.md +└── my-app-a81f32c4/ # 单个 Workspace + ├── .workspace # 该 key 对应的 Workspace 路径 + ├── MEMORY.md + └── testing-conventions.md +``` + +`user` 是保留目录名。之所以安全：生成的 workspace memory key 一律是 `-<8 位十六进制>`，必然含连字符——不含连字符的名字永远不会被生成出来。 + +workspace memory key 为 `<安全 basename>-<真实路径 sha256 的 8 位十六进制>`。身份只由实际目录决定，与 Git 无关：指向同一目录的两个软链接得到同一 key；目录移动或重命名后视为新的 Workspace（旧记忆仍以旧 key 留在磁盘上）。PenguinHarness 自动创建的临时 Workspace（位于 `agents//workspaces/` 下）没有 Workspace 作用域，子 Agent 继承该临时 Workspace 时同样没有：临时 Workspace 是每个 Session 分配一个，之后不会有任何 Session 再跑进去读它。这类会话仍然拥有用户作用域——它能学到的东西本来也属于那一层。 + +主题文件按语义划分，不按 Task、Session 或日期划分，并带 frontmatter： + +```markdown +--- +name: testing-conventions +description: 项目的测试环境和验证规则 +updated_at: 2026-08-07 +--- + +- 集成测试必须连接真实数据库，不使用 mock repository。 +``` + +frontmatter 只有这三个字段——记忆属于哪一层由所在目录表达，不设 `type` 字段（早期文件里残留的 `type:` 行会被当作未知字段忽略）。值得保存的是：用户是谁（角色、专长、长期偏好）以及希望 agent 如何工作（连同原因）；无法仅从代码推导的决策、约束与计划；外部系统、文档与服务的稳定入口。记错的主题连同其索引行一并删除；日期写绝对日期（`YYYY-MM-DD`），相对日期对后续 Session 没有意义。不应保存：可从代码、配置或 Git 历史直接获得的事实；短期任务进度与调试流水；凭据、Token 等敏感值；未经确认的推测；大段对话原文。 + +每份 `MEMORY.md` 一行列一条记忆——`- [标题](file.md) — 一句钩子`，链接相对本作用域目录——并与记忆文件同轮更新，两者永不脱节。 + +进入上下文的只有索引，入口是模板的 `{{MEMORY}}` 占位符。它展开为 `memory.prompt`——记忆的用途、写入规范，以及 `## User memory` 小节与其索引（`{{USER_MEMORY_INDEX}}`）——持久 Workspace 的会话再追加 `memory.workspace_prompt`（`## Workspace memory` 小节，含 `{{WORKSPACE_MEMORY_INDEX}}`）。两个提示词都是 Agent 级配置，可在设置页记忆标签直接编辑；区块和模板其他小节一样用 Markdown 标题组织。`User Memory Dir` 行是字面模式 `/agents//agent_state/memory/user`，模型可据 Environment 段自行拼出；`Workspace Memory Dir` 行经 `{{WORKSPACE_MEMORY_DIR}}` 直接渲染为解析后的路径——它的末段（工作区记忆键）是路径哈希，模型无从自行推得。 + +空索引会注入一句"尚未保存任何内容"的占位说明。每个作用域最多注入 200 行索引（按约定每条记忆一行），再以总计 25000 字符兜底——防住行数不多但单行超长的索引；超出上限的部分以截断提示替代、由模型自行读取完整 `MEMORY.md`，磁盘上的文件不受影响。默认记忆提示词只声明行数上限，并要求每行索引保持在约 150 字符以内，模型在撞线之前就会把索引保持在限内；字符兜底仅存在于代码中。主题正文由模型按需读取。 + +两半之所以是两个独立配置键：替换引擎没有条件分支，临时 Workspace 绝不能被塞进 Workspace 小节（它的目录行和作用域二选一规则），所以那一半在临时 Workspace 下干脆不追加。Harness 只负责确定记忆位置并限制写入边界，判断什么值得保存、如何划分主题、如何维护索引都由模型用现有文件工具完成。 + +模板中没有 `{{MEMORY}}` 的 Agent（例如创建于记忆功能之前）不会注入任何内容；记忆标签页会给出提示并提供一键插入占位符（插到 `# Environment` 之前，即默认模板中的位置）——不存在任何自动拼接。组装后的完整提示词记录在 `session_meta`。 + +查看、删除或让 Agent 修改已保存的记忆，见设置页的[记忆标签](/web-app#agent-设置agents)。 + ## Vault `agent_state/.vault.toml` 是 Agent 级的环境变量保险库：隐藏文件，落盘权限 0600。 diff --git a/packages/docs/content/server-api.en.md b/packages/docs/content/server-api.en.md index 87ca63df..d48082bb 100644 --- a/packages/docs/content/server-api.en.md +++ b/packages/docs/content/server-api.en.md @@ -129,6 +129,10 @@ The paths below omit the `/api/projects/:projectId` prefix. | DELETE | /agents/:agentId | Delete an Agent | | GET / PUT | /agents/:agentId/config | Read / write config (AGENTS.md + system_config.yaml; PUT preserves YAML comments) | | GET / PUT | /agents/:agentId/vault | Vault environment variables (values masked; PUT is a full replace) | +| GET | /agents/:agentId/memory | Memory overview: the switch, whether the template carries `{{MEMORY}}`, and one entry per scope — the user scope (`user`, `kind: "user"`) first, then the Workspaces | +| POST | /agents/:agentId/memory/template-placeholder | Insert the `{{MEMORY}}` placeholder into the prompt template (idempotent; the explicit adoption path for an Agent created before Memory) | +| GET | /agents/:agentId/memory/scopes/:key/files | List one scope's topic files (frontmatter + stats); `:key` is a workspace key or `user` | +| GET / DELETE | /agents/:agentId/memory/scopes/:key/files/:name | Read one topic file / delete it (also pruning its `MEMORY.md` index lines) | | GET | /agents/:agentId/export | Export the Agent State snapshot (tar.gz download) | | POST | /agents/:agentId/import | Import a snapshot: `{dataBase64, confirm?}`; 409 on version conflict without confirm | | GET / POST | /agents/:agentId/skills | List / install installed Skills | diff --git a/packages/docs/content/server-api.zh.md b/packages/docs/content/server-api.zh.md index eda4860f..c96a0b37 100644 --- a/packages/docs/content/server-api.zh.md +++ b/packages/docs/content/server-api.zh.md @@ -129,6 +129,10 @@ curl -c cookies.txt -H "Content-Type: application/json" \ | DELETE | /agents/:agentId | 删除 Agent | | GET / PUT | /agents/:agentId/config | 读写配置（AGENTS.md + system_config.yaml，PUT 保留 YAML 注释） | | GET / PUT | /agents/:agentId/vault | Vault 环境变量（值掩码显示；PUT 全表替换） | +| GET | /agents/:agentId/memory | 记忆总览：开关、模板是否含 `{{MEMORY}}`，以及各作用域条目——用户作用域（`user`，`kind: "user"`）在前，其后为各 Workspace | +| POST | /agents/:agentId/memory/template-placeholder | 向提示词模板插入 `{{MEMORY}}` 占位符（幂等；创建于记忆功能之前的 Agent 的显式采用路径） | +| GET | /agents/:agentId/memory/scopes/:key/files | 列出单个作用域的主题文件（frontmatter + 文件信息）；`:key` 为 workspace key 或 `user` | +| GET / DELETE | /agents/:agentId/memory/scopes/:key/files/:name | 读取单个主题文件 / 删除它（并同步清理其 `MEMORY.md` 索引行） | | GET | /agents/:agentId/export | 导出 Agent State 快照（tar.gz 下载） | | POST | /agents/:agentId/import | 导入快照：`{dataBase64, confirm?}`；版本冲突且未确认时返回 409 | | GET / POST | /agents/:agentId/skills | 已安装 Skill 列表 / 安装 | diff --git a/packages/docs/content/sessions-and-traces.en.md b/packages/docs/content/sessions-and-traces.en.md index c8f73071..aa1ce788 100644 --- a/packages/docs/content/sessions-and-traces.en.md +++ b/packages/docs/content/sessions-and-traces.en.md @@ -30,7 +30,9 @@ The data root is the `PENGUIN_HOME` environment variable, defaulting to `~/.peng └── agents/ └── / ├── agent_state/ # system_config.yaml, AGENTS.md, .vault.toml, - │ # tools/, memory/, skills/, schedule/ + │ # tools/, skills/, schedule/ + │ └── memory/ # Memory: user/ plus one directory per Workspace, + │ # each with its own MEMORY.md index ├── traces/ │ └── /_.jsonl ├── scratchpad/ # temp files, one subdirectory per Session id (e.g. pasted images) diff --git a/packages/docs/content/sessions-and-traces.zh.md b/packages/docs/content/sessions-and-traces.zh.md index b15a4f02..9ab8d85d 100644 --- a/packages/docs/content/sessions-and-traces.zh.md +++ b/packages/docs/content/sessions-and-traces.zh.md @@ -30,7 +30,9 @@ PenguinHarness 的全部运行数据都落在本地文件系统：配置是可 └── agents/ └── / ├── agent_state/ # system_config.yaml、AGENTS.md、.vault.toml、 - │ # tools/、memory/、skills/、schedule/ + │ # tools/、skills/、schedule/ + │ └── memory/ # 记忆：user/ 加每个 Workspace 一个目录， + │ # 各自带一份 MEMORY.md 索引 ├── traces/ │ └── /_.jsonl ├── scratchpad/ # 临时文件，按 Session id 建子目录（如粘贴的图片） diff --git a/packages/docs/content/web-app.en.md b/packages/docs/content/web-app.en.md index 33f36575..bcd52212 100644 --- a/packages/docs/content/web-app.en.md +++ b/packages/docs/content/web-app.en.md @@ -68,11 +68,14 @@ The list page creates and deletes Agents; clicking through opens the `/agents/:a | --- | --- | | Overview | Basic info, export / import of Agent State snapshots, and restoring the default configuration (overwrites customizations, keeping only name/description) | | Prompt | AGENTS.md and system_prompt | +| Memory | The Agent-level switch, then every memory grouped by scope — user memory first, then one group per Workspace, each group collapsible with an add entry — with view / delete / edit-via-chat actions per row | | Runtime | Runtime parameters such as max_turns, model.*, compaction.* | | Tools | Built-in tool table (incl. per-tool call_description switches) and MCP server JSON configuration | | Vault | Environment-variable entries with masked values | | Schedule | Scheduled tasks (TOML-defined): create, edit, toggle, delete | +The Memory tab's switch writes immediately rather than joining the tab-level Save, so turning Memory off never drags an unrelated half-finished edit along with it. Turning it off keeps every file and the tab fully usable: it only stops Memory from entering the agent's context and from preparing directories for new Sessions. Scope groups collapse on header click, with the collapse state remembered in the browser per user × Project × Agent; each group header carries an **Add** text entry to its collapse arrow's left (the models page's group convention) — like editing, it goes through a bridge modal (a required content-or-source field plus a live prompt preview) into a new chat, where the agent organizes the content into that scope; the drafts stay deliberately minimal, since the save mechanics live in the Memory prompt. Rows are read-only on purpose, their actions icon-only like the skills tab's — **View** renders the body in a right drawer on desktop and a bottom sheet on narrow screens (the chat page panels' interaction, with half / full snap points), **Delete** confirms and also removes the file's `MEMORY.md` index lines, and **Edit** opens a bridge modal first (what to change plus a prompt preview), then jumps to a new chat with this agent; a Workspace scope's chat also pins that Workspace, so the Session reads the very index it is changing. Below the groups the two memory prompts (`memory.prompt` / `memory.workspace_prompt`) are edited in place, and when the prompt template carries no `{{MEMORY}}` placeholder (an agent created before Memory) it shows a hint with a one-click insert — an explicit, idempotent config write. The agent list cards' stat line also shows the memory count, deep-linking to this tab. For the storage model, see [Configuration](/configuration#memory). + Scheduled tasks fire on a fixed period (minimum 5 minutes) and run only while the service is running. ## Skill Library (/skills) diff --git a/packages/docs/content/web-app.zh.md b/packages/docs/content/web-app.zh.md index bb2251e5..6731dfc1 100644 --- a/packages/docs/content/web-app.zh.md +++ b/packages/docs/content/web-app.zh.md @@ -67,11 +67,14 @@ penguin web | --- | --- | | Overview | 基本信息、Agent State 快照的导出 / 导入，以及还原为默认配置（覆盖自定义内容，仅保留名称与描述） | | Prompt | AGENTS.md 与 system_prompt | +| Memory | Agent 级开关，以及按作用域分组的全部记忆——用户记忆在前、每个 Workspace 一组，分组可折叠并带添加入口——每行提供查看 / 删除 / 对话编辑 | | Runtime | max_turns、model.*、compaction.* 等运行参数 | | Tools | 内置工具表格（含条目级 call_description 开关）与 MCP Server 的 JSON 配置 | | Vault | 环境变量条目，值以掩码显示 | | Schedule | 定时任务（TOML 定义）：创建、编辑、启停、删除 | +Memory 标签页的开关拨动即写入，不参与标签页的保存按钮，因此关闭记忆不会连带把别处未改完的内容一起存盘。关闭记忆也不会删除任何文件，标签页仍可正常查看与删除——只是记忆不再进入 Agent 上下文，也不再为新 Session 准备目录。作用域分组可折叠，折叠状态按「用户 × Project × Agent」记在浏览器本地；每组标题的折叠箭头左侧有**添加**文字入口（与模型页的分组约定一致）——和编辑一样经中转弹窗（必填的内容或来源 + 引导语实时预览）跳到新对话，由 agent 整理保存到该作用域；引导语刻意极简，写入规范由记忆提示词承载。记忆行有意不提供就地编辑，操作与技能页一致为纯图标按钮：**查看**在桌面端从右侧抽屉、窄屏从底部弹层（与对话页面板同一交互，半屏 / 全屏两档吸附）渲染正文；**删除**先确认，并同步清掉该文件在 `MEMORY.md` 中的索引行；**编辑**先打开中转弹窗（修改要求 + 引导语预览），再跳到与该 Agent 的新对话——Workspace 作用域的对话还会锁定对应 Workspace，让会话恰好读到它正在修改的那份索引。标签页下方可直接编辑两段记忆提示词（`memory.prompt` / `memory.workspace_prompt`）；当提示词模板中没有 `{{MEMORY}}` 占位符（创建于记忆功能之前的 agent）时会给出提示并提供一键插入——显式且幂等的配置写入。Agent 列表卡片的统计行也显示记忆条数，点击直达本标签页。存储模型见[配置说明](/configuration#记忆)。 + 定时任务按固定周期触发（最短 5 分钟），且仅在服务运行期间执行。 ## Skill 库（/skills） diff --git a/packages/server/src/api/types.ts b/packages/server/src/api/types.ts index 8580124c..21113fcf 100644 --- a/packages/server/src/api/types.ts +++ b/packages/server/src/api/types.ts @@ -506,6 +506,8 @@ export interface AgentSummary { scheduleCount: number; /** Installed Skill count (number of agent_state/skills// directories with a SKILL.md). */ skillCount: number; + /** Memory count (topic files summed over the scope directories under agent_state/memory/, independent of the memory switch). */ + memoryCount: number; } export interface AgentsResponse { @@ -537,6 +539,15 @@ export interface AgentCompactionConfigDto { prompt?: string; } +/** Memory config. All fields report effective values (a config with no `memory` section reads as enabled with the built-in prompts, matching core); the prompts are edited on the Memory tab. */ +export interface AgentMemoryConfigDto { + enabled: boolean; + /** The always-injected half of the `{{MEMORY}}` block (carries `{{USER_MEMORY_INDEX}}`; the User directory is literal text). */ + prompt: string; + /** Appended only in a persistent Workspace (carries `{{WORKSPACE_MEMORY_INDEX}}` and the rendered `{{WORKSPACE_MEMORY_DIR}}` directory). */ + workspacePrompt: string; +} + /** Structured view of system_config.yaml (for the edit form). */ export interface AgentConfigDto { name?: string; @@ -547,6 +558,7 @@ export interface AgentConfigDto { maxTurns?: number; model?: AgentModelConfigDto; compaction?: AgentCompactionConfigDto; + memory: AgentMemoryConfigDto; toolsBuiltin: ToolDefinitionConfig[]; mcpServers: MCPServerConfig[]; } @@ -571,11 +583,70 @@ export interface AgentConfigUpdateRequest { maxTurns?: number; model?: AgentModelConfigDto; compaction?: AgentCompactionConfigDto; + memory?: Partial; toolsBuiltin?: ToolDefinitionConfig[]; mcpServers?: MCPServerConfig[]; }; } +// --------------------------------------------------------------------------- +// Memory +// --------------------------------------------------------------------------- + +/** One Memory scope directory: `agent_state/memory/user/` or `agent_state/memory//`. */ +export interface MemoryScopeInfo { + /** Directory name under `memory/`: `user`, or a Workspace's `-` key. */ + scopeKey: string; + /** `user` — the scope every Session reads, temporary Workspaces included; `workspace` — one Workspace's scope. */ + kind: "user" | "workspace"; + /** Workspace path the key was derived from, read from the directory's `.workspace` marker; unset on the user scope (it stands for no path) and for a directory edited by hand. */ + workspacePath?: string; + /** Number of Markdown topic files in the directory (the `MEMORY.md` index not counted). */ + fileCount: number; + /** Most recent topic-file mtime in the directory (ISO 8601); unset when the directory holds no topic file. */ + updatedAt?: string; +} + +/** One Memory topic file, as listed (frontmatter only — the body is fetched per file). */ +export interface MemoryFileInfo { + /** File name inside the scope directory, e.g. `prefers-pnpm.md`. */ + name: string; + /** Frontmatter `name`; falls back to the file name. */ + title: string; + /** Frontmatter `description`; empty when the file declares none. */ + description: string; + /** Frontmatter `updated_at`, verbatim. */ + updatedAt?: string; + /** File size in bytes. */ + size: number; + /** File mtime (ISO 8601). */ + modifiedAt: string; +} + +/** GET …/memory — the tab's landing payload: the switch and every scope group, user scope first. */ +export interface MemoryOverviewResponse { + /** Whether Memory reaches the model context (the Agent-level switch). */ + enabled: boolean; + /** Whether the prompt template carries the `{{MEMORY}}` placeholder. An Agent created before Memory has none and injects nothing; POST …/memory/template-placeholder inserts it explicitly. */ + templateHasMemory: boolean; + /** Absolute path of `agent_state/memory/`. */ + memoryDir: string; + scopes: MemoryScopeInfo[]; +} + +/** GET …/memory/scopes/:key/files */ +export interface MemoryFilesResponse { + scopeKey: string; + files: MemoryFileInfo[]; +} + +/** GET …/memory/scopes/:key/files/:name */ +export interface MemoryFileResponse { + scopeKey: string; + file: MemoryFileInfo; + content: string; +} + // --------------------------------------------------------------------------- // Session // --------------------------------------------------------------------------- diff --git a/packages/server/src/app.ts b/packages/server/src/app.ts index 0334c429..c4cc9580 100644 --- a/packages/server/src/app.ts +++ b/packages/server/src/app.ts @@ -46,6 +46,7 @@ import { membersRoutes } from "./http/routes/members.js"; import { modelsRoutes } from "./http/routes/models.js"; import { chatDefaultsRoutes } from "./http/routes/chat-defaults.js"; import { vaultRoutes } from "./http/routes/vault.js"; +import { memoryRoutes } from "./http/routes/memory.js"; import { scheduleRoutes } from "./http/routes/schedules.js"; import { benchmarksRoutes } from "./http/routes/benchmarks.js"; import { agentSkillsRoutes, skillLibraryRoutes } from "./http/routes/skills.js"; @@ -70,6 +71,7 @@ import { AdminService } from "./services/admin-service.js"; import { DesktopService } from "./services/desktop-service.js"; import { desktopRoutes } from "./http/routes/desktop.js"; import { AgentConfigService } from "./services/agent-config-service.js"; +import { MemoryService } from "./services/memory-service.js"; import { AgentService } from "./services/agent-service.js"; import { BenchmarkService } from "./services/benchmark-service.js"; import { SnapshotService } from "./services/snapshot-service.js"; @@ -106,6 +108,7 @@ export interface AppDeps { projectConfigService: ProjectConfigService; agentService: AgentService; agentConfigService: AgentConfigService; + memoryService: MemoryService; sessionService: SessionService; traceService: TraceService; /** Trace-file index (derived cache + reconciler); routes use it for delete-time coherence. */ @@ -180,6 +183,7 @@ export function buildAppDeps(config: ServerConfig, overrides: BuildDepsOverrides const projectConfigService = new ProjectConfigService(config.root); const agentConfigService = new AgentConfigService(config.root); const agentService = new AgentService(config.root, agentsRepo, agentConfigService); + const memoryService = new MemoryService(config.root, agentConfigService); // Session-origin registry: session_meta is the single source of truth (no DB column); // shared by the manager (subagent registration), the loader (self-heal rebuild), // SessionService (creation / adoption / lazy list resolution), and the Trace index / @@ -314,6 +318,7 @@ export function buildAppDeps(config: ServerConfig, overrides: BuildDepsOverrides projectConfigService, agentService, agentConfigService, + memoryService, sessionService, traceService, traceIndex, @@ -435,6 +440,7 @@ export function createApp(deps: AppDeps): Hono { app.route("/api/projects/:projectId/dirs", dirsRoutes(deps)); app.route("/api/projects/:projectId/agents/:agentId/config", agentConfigRoutes(deps)); app.route("/api/projects/:projectId/agents/:agentId/vault", vaultRoutes(deps)); + app.route("/api/projects/:projectId/agents/:agentId/memory", memoryRoutes(deps)); app.route("/api/projects/:projectId/agents/:agentId/schedules", scheduleRoutes(deps)); app.route("/api/projects/:projectId/agents/:agentId/benchmarks", benchmarksRoutes(deps)); app.route("/api/projects/:projectId/agents/:agentId/skills", agentSkillsRoutes(deps)); diff --git a/packages/server/src/http/routes/memory.ts b/packages/server/src/http/routes/memory.ts new file mode 100644 index 00000000..d45facbb --- /dev/null +++ b/packages/server/src/http/routes/memory.ts @@ -0,0 +1,67 @@ +/** + * Memory routes (`agent_state/memory/`), all Project-member operations: + * GET /api/projects/:p/agents/:a/memory # switch + scope groups (user scope first) + * POST /api/projects/:p/agents/:a/memory/template-placeholder # insert the {{MEMORY}} placeholder into the template + * GET /api/projects/:p/agents/:a/memory/scopes/:key/files # one scope's topic files + * GET …/memory/scopes/:key/files/:name # one topic file's content + * DELETE …/memory/scopes/:key/files/:name # delete + prune its index lines + * + * Deliberately read + delete only: content edits go through a chat Session where the model + * maintains the files and their `MEMORY.md` index together. The `memory.enabled` switch is + * Agent configuration and lives on PUT …/agents/:a/config. + * + * No route accepts an absolute path: a file is addressed by `agentId` + `scopeKey` + a name + * inside that scope, and MemoryService re-checks that the resolved path stayed inside the + * Agent's Memory directory. + */ +import { Hono } from "hono"; +import type { Context } from "hono"; +import type { AppEnv } from "../../auth/middleware.js"; +import type { AppDeps } from "../../app.js"; +import { pathParam, requireValidId } from "../validate.js"; + +export function memoryRoutes(deps: AppDeps): Hono { + const app = new Hono(); + + /** Shared preamble: id validation before any path construction (FD-4), then the Project membership check. */ + const scope = (c: Context) => { + const projectId = requireValidId(c, "projectId"); + const agentId = requireValidId(c, "agentId"); + deps.projectService.requireProjectAccess(c.var.user.userId, projectId); + return { projectId, agentId }; + }; + + app.get("/", async (c) => { + const { projectId, agentId } = scope(c); + return c.json(await deps.memoryService.overview(projectId, agentId)); + }); + + // The explicit adoption path for a template that predates Memory (idempotent config write). + app.post("/template-placeholder", async (c) => { + const { projectId, agentId } = scope(c); + return c.json(await deps.memoryService.insertTemplatePlaceholder(projectId, agentId)); + }); + + app.get("/scopes/:scopeKey/files", async (c) => { + const { projectId, agentId } = scope(c); + const key = pathParam(c, "scopeKey"); + return c.json(await deps.memoryService.listFiles(projectId, agentId, key)); + }); + + app.get("/scopes/:scopeKey/files/:fileName", async (c) => { + const { projectId, agentId } = scope(c); + const key = pathParam(c, "scopeKey"); + const name = pathParam(c, "fileName"); + return c.json(await deps.memoryService.readFile(projectId, agentId, key, name)); + }); + + app.delete("/scopes/:scopeKey/files/:fileName", async (c) => { + const { projectId, agentId } = scope(c); + const key = pathParam(c, "scopeKey"); + const name = pathParam(c, "fileName"); + await deps.memoryService.deleteFile(projectId, agentId, key, name); + return c.body(null, 204); + }); + + return app; +} diff --git a/packages/server/src/services/agent-config-service.ts b/packages/server/src/services/agent-config-service.ts index 432a1867..9adc5a9b 100644 --- a/packages/server/src/services/agent-config-service.ts +++ b/packages/server/src/services/agent-config-service.ts @@ -10,6 +10,8 @@ import fs from "node:fs/promises"; import { parseDocument, parse as parseYaml } from "yaml"; import { + DEFAULT_MEMORY_PROMPT, + DEFAULT_MEMORY_WORKSPACE_PROMPT, agentsMdPath, agentStateDir, agentStateVersion, @@ -30,6 +32,7 @@ import type { AgentConfigUpdateRequest, AgentModelConfigDto, AgentCompactionConfigDto, + AgentMemoryConfigDto, VaultEntryInfo, VaultResponse, VaultUpdateRequest, @@ -112,6 +115,7 @@ export class AgentConfigService { const parsed = asRecord(parseYaml(systemConfigYaml)); const model = asRecord(parsed.model); const compaction = asRecord(parsed.compaction); + const memory = asRecord(parsed.memory); const tools = asRecord(parsed.tools); let agentsMd = ""; @@ -140,6 +144,19 @@ export class AgentConfigService { : {}), ...(typeof compaction.prompt === "string" ? { prompt: compaction.prompt } : {}), }; + // Memory's effective state, not the literal YAML: core treats anything but an explicit + // `false` as on and falls back to the built-in prompts, so a config predating the section + // reports the values its Sessions actually run with (whether anything reaches the prompt + // still depends on the template carrying {{MEMORY}}, which the Memory tab reports + // separately). + const memoryDto: AgentMemoryConfigDto = { + enabled: memory.enabled !== false, + prompt: typeof memory.prompt === "string" ? memory.prompt : DEFAULT_MEMORY_PROMPT, + workspacePrompt: + typeof memory.workspace_prompt === "string" + ? memory.workspace_prompt + : DEFAULT_MEMORY_WORKSPACE_PROMPT, + }; const config: AgentConfigDto = { ...(typeof parsed.name === "string" ? { name: parsed.name } : {}), ...(typeof parsed.description === "string" ? { description: parsed.description } : {}), @@ -148,6 +165,7 @@ export class AgentConfigService { ...(typeof parsed.max_turns === "number" ? { maxTurns: parsed.max_turns } : {}), ...(Object.keys(modelDto).length > 0 ? { model: modelDto } : {}), ...(Object.keys(compactionDto).length > 0 ? { compaction: compactionDto } : {}), + memory: memoryDto, toolsBuiltin: Array.isArray(tools.builtin) ? (tools.builtin as ToolDefinitionConfig[]) : [], mcpServers: Array.isArray(tools.mcpServers) ? (tools.mcpServers as MCPServerConfig[]) : [], }; @@ -248,6 +266,14 @@ export class AgentConfigService { setIfProvided(["compaction", "mode"], optionalEnum(compaction, "mode", COMPACTION_MODES)); setIfProvided(["compaction", "prompt"], optionalString(compaction, "prompt")); } + if (cfg.memory !== undefined) { + // The toggle only decides whether Memory reaches the context and whether Workspace + // directories are prepared; existing Memory files are never touched by it. + const memory = asRecord(cfg.memory); + setIfProvided(["memory", "enabled"], optionalBoolean(memory, "enabled")); + setIfProvided(["memory", "prompt"], optionalString(memory, "prompt")); + setIfProvided(["memory", "workspace_prompt"], optionalString(memory, "workspacePrompt")); + } if (cfg.toolsBuiltin !== undefined) { doc.setIn(["tools", "builtin"], validateToolsBuiltin(cfg.toolsBuiltin)); } diff --git a/packages/server/src/services/agent-service.ts b/packages/server/src/services/agent-service.ts index 018876b0..855754bb 100644 --- a/packages/server/src/services/agent-service.ts +++ b/packages/server/src/services/agent-service.ts @@ -21,6 +21,7 @@ import { createAgent as coreCreateAgent, isValidId, loadAgentVault, + memoryDir, scheduleDir, skillsDir, systemConfigPath, @@ -28,6 +29,7 @@ import { import type { AgentsRepo } from "../db/repos/agents.js"; import { SEMANTIC_ID_PATTERN, SEMANTIC_ID_RULE } from "./ids.js"; import type { AgentConfigService } from "./agent-config-service.js"; +import { isTopicFileName } from "./memory-service.js"; export interface AgentListItem { agentId: string; @@ -46,6 +48,8 @@ export interface AgentListItem { scheduleCount: number; /** Number of installed Skills (count of skills// directories that contain a SKILL.md). */ skillCount: number; + /** Number of memory topic files across every scope directory under memory/ (independent of the memory switch, like skillCount). */ + memoryCount: number; } export class AgentService { @@ -91,13 +95,15 @@ export class AgentService { ); return Promise.all( sorted.map(async (row) => { - const [meta, updatedAt, vaultKeyCount, scheduleCount, skillCount] = await Promise.all([ - this.agentConfig.readCardMeta(projectId, row.agentId), - this.configUpdatedAt(projectId, row.agentId), - this.vaultKeyCount(projectId, row.agentId), - this.scheduleCount(projectId, row.agentId), - this.skillCount(projectId, row.agentId), - ]); + const [meta, updatedAt, vaultKeyCount, scheduleCount, skillCount, memoryCount] = + await Promise.all([ + this.agentConfig.readCardMeta(projectId, row.agentId), + this.configUpdatedAt(projectId, row.agentId), + this.vaultKeyCount(projectId, row.agentId), + this.scheduleCount(projectId, row.agentId), + this.skillCount(projectId, row.agentId), + this.memoryCount(projectId, row.agentId), + ]); return { agentId: row.agentId, ...meta, @@ -106,6 +112,7 @@ export class AgentService { vaultKeyCount, scheduleCount, skillCount, + memoryCount, }; }), ); @@ -154,6 +161,30 @@ export class AgentService { return present.filter(Boolean).length; } + /** Number of memory topic files: regular `*.md` files (minus each scope's MEMORY.md index) summed over the scope directories under memory/ (0 if the directory doesn't exist). */ + private async memoryCount(projectId: string, agentId: string): Promise { + const base = memoryDir(this.root, projectId, agentId); + let scopes: Dirent[]; + try { + scopes = await fs.readdir(base, { withFileTypes: true }); + } catch { + return 0; + } + const counts = await Promise.all( + scopes + .filter((d) => d.isDirectory()) + .map(async (d) => { + try { + const files = await fs.readdir(path.join(base, d.name), { withFileTypes: true }); + return files.filter((f) => f.isFile() && isTopicFileName(f.name)).length; + } catch { + return 0; + } + }), + ); + return counts.reduce((sum, n) => sum + n, 0); + } + /** Last config modification time: the later of system_config.yaml and AGENTS.md mtime; omitted if neither is readable. */ private async configUpdatedAt(projectId: string, agentId: string): Promise { const paths = [ @@ -251,6 +282,7 @@ export class AgentService { scheduleCount: 0, // Read the real count: coreCreateAgent seeds the default skill set for default_agent. skillCount: await this.skillCount(projectId, agentId), + memoryCount: 0, }; } } diff --git a/packages/server/src/services/memory-service.ts b/packages/server/src/services/memory-service.ts new file mode 100644 index 00000000..c2cb1424 --- /dev/null +++ b/packages/server/src/services/memory-service.ts @@ -0,0 +1,358 @@ +/** + * Memory management (`agent_state/memory/`): the Web App's read/delete access to what the Agent + * remembers between Sessions. + * + * The layout is core's (see core's state/memory.ts): one directory per scope holding Markdown + * topic files and that scope's own `MEMORY.md` index — `memory/user/` for the User scope and + * `memory//` for each Workspace. The User scope is addressed through the same + * `scopeKey` parameter as any Workspace (`USER_SCOPE_KEY`), so it needs no routes of its own; + * the one difference is that it may be created on demand, since it belongs to the Agent rather + * than to a Session that has run. + * + * The API is deliberately read + delete only: content changes go through a chat Session where + * the model edits the same files, keeping frontmatter and index in step. The one write this + * service performs is mechanical — deleting a topic file also drops its `]()` index lines, + * so the index never lists a file that is gone. + * + * This service never invents paths from client input — a request names an `agentId`, a + * `scopeKey` and a file name inside that scope, each validated against a character rule and + * then re-checked for containment after resolution, so neither `..` nor a symlink can reach + * outside the Agent's Memory directory. + * + * Files are the source of truth and are read fresh on every request (they are small, requests + * are rare, and the model edits the same files from its side). + */ +import fs from "node:fs/promises"; +import path from "node:path"; +import { + MEMORY_INDEX_FILENAME, + USER_SCOPE_KEY, + hasMemoryPlaceholder, + insertMemoryPlaceholder, + memoryDir, + memoryScopeDir, + parseMemoryFrontmatter, + readWorkspaceMarker, +} from "@prismshadow/penguin-core"; +import type { + MemoryFileInfo, + MemoryFilesResponse, + MemoryFileResponse, + MemoryOverviewResponse, + MemoryScopeInfo, +} from "../api/types.js"; +import { HttpError } from "../http/errors.js"; +import { badRequest } from "../http/validate.js"; +import type { AgentConfigService } from "./agent-config-service.js"; + +/** Scope directory names: what core's key generator produces (a safe base may start with `_`), plus the leeway of a hand-made directory. Excludes `.`/`..` and any separator, so the name can never climb out of `memory/`. */ +const SCOPE_KEY_PATTERN = /^[A-Za-z0-9_][A-Za-z0-9._-]*$/; + +/** + * Whether a name is a topic file: any Markdown file that is not a dotfile (`.workspace` is the + * Harness's), carries no path separator, and is not the index under any casing — macOS and + * Windows resolve `memory.md` to `MEMORY.md`. The model writes these files with the ordinary + * file tools, so non-ASCII names are as legitimate as kebab-case ones; the same rule guards + * client-supplied names, with containment re-checked after resolution. Exported so the Agent + * list's memory count (agent-service) shares this one definition of "a memory". + */ +export function isTopicFileName(name: string): boolean { + return ( + name.length > 0 && + !name.startsWith(".") && + !/[/\\]/.test(name) && + name.toLowerCase().endsWith(".md") && + name.toLowerCase() !== MEMORY_INDEX_FILENAME.toLowerCase() + ); +} + +export class MemoryService { + constructor( + private readonly root: string, + private readonly agentConfigService: AgentConfigService, + ) {} + + /** The tab's landing payload: the Agent-level switch and one entry per scope directory. */ + async overview(projectId: string, agentId: string): Promise { + const view = await this.agentConfigService.getConfig(projectId, agentId); + return { + enabled: view.config.memory.enabled, + templateHasMemory: hasMemoryPlaceholder(view.config.systemPrompt), + memoryDir: memoryDir(this.root, projectId, agentId), + scopes: await this.listScopes(projectId, agentId), + }; + } + + /** + * Inserts the `{{MEMORY}}` placeholder into the Agent's prompt template — the explicit + * adoption path for an Agent created before Memory shipped; nothing inserts automatically. + * Idempotent: a template that already carries it is left as it is (the refreshed overview + * reports `templateHasMemory` either way). + */ + async insertTemplatePlaceholder( + projectId: string, + agentId: string, + ): Promise { + const view = await this.agentConfigService.getConfig(projectId, agentId); + const next = insertMemoryPlaceholder(view.config.systemPrompt); + if (next !== view.config.systemPrompt) { + await this.agentConfigService.updateConfig(projectId, agentId, { + config: { systemPrompt: next }, + }); + } + return this.overview(projectId, agentId); + } + + /** + * The scope directories under `memory/`: the User scope first — always listed, even before it + * exists on disk, so a memory can be filed there without waiting for a Session to create it — + * then the Workspaces, newest activity first (one with no topic file yet sorts last, by key). + */ + async listScopes(projectId: string, agentId: string): Promise { + const base = memoryDir(this.root, projectId, agentId); + let entries: string[] = []; + try { + entries = (await fs.readdir(base, { withFileTypes: true })) + .filter((e) => e.isDirectory() && e.name !== USER_SCOPE_KEY) + .map((e) => e.name); + } catch { + // No memory/ directory yet (never initialized): the User scope entry below still stands. + } + const workspaces = await Promise.all( + entries.map((key) => this.scopeInfo(path.join(base, key), key, "workspace")), + ); + workspaces.sort((a, b) => { + if (a.updatedAt && b.updatedAt) return b.updatedAt.localeCompare(a.updatedAt); + if (a.updatedAt) return -1; + if (b.updatedAt) return 1; + return a.scopeKey.localeCompare(b.scopeKey); + }); + const userScope = await this.scopeInfo(path.join(base, USER_SCOPE_KEY), USER_SCOPE_KEY, "user"); + return [userScope, ...workspaces]; + } + + private async scopeInfo( + dir: string, + key: string, + kind: MemoryScopeInfo["kind"], + ): Promise { + const files = await this.topicFileNames(dir); + let latest = 0; + for (const name of files) { + try { + const stat = await fs.stat(path.join(dir, name)); + latest = Math.max(latest, stat.mtimeMs); + } catch { + // Raced with a delete: leave it out of the timestamp. + } + } + const workspacePath = kind === "workspace" ? await readWorkspaceMarker(dir) : undefined; + return { + scopeKey: key, + kind, + ...(workspacePath !== undefined ? { workspacePath } : {}), + fileCount: files.length, + ...(latest > 0 ? { updatedAt: new Date(latest).toISOString() } : {}), + }; + } + + /** Topic files of one scope: regular Markdown files only — the index, the `.workspace` marker, stray directories and symlinks (dirents that are not regular files) all stay out. */ + private async topicFileNames(dir: string): Promise { + try { + return (await fs.readdir(dir, { withFileTypes: true })) + .filter((e) => e.isFile() && isTopicFileName(e.name)) + .map((e) => e.name) + .sort((a, b) => a.localeCompare(b)); + } catch { + return []; + } + } + + async listFiles( + projectId: string, + agentId: string, + scopeKey: string, + ): Promise { + const dir = await this.requireScopeDir(projectId, agentId, scopeKey); + const names = await this.topicFileNames(dir); + const files: MemoryFileInfo[] = []; + for (const name of names) { + const info = await this.fileInfo(dir, name); + if (info) files.push(info); + } + return { scopeKey, files }; + } + + async readFile( + projectId: string, + agentId: string, + scopeKey: string, + fileName: string, + ): Promise { + const dir = await this.requireScopeDir(projectId, agentId, scopeKey); + const target = this.resolveFile(dir, fileName); + try { + // lstat, not stat: a symlink planted among the topic files (the model can create one with + // its file tools) must not lead the read outside the Memory directory. A non-regular file + // — or one deleted between the calls — is a 404, never a 500. + const stat = await fs.lstat(target); + if (!stat.isFile()) throw new Error("not a regular file"); + const content = await fs.readFile(target, "utf8"); + return { + scopeKey, + file: this.describe(fileName, content, stat.size, stat.mtime), + content, + }; + } catch { + throw new HttpError(404, "memory_file_not_found", `Memory file not found: ${fileName}`); + } + } + + /** + * Deletes a topic file and mechanically drops its lines from the scope's `MEMORY.md`: any + * line whose Markdown link target is exactly this file (`]()`) goes; a mention of the + * name in ordinary prose, or a link to a different file, survives. This is the only write the + * API performs on Memory content — everything else is the model's, through a chat Session. + */ + async deleteFile( + projectId: string, + agentId: string, + scopeKey: string, + fileName: string, + ): Promise { + const dir = await this.requireScopeDir(projectId, agentId, scopeKey); + const target = this.resolveFile(dir, fileName); + try { + await fs.unlink(target); + } catch { + throw new HttpError(404, "memory_file_not_found", `Memory file not found: ${fileName}`); + } + await this.pruneIndexLines(dir, fileName); + } + + /** Removes index lines linking to a deleted file. Best-effort: a missing or unreadable index is left alone. */ + private async pruneIndexLines(dir: string, fileName: string): Promise { + const indexPath = path.join(dir, MEMORY_INDEX_FILENAME); + let content: string; + try { + content = await fs.readFile(indexPath, "utf8"); + } catch { + return; + } + // Match the mechanical link forms an index line may use: `](file)`, `](./file)`, + // `]()`, `](file "title")`. A prose mention without the link form survives. + const escaped = fileName.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); + const link = new RegExp(`\\]\\(\\s*?\\s*(?:"[^"]*"\\s*)?\\)`); + const lines = content.split("\n"); + const kept = lines.filter((line) => !link.test(line)); + if (kept.length === lines.length) return; + await fs.writeFile(indexPath, kept.join("\n"), "utf8"); + } + + private async fileInfo(dir: string, name: string): Promise { + try { + const [content, stat] = await Promise.all([ + fs.readFile(path.join(dir, name), "utf8"), + fs.stat(path.join(dir, name)), + ]); + return this.describe(name, content, stat.size, stat.mtime); + } catch { + // Raced with a delete, or not readable: leave it out of the listing rather than failing the request. + return null; + } + } + + /** One listing entry: frontmatter (falling back to the file name for an unparsed file) plus file stats. */ + private describe(name: string, content: string, size: number, mtime: Date): MemoryFileInfo { + const meta = parseMemoryFrontmatter(content, name); + return { + name, + title: meta?.name ?? name, + description: meta?.description ?? "", + ...(meta?.updatedAt !== undefined ? { updatedAt: meta.updatedAt } : {}), + size, + modifiedAt: mtime.toISOString(), + }; + } + + /** + * Validates a scope key and returns its directory, 404 if it does not exist. The key is only + * ever a single directory name. + * + * A Workspace directory is created by a Session, so this API never creates one — a key with no + * directory means no Session has run there and there is nothing to list. The User scope is the + * exception: it belongs to the Agent rather than to any Session, so it is created on demand + * (Agents that predate Memory have no `memory/user/` until their next Session). + */ + private async requireScopeDir( + projectId: string, + agentId: string, + scopeKey: string, + ): Promise { + await this.agentConfigService.requireExists(projectId, agentId); + if (!SCOPE_KEY_PATTERN.test(scopeKey)) { + throw badRequest("scopeKey is invalid."); + } + const dir = memoryScopeDir(this.root, projectId, agentId, scopeKey); + if (scopeKey === USER_SCOPE_KEY) { + await fs.mkdir(dir, { recursive: true }); + return this.requireRealScopeDir(projectId, agentId, scopeKey, dir); + } + try { + if ((await fs.stat(dir)).isDirectory()) { + return this.requireRealScopeDir(projectId, agentId, scopeKey, dir); + } + } catch { + // Fall through to the 404 below. + } + throw new HttpError( + 404, + "memory_scope_not_found", + `No Memory directory for scope: ${scopeKey}`, + ); + } + + /** + * Symlink hardening for the scope directory itself: a scope smuggled in as a symlink (the + * model can create one with its file tools) would carry every read and delete outside + * `memory/`, so the resolved real path must be exactly `/` — not a + * link's target, wherever it points. + */ + private async requireRealScopeDir( + projectId: string, + agentId: string, + scopeKey: string, + dir: string, + ): Promise { + try { + const realBase = await fs.realpath(memoryDir(this.root, projectId, agentId)); + if ((await fs.realpath(dir)) === path.join(realBase, scopeKey)) return dir; + } catch { + // Unresolvable path: treat as absent. + } + throw new HttpError( + 404, + "memory_scope_not_found", + `No Memory directory for scope: ${scopeKey}`, + ); + } + + /** + * The absolute path of a topic file inside a scope directory. The name must pass the + * character rule, must not be the reserved index name, and the joined path is checked + * for containment — belt and braces, since the rule already excludes separators. + */ + private resolveFile(dir: string, fileName: string): string { + if (!isTopicFileName(fileName)) { + throw badRequest( + `File name must be a Markdown topic file — no path, no leading dot, and not the ${MEMORY_INDEX_FILENAME} index.`, + ); + } + const target = path.join(dir, fileName); + const rel = path.relative(dir, target); + if (rel !== fileName || rel.includes(path.sep)) { + throw badRequest("File name must not contain a path."); + } + return target; + } +} diff --git a/packages/server/test/memory.test.ts b/packages/server/test/memory.test.ts new file mode 100644 index 00000000..26c60ead --- /dev/null +++ b/packages/server/test/memory.test.ts @@ -0,0 +1,299 @@ +/** + * Integration tests for the Memory routes (agent_state/memory/): the overview reports the + * Agent-level switch and one entry per scope (user scope first), topic files can be listed / + * read / deleted, deleting a file prunes its index lines, path traversal in a scope key or + * file name is rejected, the switch round-trips through the Agent config without touching any + * file, and non-members see 404. + */ +import fs from "node:fs/promises"; +import path from "node:path"; +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { + MEMORY_INDEX_FILENAME, + USER_SCOPE_KEY, + memoryDir, + memoryScopeDir, +} from "@prismshadow/penguin-core"; +import type { + AgentConfigResponse, + MemoryFileResponse, + MemoryFilesResponse, + MemoryOverviewResponse, + ProjectCreateResponse, +} from "../src/api/types.js"; +import { apiClient, createTestApp, provisionUser } from "./helpers.js"; +import type { TestApp } from "./helpers.js"; + +const WORKSPACE_KEY = "my-app-a81f32c4"; +// The `type:` line is the retired field earlier files may still carry — listing must ignore it. +const TOPIC = `--- +name: testing-conventions +description: how tests are run here +type: feedback +updated_at: 2026-08-07 +--- + +- Integration tests talk to a real database. +`; + +describe("memory api", () => { + let t: TestApp; + let owner: ReturnType; + let outsider: ReturnType; + let projectId: string; + let memoryPath: string; + let configPath: string; + /** The Workspace Memory directory a Session would have created. */ + let wsDir: string; + + beforeEach(async () => { + t = await createTestApp(); + const a = await provisionUser(t.app, "owner_a"); + const c = await provisionUser(t.app, "outsider_c"); + owner = apiClient(t.app, a.cookie); + outsider = apiClient(t.app, c.cookie); + const created = (await ( + await owner.post("/api/projects", { projectId: "owner_a-memory", name: "memory project" }) + ).json()) as ProjectCreateResponse; + projectId = created.project.projectId; + memoryPath = `/api/projects/${projectId}/agents/default_agent/memory`; + configPath = `/api/projects/${projectId}/agents/default_agent/config`; + wsDir = memoryScopeDir(t.root, projectId, "default_agent", WORKSPACE_KEY); + await fs.mkdir(wsDir, { recursive: true }); + await fs.writeFile(path.join(wsDir, ".workspace"), "/home/dev/my-app\n", "utf8"); + }); + + afterEach(async () => { + await t.cleanup(); + }); + + const filesPath = (key = WORKSPACE_KEY) => `${memoryPath}/scopes/${key}/files`; + + it("overview reports the switch, the user scope, and one entry per Workspace", async () => { + await fs.writeFile(path.join(wsDir, "testing-conventions.md"), TOPIC, "utf8"); + + const body = (await (await owner.get(memoryPath)).json()) as MemoryOverviewResponse; + expect(body.enabled).toBe(true); + // A freshly created Agent gets the current default template, which carries {{MEMORY}}. + expect(body.templateHasMemory).toBe(true); + expect(body.memoryDir).toBe(memoryDir(t.root, projectId, "default_agent")); + expect(body.scopes).toHaveLength(2); + // The user scope leads the list (default_agent was initialized with it on disk). + expect(body.scopes[0]).toMatchObject({ + scopeKey: USER_SCOPE_KEY, + kind: "user", + fileCount: 0, + }); + expect(body.scopes[0]?.workspacePath).toBeUndefined(); + expect(body.scopes[1]).toMatchObject({ + scopeKey: WORKSPACE_KEY, + kind: "workspace", + workspacePath: "/home/dev/my-app", + fileCount: 1, + }); + expect(body.scopes[1]?.updatedAt).toBeTruthy(); + }); + + it("does not count a scope's MEMORY.md index as a topic file", async () => { + await fs.writeFile(path.join(wsDir, MEMORY_INDEX_FILENAME), "- [t](t.md) — hook\n", "utf8"); + const list = (await (await owner.get(filesPath())).json()) as MemoryFilesResponse; + expect(list.files).toHaveLength(0); + // Nor can the index be fetched or deleted as a topic file — under any casing, since + // macOS/Windows resolve memory.md to MEMORY.md. + expect((await owner.get(`${filesPath()}/${MEMORY_INDEX_FILENAME}`)).status).toBe(400); + expect((await owner.delete(`${filesPath()}/${MEMORY_INDEX_FILENAME}`)).status).toBe(400); + expect((await owner.get(`${filesPath()}/memory.md`)).status).toBe(400); + expect((await owner.delete(`${filesPath()}/Memory.Md`)).status).toBe(400); + }); + + it("accepts a workspace key starting with an underscore, as core generates for _site-style directories", async () => { + const key = "_site-1a2b3c4d"; + const dir = memoryScopeDir(t.root, projectId, "default_agent", key); + await fs.mkdir(dir, { recursive: true }); + await fs.writeFile(path.join(dir, "notes.md"), "---\nname: n\n---\nbody\n", "utf8"); + const list = (await (await owner.get(filesPath(key))).json()) as MemoryFilesResponse; + expect(list.files.map((f) => f.name)).toEqual(["notes.md"]); + }); + + it("lists, reads and deletes a non-ASCII topic file the model wrote", async () => { + const name = "项目背景.md"; + await fs.writeFile(path.join(wsDir, name), "---\nname: 项目背景\n---\n正文\n", "utf8"); + const list = (await (await owner.get(filesPath())).json()) as MemoryFilesResponse; + expect(list.files.map((f) => f.name)).toContain(name); + const encoded = `${filesPath()}/${encodeURIComponent(name)}`; + expect((await owner.get(encoded)).status).toBe(200); + expect((await owner.delete(encoded)).status).toBe(204); + expect(await fs.readdir(wsDir)).not.toContain(name); + }); + + it("neither lists nor follows a symlinked topic file", async () => { + const outside = path.join(t.root, "outside-secret.txt"); + await fs.writeFile(outside, "secret", "utf8"); + await fs.symlink(outside, path.join(wsDir, "leak.md")); + const list = (await (await owner.get(filesPath())).json()) as MemoryFilesResponse; + expect(list.files.map((f) => f.name)).not.toContain("leak.md"); + // Direct addressing must not follow the link either. + expect((await owner.get(`${filesPath()}/leak.md`)).status).toBe(404); + }); + + it("404s a scope directory smuggled in as a symlink", async () => { + const outside = path.join(t.root, "outside-dir"); + await fs.mkdir(outside, { recursive: true }); + await fs.writeFile(path.join(outside, "loot.md"), "---\nname: l\n---\nx\n", "utf8"); + await fs.symlink(outside, memoryScopeDir(t.root, projectId, "default_agent", "evil-12345678")); + expect((await owner.get(filesPath("evil-12345678"))).status).toBe(404); + }); + + it("lists the user scope of an Agent that predates Memory, creating it on demand", async () => { + // A Workspace directory comes from a Session, but the user scope belongs to the Agent. + await fs.rm(memoryScopeDir(t.root, projectId, "default_agent", USER_SCOPE_KEY), { + recursive: true, + force: true, + }); + const list = (await (await owner.get(filesPath(USER_SCOPE_KEY))).json()) as MemoryFilesResponse; + expect(list.files).toHaveLength(0); + await expect( + fs.stat(memoryScopeDir(t.root, projectId, "default_agent", USER_SCOPE_KEY)), + ).resolves.toBeTruthy(); + }); + + it("still 404s a Workspace key with no directory, so only the user scope is auto-created", async () => { + const res = await owner.get(filesPath("never-run-0badc0de")); + expect(res.status).toBe(404); + }); + + it("lists and reads topic files with their frontmatter, ignoring the .workspace marker", async () => { + await fs.writeFile(path.join(wsDir, "testing-conventions.md"), TOPIC, "utf8"); + + const list = (await (await owner.get(filesPath())).json()) as MemoryFilesResponse; + expect(list.files).toHaveLength(1); + expect(list.files[0]).toMatchObject({ + name: "testing-conventions.md", + title: "testing-conventions", + description: "how tests are run here", + updatedAt: "2026-08-07", + }); + // The retired type field stays out of the DTO even when the file still declares it. + expect(list.files[0]).not.toHaveProperty("type"); + + const read = (await ( + await owner.get(`${filesPath()}/testing-conventions.md`) + ).json()) as MemoryFileResponse; + expect(read.content).toBe(TOPIC); + }); + + it("deletes a topic file and prunes its index lines, leaving other lines alone", async () => { + await fs.writeFile(path.join(wsDir, "testing-conventions.md"), TOPIC, "utf8"); + await fs.writeFile(path.join(wsDir, "release-process.md"), "---\nname: r\n---\nbody\n", "utf8"); + await fs.writeFile( + path.join(wsDir, MEMORY_INDEX_FILENAME), + "- [Testing](./testing-conventions.md) — how tests are run here\n" + + "- [Release](release-process.md) — release steps\n" + + "Prose mentioning testing-conventions.md survives.\n", + "utf8", + ); + + expect((await owner.delete(`${filesPath()}/testing-conventions.md`)).status).toBe(204); + expect(await fs.readdir(wsDir)).not.toContain("testing-conventions.md"); + const index = await fs.readFile(path.join(wsDir, MEMORY_INDEX_FILENAME), "utf8"); + expect(index).not.toContain("](testing-conventions.md)"); + expect(index).toContain("- [Release](release-process.md) — release steps"); + // A plain-prose mention is not a link to the file; the mechanical edit leaves it be. + expect(index).toContain("Prose mentioning testing-conventions.md survives."); + + expect((await owner.delete(`${filesPath()}/testing-conventions.md`)).status).toBe(404); + }); + + it("rejects traversal and non-Markdown names, and an unknown Workspace", async () => { + // A key or file name that could climb out of the Memory directory never reaches the filesystem + // (the separator is percent-encoded, so it arrives as one path segment and is ours to reject). + expect((await owner.get(filesPath("..%2Fescape"))).status).toBe(400); + expect((await owner.get(`${filesPath()}/..%2Fescape.md`)).status).toBe(400); + expect((await owner.get(`${filesPath()}/notes.txt`)).status).toBe(400); + expect((await owner.delete(`${filesPath()}/.workspace`)).status).toBe(400); + expect((await owner.get(filesPath("never-seen-0badc0de"))).status).toBe(404); + }); + + it("toggles the Agent-level switch through the config route without touching any file", async () => { + await fs.writeFile(path.join(wsDir, "testing-conventions.md"), TOPIC, "utf8"); + + const off = await owner.put(configPath, { config: { memory: { enabled: false } } }); + expect(off.status).toBe(200); + expect(((await off.json()) as AgentConfigResponse).config.memory.enabled).toBe(false); + + // Turning Memory off keeps the files and the management API working; it only stops Memory + // from reaching the model's context. + const body = (await (await owner.get(memoryPath)).json()) as MemoryOverviewResponse; + expect(body.enabled).toBe(false); + expect(body.scopes.find((s) => s.scopeKey === WORKSPACE_KEY)?.fileCount).toBe(1); + + const on = await owner.put(configPath, { config: { memory: { enabled: true } } }); + expect(((await on.json()) as AgentConfigResponse).config.memory.enabled).toBe(true); + }); + + it("reports a template without the {{MEMORY}} placeholder and inserts it on request", async () => { + // Simulate an Agent from before Memory: replace the template with one lacking the placeholder. + const put = await owner.put(configPath, { + config: { systemPrompt: "# Role\nDo things.\n# Environment" }, + }); + expect(put.status).toBe(200); + let body = (await (await owner.get(memoryPath)).json()) as MemoryOverviewResponse; + expect(body.templateHasMemory).toBe(false); + + const inserted = await owner.post(`${memoryPath}/template-placeholder`, {}); + expect(inserted.status).toBe(200); + body = (await inserted.json()) as MemoryOverviewResponse; + expect(body.templateHasMemory).toBe(true); + + const cfg = (await (await owner.get(configPath)).json()) as AgentConfigResponse; + // Inserted at the position the default template gives it: before # Environment. + expect(cfg.config.systemPrompt.indexOf("{{MEMORY}}")).toBeGreaterThan(-1); + expect(cfg.config.systemPrompt.indexOf("{{MEMORY}}")).toBeLessThan( + cfg.config.systemPrompt.indexOf("# Environment"), + ); + + // Idempotent: a second call changes nothing and still succeeds. + const again = await owner.post(`${memoryPath}/template-placeholder`, {}); + expect(again.status).toBe(200); + const cfgAgain = (await (await owner.get(configPath)).json()) as AgentConfigResponse; + expect(cfgAgain.config.systemPrompt).toBe(cfg.config.systemPrompt); + }); + + it("round-trips the memory prompts through the config route, reporting defaults until set", async () => { + const before = (await (await owner.get(configPath)).json()) as AgentConfigResponse; + // A fresh default agent stores the built-in prompts in its own yaml. + expect(before.config.memory.prompt).toContain("{{USER_MEMORY_INDEX}}"); + expect(before.config.memory.workspacePrompt).toContain("## Workspace memory"); + + const put = await owner.put(configPath, { + config: { memory: { prompt: "# Memory\ncustom {{USER_MEMORY_INDEX}}" } }, + }); + expect(put.status).toBe(200); + const after = (await put.json()) as AgentConfigResponse; + expect(after.config.memory.prompt).toBe("# Memory\ncustom {{USER_MEMORY_INDEX}}"); + // The untouched half keeps its value. + expect(after.config.memory.workspacePrompt).toContain("## Workspace memory"); + }); + + it("reports the memory count on the Agent list, summed across scopes minus the indexes", async () => { + await fs.writeFile(path.join(wsDir, "testing-conventions.md"), TOPIC, "utf8"); + await fs.writeFile(path.join(wsDir, MEMORY_INDEX_FILENAME), "- [t](t.md) — hook\n", "utf8"); + const userDir = memoryScopeDir(t.root, projectId, "default_agent", USER_SCOPE_KEY); + await fs.mkdir(userDir, { recursive: true }); + await fs.writeFile(path.join(userDir, "prefers-pnpm.md"), "---\nname: p\n---\nx\n", "utf8"); + + const body = (await (await owner.get(`/api/projects/${projectId}/agents`)).json()) as { + agents: { agentId: string; memoryCount: number }[]; + }; + const agent = body.agents.find((a) => a.agentId === "default_agent"); + expect(agent?.memoryCount).toBe(2); + }); + + it("404s for a non-member on every Memory route", async () => { + expect((await outsider.get(memoryPath)).status).toBe(404); + expect((await outsider.post(`${memoryPath}/template-placeholder`, {})).status).toBe(404); + expect((await outsider.get(filesPath())).status).toBe(404); + expect((await outsider.get(`${filesPath()}/x.md`)).status).toBe(404); + expect((await outsider.delete(`${filesPath()}/x.md`)).status).toBe(404); + }); +}); diff --git a/packages/web/src/api/endpoints.ts b/packages/web/src/api/endpoints.ts index b69bb8f3..598e3b1e 100644 --- a/packages/web/src/api/endpoints.ts +++ b/packages/web/src/api/endpoints.ts @@ -35,6 +35,9 @@ import type { MemberAddRequest, MemberAddResponse, MembersResponse, + MemoryFileResponse, + MemoryFilesResponse, + MemoryOverviewResponse, MessagesResponse, ModelsResponse, ModelsUpdateRequest, @@ -205,6 +208,43 @@ export const putVault = (projectId: string, agentId: string, body: VaultUpdateRe { method: "PUT", body }, ); +// Memory (Agent-level, agent_state/memory/) ------------------------------------------------- + +/** Base path of an Agent's Memory API; the scope key and file name are single path segments (never a path). */ +const memoryBase = (projectId: string, agentId: string) => + `/api/projects/${encodeURIComponent(projectId)}/agents/${encodeURIComponent(agentId)}/memory`; + +const memoryFilesBase = (projectId: string, agentId: string, scopeKey: string) => + `${memoryBase(projectId, agentId)}/scopes/${encodeURIComponent(scopeKey)}/files`; + +export const getMemoryOverview = (projectId: string, agentId: string) => + apiFetch(memoryBase(projectId, agentId)); + +/** Inserts the {{MEMORY}} placeholder into the agent's prompt template (idempotent) — the explicit adoption path for an agent created before Memory. */ +export const insertMemoryPlaceholder = (projectId: string, agentId: string) => + apiFetch(`${memoryBase(projectId, agentId)}/template-placeholder`, { + method: "POST", + body: {}, + }); + +export const getMemoryFiles = (projectId: string, agentId: string, scopeKey: string) => + apiFetch(memoryFilesBase(projectId, agentId, scopeKey)); + +export const getMemoryFile = (projectId: string, agentId: string, scopeKey: string, name: string) => + apiFetch( + `${memoryFilesBase(projectId, agentId, scopeKey)}/${encodeURIComponent(name)}`, + ); + +export const deleteMemoryFile = ( + projectId: string, + agentId: string, + scopeKey: string, + name: string, +) => + apiFetch(`${memoryFilesBase(projectId, agentId, scopeKey)}/${encodeURIComponent(name)}`, { + method: "DELETE", + }); + // Agent & its configuration ---------------------------------------------------------------- export const listAgents = (projectId: string) => diff --git a/packages/web/src/features/agents/agent-settings-page.tsx b/packages/web/src/features/agents/agent-settings-page.tsx index 822b1581..d0786aca 100644 --- a/packages/web/src/features/agents/agent-settings-page.tsx +++ b/packages/web/src/features/agents/agent-settings-page.tsx @@ -1,8 +1,8 @@ /** - * Agent settings page: seven tabs — + * Agent settings page: eight tabs — * Overview (name/description/State path/active count/State version + snapshot * export-import + restore default configuration), Prompt (AGENTS.md and system_prompt editors + placeholder - * reference), Runtime (max_turns, model.*, compaction.*), Tools (editable built-in + * reference), Memory (memory-tab.tsx), Runtime (max_turns, model.*, compaction.*), Tools (editable built-in * tools table, MCP Server read-only JSON), Skills (skills-tab.tsx), Vault * (vault-tab.tsx), Schedule (schedules-tab.tsx). * Save = PUT config (sends only the changed keys; YAML comments are preserved @@ -34,11 +34,13 @@ import { Switch } from "../../components/ui/switch"; import { ConfirmModal, useSaveConfirm } from "../../components/ui/confirm-modal"; import { Skeleton } from "../../components/ui/skeleton"; import { SkillsTab } from "./skills-tab"; +import { MemoryTab } from "./memory-tab"; import { VaultTab } from "./vault-tab"; import { SchedulesTab } from "./schedules-tab"; import { thinkingLevelOptionsFor } from "../chat/thinking-level"; -type TabKey = "overview" | "prompt" | "runtime" | "tools" | "skills" | "vault" | "schedules"; +type TabKey = + "overview" | "prompt" | "memory" | "runtime" | "tools" | "skills" | "vault" | "schedules"; /** * Dropdown rows from a dictionary's [value, description] pairs (exported for unit tests). @@ -91,6 +93,7 @@ export function AgentSettingsPage() { const TABS = [ { key: "overview", label: S.agent.tabOverview }, { key: "prompt", label: S.agent.tabPrompt }, + { key: "memory", label: S.agent.tabMemory }, { key: "runtime", label: S.agent.tabRuntime }, { key: "tools", label: S.agent.tabTools }, { key: "skills", label: S.agent.tabSkills }, @@ -129,20 +132,29 @@ export function AgentSettingsPage() { // Only the initial config load failure renders inline (the page can't show without it); saves/imports report via toast. const [error, setError] = useState(null); - const load = useCallback(() => { - if (!projectId || !agentId) return; - setData(null); - setError(null); - api - .getAgentConfig(projectId, agentId) - .then(setData) - .catch((e: unknown) => setError(apiErrorText(e))); - }, [projectId, agentId]); + const load = useCallback( + (opts?: { keepStale?: boolean }) => { + if (!projectId || !agentId) return; + // keepStale refreshes in place: dropping data would skeleton the page, unmount the tab + // tree and lose unsaved editor state. Identity changes and whole-state replacements + // (import / config reset) still clear, so no stale agent's config ever shows. + if (!opts?.keepStale) setData(null); + setError(null); + api + .getAgentConfig(projectId, agentId) + .then(setData) + .catch((e: unknown) => setError(apiErrorText(e))); + }, + [projectId, agentId], + ); useEffect(() => { load(); }, [load]); + /** Memory-tab config writes (switch, placeholder insert, prompt save): refresh the page's config copy without unmounting the tabs. */ + const refreshConfig = useCallback(() => load({ keepStale: true }), [load]); + /** Snapshot import succeeded: show the new version and reload the whole config (import overwrites the entire Agent State, so every tab's data needs a refresh). */ const onImported = useCallback( (version: number) => { @@ -167,8 +179,13 @@ export function AgentSettingsPage() { const res = await api.putAgentConfig(projectId, agentId, update); setData(res); toastSuccess(S.common.saved); - // Name/description changes affect the breadcrumb and list display. - if (update.config?.name !== undefined || update.config?.description !== undefined) { + // Name/description changes affect the breadcrumb and list display; a builtin-tools + // change moves the card's tool count. + if ( + update.config?.name !== undefined || + update.config?.description !== undefined || + update.config?.toolsBuiltin !== undefined + ) { void reloadAgents(); } } catch (e) { @@ -232,6 +249,7 @@ export function AgentSettingsPage() { /> )} {tab === "prompt" && } + {tab === "memory" && } {tab === "runtime" && } {tab === "tools" && } {tab === "skills" && } diff --git a/packages/web/src/features/agents/agents-page.tsx b/packages/web/src/features/agents/agents-page.tsx index 46f07636..1e307eb9 100644 --- a/packages/web/src/features/agents/agents-page.tsx +++ b/packages/web/src/features/agents/agents-page.tsx @@ -5,8 +5,8 @@ * Info column has three lines: title line (small avatar + bold name + agentId); single-line * truncated description; and a stats line — icon + number only (Session count / tool count) plus * relative time (today/yesterday/n days ago), with meaning folded into the hover title; the - * tool / vault-key / schedule / skill counts deep-link to the settings page's matching tab - * (?tab=tools|vault|schedules|skills). + * memory / tool / skill / vault-key / schedule counts deep-link to the settings page's matching + * tab (?tab=memory|tools|skills|vault|schedules) and appear in the settings tabs' order. * Buttons sit to the right of the sparkline: "New Chat" (draft state, same as sidebar group * header) and "Settings" (goes to settings page) show text labels; "Usage" / "Traces" (deep link * via ?agentId= to the usage center / trace observability; traces use an eye line icon = @@ -65,6 +65,9 @@ const CARD_ICONS = { usage: "M4 20V10m6 10V4m6 16v-7m4 7H2", /** Traces (eye line icon: observability; follows text color, no fill) */ traces: "M2 12s3.6-7 10-7 10 7 10 7-3.6 7-10 7-10-7-10-7zM12 9a3 3 0 1 0 0 6 3 3 0 0 0 0-6z", + /** Memory (brain: two hemispheres + inner fold, lucide simplified), opens the settings tab */ + memory: + "M12 5a3 3 0 1 0-5.997.125 4 4 0 0 0-2.526 5.77 4 4 0 0 0 .556 6.588A4 4 0 1 0 12 18ZM12 5a3 3 0 1 1 5.997.125 4 4 0 0 1 2.526 5.77 4 4 0 0 1-.556 6.588A4 4 0 1 1 12 18ZM15 13a4.5 4.5 0 0 1-3-4 4.5 4.5 0 0 1-3 4", } as const; /** @@ -152,7 +155,10 @@ export function AgentsPage() { * page lands directly on the matching tab (unknown keys fall back to Overview there, so * "skills" is harmless until the Skills tab ships). */ - const openSettingsTab = (agentId: string, tab: "tools" | "vault" | "schedules" | "skills") => { + const openSettingsTab = ( + agentId: string, + tab: "tools" | "vault" | "schedules" | "skills" | "memory", + ) => { setCurrentAgentId(agentId); navigate(`/agents/${agentId}?tab=${tab}`); }; @@ -245,10 +251,10 @@ export function AgentsPage() {

{/* Stats on their own line: same color/font size as the description; each reserves a minimum width so they align vertically across cards; meaning - folded into the hover title. Tool/vault/schedule/skill counts are buttons - deep-linking to the matching settings tab (also for built-in Agents — - their Settings entry point has no gating either); session count and - last-modified stay plain text */} + folded into the hover title. Memory/tool/skill/vault/schedule counts are + buttons deep-linking to the matching settings tab, listed in the settings + tabs' order (also for built-in Agents — their Settings entry point has no + gating either); session count and last-modified stay plain text */}
{a.sessionCount} + + - 0 ? `${base}${trimmed}` : base; +} + +/** + * The add modal's draft: content (pasted text, a file path, or a URL — the agent reads sources + * itself, so no source classification is needed) into the scope the button was clicked on. + * Content is required; the modal keeps its actions disabled while it is empty. + */ +export function buildMemoryAddPrompt(kind: "user" | "workspace", content: string): string { + return `${S.memory.addPromptLead[kind]}\n${content.trim()}`; +} diff --git a/packages/web/src/features/agents/memory-tab.tsx b/packages/web/src/features/agents/memory-tab.tsx new file mode 100644 index 00000000..421e299d --- /dev/null +++ b/packages/web/src/features/agents/memory-tab.tsx @@ -0,0 +1,759 @@ +/** + * Agent settings page "Memory" tab: the switch, then every memory the Agent keeps, grouped by + * scope — user memory first (read by every Session), then one group per Workspace (labeled by + * its `.workspace` path, newest activity first). + * + * The tab is read + delete only, matching the API: a memory's content is the model's document, + * so both "edit" and each scope header's "add" open a bridge modal first — a text field and + * a live preview of the generated prompt, the same shape as the skill import modal — and then + * jump to a new chat with this Agent and the prompt as the prefilled draft (the same + * draft-cache route). For a Workspace scope the draft also pins that Workspace, so the Session + * is injected with the very index it is about to change. Deleting confirms first and also + * drops the file's MEMORY.md index lines (server-side). + * + * The switch writes immediately rather than joining a tab-level Save, so turning Memory off + * never drags an unrelated half-finished edit along with it. Off keeps every file and this tab + * fully usable; it only stops Memory from entering the context and from preparing directories + * for new Sessions. + */ +import { useCallback, useEffect, useRef, useState } from "react"; +import type { RefObject } from "react"; +import { useNavigate } from "react-router"; +import type { MemoryFileInfo, MemoryScopeInfo } from "@prismshadow/penguin-server/api"; +import * as api from "../../api/endpoints"; +import { S } from "../../lib/strings"; +import { apiErrorText } from "../../lib/api-error"; +import { formatRelativeDate } from "../../lib/format"; +import { useAuth } from "../../state/auth"; +import { useLocale } from "../../state/locale"; +import { useProject } from "../../state/project"; +import { Button } from "../../components/ui/button"; +import { GlyphIcon } from "../../components/ui/glyph-icon"; +import { Modal } from "../../components/ui/modal"; +import { Textarea } from "../../components/ui/input"; +import { Switch } from "../../components/ui/switch"; +import { Chevron } from "../../components/ui/chevron"; +import { Drawer } from "../../components/ui/drawer"; +import { Sheet, type SheetSnap } from "../../components/ui/sheet"; +import { ConfirmModal, useSaveConfirm } from "../../components/ui/confirm-modal"; +import { SkeletonList } from "../../components/ui/skeleton"; +import { toastError, toastSuccess } from "../../components/ui/toast"; +import { Md } from "../chat/md"; +import { DRAFT_SESSION_ID } from "../chat/chat-page"; +import { draftKey, loadDraft, saveDraft } from "../chat/draft-cache"; +import { buildMemoryAddPrompt, buildMemoryEditPrompt } from "./memory-chat-prompts"; + +/** The body without its frontmatter block: the drawer's metadata header already shows those fields, so rendering the raw YAML too would only repeat them. */ +function bodyWithoutFrontmatter(content: string): string { + return content.replace(/^\ufeff?---\r?\n[\s\S]*?\r?\n---\r?\n?/, ""); +} + +/** Same breakpoint as the chat page's panels: \u22651024px the view opens as a side Drawer, below it as a bottom Sheet. */ +const DESKTOP_QUERY = "(min-width: 1024px)"; + +/** Row-action glyphs (icon-only buttons, the skills tab's affordance): view = the agents page's eye, edit = the shared pencil-line, delete = the shared trash can. */ +const EYE_ICON = + "M2 12s3.6-7 10-7 10 7 10 7-3.6 7-10 7-10-7-10-7zM12 9a3 3 0 1 0 0 6 3 3 0 0 0 0-6z"; +const PENCIL_ICON = "M12 20h9M16.5 3.5a2.1 2.1 0 0 1 3 3L7 19l-4 1 1-4L16.5 3.5z"; +const TRASH_ICON = + "M4 7h16M9 7V5a1 1 0 0 1 1-1h4a1 1 0 0 1 1 1v2m3 0l-1 13a2 2 0 0 1-2 2H9a2 2 0 0 1-2-2L6 7m4 4v6m4-6v6"; +/** "Add" plus glyph on the group headers, matching the models page's per-group add entry. */ +const PLUS_ICON = "M12 5v14M5 12h14"; + +/** + * Collapsed scope keys, persisted per user \u00d7 Project \u00d7 Agent (localStorage, same conventions as + * the chat draft cache: userId in the key against cross-account leaks, tolerant reads, silent + * best-effort writes). Only collapsed keys are stored, so new scopes start expanded. + */ +const collapsedStoreKey = (userId: string, projectId: string, agentId: string): string => + `penguin.memoryCollapsed.${userId}.${projectId}.${agentId}`; + +function readCollapsedScopes(key: string | null): Set { + if (!key) return new Set(); + try { + const parsed: unknown = JSON.parse(localStorage.getItem(key) ?? "[]"); + if (!Array.isArray(parsed)) return new Set(); + return new Set(parsed.filter((s): s is string => typeof s === "string")); + } catch { + return new Set(); + } +} + +function writeCollapsedScopes(key: string | null, collapsed: ReadonlySet): void { + if (!key) return; + try { + localStorage.setItem(key, JSON.stringify([...collapsed])); + } catch { + // Quota / private browsing: the collapse state just won't survive the visit. + } +} + +/** One scope group as the tab renders it: the overview entry plus its listed files. */ +interface ScopeGroup { + scope: MemoryScopeInfo; + files: MemoryFileInfo[]; +} + +/** A memory selected for an action (view drawer / delete confirm). */ +interface Selected { + scope: MemoryScopeInfo; + file: MemoryFileInfo; +} + +export function MemoryTab({ + agentId, + onConfigChanged, +}: { + agentId: string; + /** Config writes happen here directly, so the settings page must refetch its own copy — otherwise a later Prompt-tab save from stale data would silently revert them (e.g. the inserted placeholder). */ + onConfigChanged?: () => void; +}) { + const navigate = useNavigate(); + const { locale } = useLocale(); + const userId = useAuth().user?.userId ?? null; + const { currentProject, setCurrentAgentId, reloadAgents } = useProject(); + const projectId = currentProject?.projectId ?? null; + + const [enabled, setEnabled] = useState(true); + const [templateHasMemory, setTemplateHasMemory] = useState(true); + const [memoryDir, setMemoryDir] = useState(""); + const [groups, setGroups] = useState(null); + // Tab-level error is the initial load failure only; actions report via toast. + const [error, setError] = useState(null); + const [switchBusy, setSwitchBusy] = useState(false); + const collapseKey = userId && projectId ? collapsedStoreKey(userId, projectId, agentId) : null; + const [collapsed, setCollapsed] = useState>(() => readCollapsedScopes(collapseKey)); + const [memoryPrompt, setMemoryPrompt] = useState(""); + const [workspacePrompt, setWorkspacePrompt] = useState(""); + const mainPromptRef = useRef(null); + const workspacePromptRef = useRef(null); + // Chip clicks steal focus, so track the last-focused prompt field instead of the current one. + const [lastPromptField, setLastPromptField] = useState<"main" | "workspace">("main"); + const { requestSave, element: saveConfirm } = useSaveConfirm(); + // Open flag and content are separate: the Sheet animates out on close, and nulling the + // content with it would empty the panel mid-exit. The stale content is simply kept. + const [viewOpen, setViewOpen] = useState(false); + const [viewing, setViewing] = useState<(Selected & { content: string }) | null>(null); + const [editing, setEditing] = useState(null); + const [editRequirement, setEditRequirement] = useState(""); + const [adding, setAdding] = useState(null); + const [addContent, setAddContent] = useState(""); + const [removing, setRemoving] = useState(null); + // ≥1024px the view opens as a right Drawer, below as a bottom Sheet — same live-updating + // breakpoint as the chat page's panels; the two are mounted mutually exclusively. + const [isDesktop, setIsDesktop] = useState(() => window.matchMedia(DESKTOP_QUERY).matches); + const [sheetSnap, setSheetSnap] = useState("half"); + + useEffect(() => { + const mq = window.matchMedia(DESKTOP_QUERY); + const onChange = (e: MediaQueryListEvent) => setIsDesktop(e.matches); + mq.addEventListener("change", onChange); + return () => mq.removeEventListener("change", onChange); + }, []); + + // The collapse state follows its storage slot (login/project switches swap the key in place). + useEffect(() => { + setCollapsed(readCollapsedScopes(collapseKey)); + }, [collapseKey]); + + const toggleCollapsed = (scopeKey: string) => { + const next = new Set(collapsed); + if (!next.delete(scopeKey)) next.add(scopeKey); + writeCollapsedScopes(collapseKey, next); + setCollapsed(next); + }; + + const load = useCallback(async () => { + if (!projectId || !agentId) return; + setGroups(null); + setError(null); + try { + const [overview, configView] = await Promise.all([ + api.getMemoryOverview(projectId, agentId), + api.getAgentConfig(projectId, agentId), + ]); + setMemoryPrompt(configView.config.memory.prompt); + setWorkspacePrompt(configView.config.memory.workspacePrompt); + setEnabled(overview.enabled); + setTemplateHasMemory(overview.templateHasMemory); + setMemoryDir(overview.memoryDir); + // Stored collapse keys for scopes that no longer exist are pruned on sight, so the + // entry doesn't linger forever in localStorage. + const live = new Set(overview.scopes.map((s) => s.scopeKey)); + setCollapsed((prev) => { + const next = new Set([...prev].filter((k) => live.has(k))); + if (next.size === prev.size) return prev; + writeCollapsedScopes(collapseKey, next); + return next; + }); + // Files are the source of truth and each scope is one request; fetch them in parallel. + setGroups( + await Promise.all( + overview.scopes.map(async (scope) => { + try { + return { + scope, + files: (await api.getMemoryFiles(projectId, agentId, scope.scopeKey)).files, + }; + } catch { + // One unreadable scope (bad hand-made directory name, raced delete) must not + // blank the whole tab; it lists as an empty group instead. + return { scope, files: [] }; + } + }), + ), + ); + } catch (e) { + setError(apiErrorText(e)); + } + }, [projectId, agentId, collapseKey]); + + useEffect(() => { + void load(); + }, [load]); + + const toggleEnabled = async (next: boolean) => { + if (!projectId) return; + setSwitchBusy(true); + try { + const res = await api.putAgentConfig(projectId, agentId, { + config: { memory: { enabled: next } }, + }); + setEnabled(res.config.memory.enabled); + toastSuccess(S.common.saved); + onConfigChanged?.(); + } catch (e) { + toastError(apiErrorText(e)); + } finally { + setSwitchBusy(false); + } + }; + + /** The explicit adoption path for an agent whose template predates Memory: one idempotent config write. */ + const insertPlaceholder = async () => { + if (!projectId) return; + try { + const overview = await api.insertMemoryPlaceholder(projectId, agentId); + setTemplateHasMemory(overview.templateHasMemory); + toastSuccess(S.memory.insertPlaceholderDone); + onConfigChanged?.(); + } catch (e) { + toastError(apiErrorText(e)); + } + }; + + /** Same contract as the Prompt tab's inserter: execCommand keeps the textarea's native undo stack, with a state-splice fallback. */ + const insertPromptToken = ( + ref: RefObject, + value: string, + setValue: (next: string) => void, + token: string, + ) => { + const el = ref.current; + if (el) { + el.focus(); + const inserted = document.execCommand?.("insertText", false, token); + if (inserted) return; // onChange updates state from e.target.value + } + const start = el ? el.selectionStart : value.length; + const end = el ? el.selectionEnd : value.length; + setValue(value.slice(0, start) + token + value.slice(end)); + requestAnimationFrame(() => { + if (!el) return; + el.focus(); + const caret = start + token.length; + el.setSelectionRange(caret, caret); + }); + }; + + /** Saves both memory prompts through the ordinary config write (confirm-first, like the other settings tabs). */ + const savePrompts = () => + requestSave(() => { + if (!projectId) return; + void api + .putAgentConfig(projectId, agentId, { + config: { memory: { prompt: memoryPrompt, workspacePrompt } }, + }) + .then((res) => { + setMemoryPrompt(res.config.memory.prompt); + setWorkspacePrompt(res.config.memory.workspacePrompt); + toastSuccess(S.common.saved); + onConfigChanged?.(); + }) + .catch((e: unknown) => toastError(apiErrorText(e))); + }); + + const openView = async (scope: MemoryScopeInfo, file: MemoryFileInfo) => { + if (!projectId) return; + try { + const res = await api.getMemoryFile(projectId, agentId, scope.scopeKey, file.name); + setSheetSnap("half"); // Every mobile open starts at the browsing height, like the chat panels. + setViewing({ scope, file: res.file, content: res.content }); + setViewOpen(true); + } catch (e) { + toastError(apiErrorText(e)); + } + }; + + const memoryFilePath = (scope: MemoryScopeInfo, file: MemoryFileInfo) => + `${memoryDir}/${scope.scopeKey}/${file.name}`; + + /** Opens the edit modal (closing the view panel if it is up): requirement field + prompt preview, then the chat jump. */ + const openEditor = (scope: MemoryScopeInfo, file: MemoryFileInfo) => { + setViewOpen(false); + setEditRequirement(""); + setEditing({ scope, file }); + }; + + const editPrompt = editing ? buildMemoryEditPrompt(editing.file.title, editRequirement) : ""; + + const copyEditPrompt = () => { + void navigator.clipboard + .writeText(editPrompt) + .then(() => toastSuccess(S.memory.editCopied)) + .catch(() => toastError(S.common.unknownError)); + }; + + /** + * The bridge-to-chat jump shared by edit and add: prefill the draft (merging over what is + * already cached, clearing a stale `/agent` handoff chip that would forward the prompt to a + * different Agent), pin this Agent — and for a Workspace scope pin that Workspace too, so the + * Session reads the very index it is about to change. + */ + const openChatWithDraft = (text: string, workspacePath: string | undefined) => { + if (!userId || !projectId) return; + const key = draftKey(userId, projectId); + saveDraft(key, { + ...loadDraft(key), + agentId, + text, + ...(workspacePath !== undefined ? { workspace: workspacePath } : {}), + skills: [], + handoffAgentId: undefined, + }); + setCurrentAgentId(agentId); + navigate(`/chat/${DRAFT_SESSION_ID}`, { + state: { + agentId, + ...(workspacePath !== undefined ? { workspace: workspacePath } : {}), + }, + }); + }; + + const openEditChat = () => { + if (editing) openChatWithDraft(editPrompt, editing.scope.workspacePath); + }; + + /** Opens the add modal for one scope: content empty, actions disabled until it is filled. */ + const openAdd = (scope: MemoryScopeInfo) => { + setAddContent(""); + setAdding(scope); + }; + + const addPrompt = adding ? buildMemoryAddPrompt(adding.kind, addContent) : ""; + + const copyAddPrompt = () => { + void navigator.clipboard + .writeText(addPrompt) + .then(() => toastSuccess(S.memory.editCopied)) + .catch(() => toastError(S.common.unknownError)); + }; + + const openAddChat = () => { + if (adding) openChatWithDraft(addPrompt, adding.workspacePath); + }; + + const confirmRemove = async () => { + if (!projectId || !removing) return; + const target = removing; + setRemoving(null); + try { + await api.deleteMemoryFile(projectId, agentId, target.scope.scopeKey, target.file.name); + toastSuccess(S.memory.deleteDone); + if (viewing && viewing.file.name === target.file.name) setViewOpen(false); + await load(); + // The agent card's memory count changed; refresh the list provider too. + void reloadAgents(); + } catch (e) { + toastError(apiErrorText(e)); + } + }; + + if (error) return

{error}

; + + const scopeTitle = (scope: MemoryScopeInfo): string => + scope.kind === "user" + ? S.memory.userScope + : (scope.workspacePath?.split(/[\\/]/).filter(Boolean).at(-1) ?? scope.scopeKey); + + // Icon-only row actions (the skills tab's affordance: neutral bordered icons, danger + // variant with red text/hover for delete); the tooltip + aria-label carry the wording. + const rowActions = (scope: MemoryScopeInfo, file: MemoryFileInfo) => ( +
+ + + +
+ ); + + const fileRow = (scope: MemoryScopeInfo, file: MemoryFileInfo) => ( +
  • +
    +

    + {file.title} +

    + {file.description && ( +

    {file.description}

    + )} +
    + + {file.updatedAt ?? formatRelativeDate(file.modifiedAt, locale)} + + {rowActions(scope, file)} +
    • + ); + + /** Metadata + rendered body of the memory in view, shared by the Drawer and the Sheet. */ + const viewMeta = viewing && ( + <> + + {viewing.file.updatedAt ?? formatRelativeDate(viewing.file.modifiedAt, locale)} + + {viewing.file.description && ( +

    {viewing.file.description}

    + )} +
    + +
    + + ); + + const viewActions = viewing && ( +
    + + +
    + ); + + return ( +
    +

    {S.memory.desc}

    + +
    +

    {S.memory.enable}

    + void toggleEnabled(v)} disabled={switchBusy} /> +
    + + {!templateHasMemory && ( +
    +

    {S.memory.templateMissing}

    + +
    + )} + + {groups === null ? ( + + ) : ( +
    + {groups.map(({ scope, files }) => { + const open = !collapsed.has(scope.scopeKey); + return ( +
    + {/* Group header (the models page's group convention): the collapse button fills + the row, the Add entry is a ghost text button to the chevron's left — a real + + + + + {/* Collapse arrow sits at the far right of the header (after the add entry); it too can be clicked to collapse. */} + +
    +
    + {/* inert while collapsed: rows at zero height shouldn't still be Tab-focusable or clickable. */} +
    + {files.length === 0 ? ( +

    + {scope.kind === "user" ? S.memory.emptyUserScope : S.memory.emptyScope} +

    + ) : ( +
      + {files.map((file) => fileRow(scope, file))} +
    + )} +
    +
    + + ); + })} +
    + )} + +
    +
    +

    + {S.memory.promptSection} +

    +

    + {S.memory.promptSectionHint} +

    +
    +