diff --git a/docs/architecture/agent-memory-system/spec.md b/docs/architecture/agent-memory-system/spec.md index 5d9a682dd..a8d013cca 100644 --- a/docs/architecture/agent-memory-system/spec.md +++ b/docs/architecture/agent-memory-system/spec.md @@ -27,7 +27,9 @@ These hold across every module and are the system's core invariants. 3. **Fail-open where losing data is the risk; fail-closed where writing wrong/stale data is the risk.** - Fail-open: triage failure → extract anyway; decision-model failure → degrade to `ADD`; neighbor recall failure → proceed with no neighbors; transient embedding-service failure → re-mark - `pending_embedding` for retry (never terminal `error`). + `pending_embedding` for retry; vector-write/dimension failures may mark rows `error`, but the + embedding drain periodically requeues a bounded batch after cooldown and the Health panel exposes a + per-agent reindex entry. - Fail-closed: a vector sidecar whose embedding identity (provider/model/dim) cannot be verified is disabled and recall serves FTS only — it never silently returns vectors from the wrong model. 4. **Never hard-delete on the durable path.** Contradiction and staleness use supersede chains and @@ -267,8 +269,8 @@ flowchart TD TRI -- KEEP --> EX["extraction → JSON candidates (≤8)"] EX --> CW["coordinateWrite per candidate"] CW --> DUP{provenance hit?} - DUP -- hit --> REV["absorbProvenanceHit
(revive → updated / noop)"] - DUP -- miss --> NB["retrieve top-10 neighbors"] + DUP -- hit --> REV["handleProvenanceHit
(restore / suppress / decision)"] + DUP -- miss --> NB["retrieveForDecision top-10 neighbors
(keyword any + no inline prune)"] NB --> DEC{decision ring} DEC --> ADD[ADD] DEC --> UPD["UPDATE (+confidence bump)"] @@ -318,11 +320,15 @@ unavailable to them. The cost saving comes from the triage gate avoiding the lar **Decision ring (`coordinateWrite`).** Each candidate: 1. Trim; empty → no-op. Teardown guard → no-op. -2. Compute the provenance key and check for an existing row. A hit calls `absorbProvenanceHit`: if it - revived an archived/superseded row it returns `updated`, else `noop(duplicate)`. -3. Otherwise retrieve up to **10** neighbors and run the decision model (`buildDecisionPrompt` → - `parseDecision`). Any decision-model failure or out-of-range target index degrades to `ADD` so a - hallucinated index can never touch the wrong row. +2. Compute the provenance key and check for an existing row. A pure scheduler-archived row is restored to + `pending_embedding`; a user/runtime-forgotten row is suppressed; an archived superseded conflict loser is + suppressed; an active duplicate returns `noop(duplicate)`. A non-archived superseded row does not revive + by provenance alone when a model path is available — its live chain head is fed into the decision ring. +3. Otherwise retrieve up to **10** decision neighbors through `retrieveForDecision`, which uses the + agent-facing keyword query in `any` mode and disables inline vector pruning. This keeps FTS-only agents + capable of semantic write decisions without requiring vectors. Any decision-model failure or out-of-range + target index degrades to `ADD` so a hallucinated index can never touch the wrong row. `UPDATE` and + `SUPERSEDE` re-read the target after the model await and only land on a still-live row. 4. Apply: `ADD` inserts with the candidate category; `UPDATE` rewrites the target content + bumps confidence + re-queues embedding and only absorbs the candidate category when the target category is `NULL`; `SUPERSEDE` inserts the merged row with the candidate category and supersedes the target; `NOOP` does @@ -337,9 +343,11 @@ path is a pure dedupe-add). **Two-phase persistence.** Phase 1 is a synchronous SQLite insert as `pending_embedding`. Phase 2 is an asynchronous, per-agent, fair, batched embedding drain: `listPendingEmbedding` filters by `agent_id` in SQL (no cross-agent starvation), one batched `getEmbeddings` call, and a single transactional upsert into DuckDB -under the per-agent lock. A transient embedding-service failure re-marks rows `pending_embedding` (self-heals, -never terminal); a dimension mismatch marks the row `error`; a vector-store write failure after a successful -embed is terminal `error`. +under the per-agent lock. A transient embedding-service failure re-marks rows `pending_embedding`; +a dimension mismatch or vector-store write failure after a successful embed marks the row `error`. The drain +does not leave `error` rows permanently stranded: when no normal pending work exists and the cooldown has +elapsed, it requeues up to the bounded retry batch and tries again. A manual per-agent reindex resets +embedding state and rebuilds from SQLite. **Cursor.** `memory_cursor_order_seq` (on `deepchat_sessions`) is written `SET = MAX(existing, floor(x), 0)`, so a late/stale extraction can never roll it back. It advances only when `extractAndStore` returns `ok: true` @@ -412,6 +420,9 @@ Important memories stretch their half-life, so they survive longer before becomi and forgetting are deliberately two different scores: a memory can rank low for recall yet still be retained. `category` is not a second scoring axis: it is ignored by retrieval, decay, RRF, and rerank logic. Its only ranking/retention effect is indirect, through the deterministic importance floor applied at write time. +Decay refresh uses the same formula whether computed in SQL or in the JavaScript fallback. When SQLite's +math functions are available, the maintenance pass updates `decay_score` in one statement and does not touch +`last_consolidated_at`; otherwise it falls back to the same calculation inside one repository transaction. --- @@ -468,6 +479,9 @@ by both current embedding fingerprint and current ready dimension before applyin row lifecycle plus dim/model before vector delete, and clears SQLite refs only if the row is still prunable with the same dim/model after deletion. Old-fingerprint or old-dimension refs remain as traceable metadata residue; maintenance does not cold-open those sidecars or let them occupy the current cleanup batch. +Consolidation timestamps are stamped with one SQL update over the same active non-internal row class. Working +memory refresh reads only the top ordered candidates it can use, and status counts are computed with a single +aggregate query rather than loading all rows. --- @@ -559,23 +573,31 @@ This is where the "stabilization" and "kernel hardening" work concentrates. - **Transactional vector upsert.** `upsert` wraps delete-then-insert in `BEGIN/COMMIT` with `ROLLBACK` on error, so there is no "deleted-but-not-inserted" hole. - **Archive / forget / restore lifecycle.** Agent-facing `forgetMemory` is a soft archive: it marks normal - rows `archived` and leaves recall correctness to status filters plus SQLite re-checks while dead-vector - pruning removes obsolete sidecar entries over time. It only requires a managed agent, so users can forget - while memory is disabled. `restoreMemory` re-marks a normal archived row `pending_embedding` and re-embeds - it, and remains gated by `canWriteAgentMemory` (managed agent · memory enabled · not disposed). Generic + rows `archived`, writes a content-free `memory/forget` runtime audit event even when the row was already + archived, and leaves recall correctness to status filters plus SQLite re-checks while dead-vector pruning + removes obsolete sidecar entries over time. It only requires a managed agent, so users can forget while + memory is disabled. `restoreMemory` re-marks a normal archived row `pending_embedding` and re-embeds it, + and remains gated by `canWriteAgentMemory` (managed agent · memory enabled · not disposed). Generic archive/forget/restore refuse `persona` and `working` rows; persona lifecycle is controlled by persona-specific routes and working rows are kernel-owned. Permanent UI delete (`deleteMemory`) hard-deletes - the row and best-effort deletes its vector. + the row and best-effort deletes its vector; if the sidecar is not already open, the manager may open the + current embedding store inside the per-agent vector lock using the known warm/current dimension. - **Embedding-drain config guard.** A background embedding drain captures the embedding identity it started with; before writing vectors, and before a reindex reset, it re-checks the agent's current `memoryEmbedding` fingerprint and discards the batch if the config changed mid-flight, so a stale drain can never write vectors from a superseded model into a freshly reset sidecar. -- **Provenance revival (`absorbProvenanceHit`).** When an archived/superseded fact is re-asserted, it is - revived and the contradicting supersede lineage is retired back into it so the revived row becomes current - truth (cycle- and cross-agent-guarded). `applyContentUpdate` keeps a row's `provenance_key` aligned with - its new content so dedup never breaks: if the new key is unowned it rewrites content + key in place; if the - key is already owned by a different row it folds this row into that owner instead (reviving the owner via - `absorbProvenanceHit`, then `markSuperseded(row, owner)`) and returns the owner's id. +- **Provenance and revival.** Provenance hits are split into classification and execution. Pure + scheduler-archived rows may be restored; rows with a user archive or runtime forget audit are suppressed + with `suppressed-user-forget`; archived rows that are also superseded are treated as conflict losers and + suppressed with `suppressed-conflict-loser`; active duplicates stay no-op. Non-archived superseded hits are + conservative without a model path and otherwise go through the decision ring. Only a decision-backed + `SUPERSEDE` collision can revive that old row and retire its former head. `applyContentUpdate` keeps a + row's `provenance_key` aligned with its new content; if the new key is owned by a suppressed row, the + update keeps the original row unchanged rather than reviving the suppressed owner. +- **Vector cleanup.** Vector deletion has three layers: direct delete opens the current sidecar when possible; + inline prune treats missing SQLite rows as prunable while retaining lifecycle/model/dimension guards for + existing rows; and warmup runs a one-shot keyset reconciliation under the vector lock to remove historical + orphan vectors from the current sidecar without using `LIMIT/OFFSET`. - **Close-safe teardown (`dispose`).** A `disposed` flag is set first so any already-fired timer's pass becomes a no-op; `canWriteAgentMemory` / `canReadAgentMemory` both include `!disposed` and are re-checked after every `await`. Dispose stops the startup maintenance timer, clears per-agent idle timers, @@ -612,12 +634,15 @@ inspect `result.ok`, not `isError`. Hard infra failures throw. `memory.restore`, `memory.getSourceSpan`, `memory.listConflicts`, `memory.resolveConflict`, `memory.listPersonaVersions`, `memory.rollbackPersona`, `memory.listPersonaDrafts`, `memory.approvePersonaDraft`, `memory.rejectPersonaDraft`, `memory.setPersonaAnchor`, -`memory.listAuditEvents`, `memory.listViewManifests`. +`memory.listAuditEvents`, `memory.listViewManifests`, `memory.reindex`. - `memory.search` is read-only: it uses a search-only depth override (default 50, route max 100) so the Memory Manager can return more than the agent's recall `topK`, excludes persona/working rows at the SQL search layer before applying result limits, and it does not bump `access_count`. - `memory.add` accepts optional `category`, runs the decision ring, and writes a `memory/add` user audit row. +- `memory.reindex` is a fire-and-forget per-agent rebuild entry for managed, memory-enabled DeepChat agents. + It returns `{ started }`, where `started=false` means the guard rejected the request or a reindex was already + in flight. - `memory.getSourceSpan` resolves a memory's `source_entry_ids` to readable role/content via the effective tape view (powers the lineage UI). - `MemoryItemSchema` carries `category`, `sourceEntryIds`, `conflictWith`, `personaState`, `isAnchor`, @@ -631,9 +656,9 @@ inspect `result.ok`, not `isError`. Hard infra failures throw. ### 15.4 Audit ledger (`agent_memory_audit`) -Background maintenance and user writes record `memory/maintenance_llm`, `memory/reflect`, `persona/evolve`, -`memory/challenge_resolved`, and `memory/add` — with `actorType` (writes use `scheduler` or `user`; `runtime` -is accepted by the schema/filter but no current write path emits it), an optional `session_id`, a terminal +Background maintenance and writes record `memory/maintenance_llm`, `memory/reflect`, `persona/evolve`, +`memory/challenge_resolved`, `memory/add`, and runtime `memory/forget` — with `actorType` +(`scheduler`, `user`, or `runtime`), an optional `session_id`, a terminal `status` (`completed`/`skipped`/`failed`), and `inputRefs`/`outputRefs` that contain only ids, action strings, counts, ratios, and booleans. No raw memory text or persona content is ever stored. @@ -694,7 +719,7 @@ enable memory (top-level Memory page / agent toggle) → cursor + effective-view span + source_entry_ids lineage + admission signals → fallback admission (visible text + tool/backstop/substantive text) or compaction → triage gate → extraction raw category candidates → normalization → decision ring - (ADD/UPDATE/SUPERSEDE/NOOP/CHALLENGE, with revival) + (ADD/UPDATE/SUPERSEDE/NOOP/CHALLENGE, with live-target recheck and decision-backed revival) → SQLite pending_embedding → cursor advances MAX + memory/extract anchor → background per-agent embedding (batched · fair · transactional) → DuckDB → [offline] self-scheduled sleep-time pass (6h cooldown): merge + conflict adjudication @@ -710,9 +735,10 @@ Coverage mirrors source under `test/main/**` (and `test/renderer/**` for UI), pi - Injection sanitization; per-session serial extraction lock; monotonic cursor; insert error classification. -- Vector upsert transaction + identity guard (fail-closed to FTS); reindex on dimension change. -- Decision ring (five branches + fallbacks); provenance revival; conflict closure / resolution; category - propagation and reflection/persona/working category guards. +- Vector upsert transaction + identity guard (fail-closed to FTS); reindex on dimension change; bounded + `error` retry; orphan-vector reconciliation. +- Decision ring (five branches + fallbacks); provenance suppression/revival matrix; live target re-check; + conflict closure / resolution; category propagation and reflection/persona/working category guards. - Dual-score forgetting / four-condition archival; offline consolidation (cooldown / budget / restart-durable / idle debounce); reflection recall; working blob; guarded persona (default-off / draft / anchor / eval gate). @@ -742,9 +768,9 @@ The real-DB FTS5/trigram (CJK) eval runs only when native `better-sqlite3` is lo - **DuckDB disk reclaim.** Per-memory hard delete and dead-vector pruning remove vector rows but do not shrink the DuckDB file; only a whole-store reset reclaims disk space (no `VACUUM`), so the file can still grow between resets. Archived/superseded vectors are correctness-safe because recall re-checks SQLite, but they - are not quality-neutral: if left in the HNSW result window they crowd out live candidates. Inline prune and - the warm-store-only, fingerprint+dimension-aware, lifecycle-guarded maintenance sweep remove current-sidecar - dead vectors over time. Management search disables inline prune so read-only search does not write DuckDB. + are not quality-neutral: if left in the HNSW result window they crowd out live candidates. Direct delete, + inline prune, and one-shot warm-store orphan reconciliation remove current-sidecar dead vectors over time. + Management search disables inline prune so read-only search does not write DuckDB. - **FTS5 native dependency.** Under vitest with an unloadable native ABI, the real FTS5/trigram eval skips; CI needs a working native build to exercise it. - **Vector query threshold.** `MemoryVectorStore.query` does not apply a distance cutoff itself; the @@ -781,6 +807,8 @@ The real-DB FTS5/trigram (CJK) eval runs only when native `better-sqlite3` is lo | `CONSOLIDATION_MERGE_SIMILARITY` | 0.85 | near-duplicate merge threshold | | `CONSOLIDATION_MAX_NEIGHBOR_SCANS` | 64 | stored-vector neighbor scans per consolidation pass | | `VECTOR_PRUNE_BATCH_LIMIT` | 256 | prunable archived/superseded/internal-kind vector refs per maintenance pass | +| `ERROR_RETRY_COOLDOWN_MS` / batch | 10min / 50 | bounded automatic retry for rows stuck in embedding `error` | +| `ORPHAN_RECONCILE_BATCH` | 512 | keyset page size for warm vector orphan reconciliation | | `RECALL_QUERY_EMBEDDING_TIMEOUT_MS` / stale / max concurrent | 800ms / 30s / 2 | foreground query-embedding soft timeout and per-agent+model cap | | `MEMORY_SEARCH_DEFAULT_LIMIT` | 50 | default management search depth | | `WORKING_BLOB_TOKEN_LIMIT` | 400 | working-memory blob size | diff --git a/src/main/presenter/memoryPresenter/index.ts b/src/main/presenter/memoryPresenter/index.ts index 667dfc64c..06b8c89f7 100644 --- a/src/main/presenter/memoryPresenter/index.ts +++ b/src/main/presenter/memoryPresenter/index.ts @@ -137,8 +137,8 @@ export class MemoryPresenter implements MemoryRuntimePort { this.maintenance = maintenanceService this.writeCoordinator = new WriteCoordinator(this.runtime, this.rows, { - retrieve: (agentId, query, now, recordAccessHits) => - this.retrieval.retrieve(agentId, query, now, recordAccessHits), + retrieveForDecision: (agentId, query, now) => + this.retrieval.retrieveForDecision(agentId, query, now), syncWorkingMemoryAfterMutation: (agentId) => this.workingMemory.syncWorkingMemoryAfterMutation(agentId), triggerEmbedding: (agentId) => this.embedding.processPendingEmbeddings(agentId), @@ -146,10 +146,14 @@ export class MemoryPresenter implements MemoryRuntimePort { }) this.management = new ManagementService(this.runtime, { - deleteVectorsForMemoryIds: (agentId, memoryIds) => - this.vectorStore.deleteVectorsForMemoryIds(agentId, memoryIds), + deleteVectorsForDeletedMemory: (agentId, memoryIds, embedding) => + this.vectorStore.deleteVectorsForMemoryIdsOpening(agentId, memoryIds, { + embeddingModel: embedding.embeddingModel, + embeddingDim: embedding.embeddingDim + }), resetAgentStore: (agentId) => this.vectorStore.resetAgentStore(agentId), isReindexing: (agentId) => this.embedding.isReindexing(agentId), + reindexEmbeddings: (agentId, force) => this.reindexEmbeddings(agentId, force), syncWorkingMemoryAfterMutation: (agentId) => this.workingMemory.syncWorkingMemoryAfterMutation(agentId), triggerEmbedding: (agentId) => this.embedding.processPendingEmbeddings(agentId), @@ -273,6 +277,10 @@ export class MemoryPresenter implements MemoryRuntimePort { return this.runtime.isEnabled(agentId) } + canReindex(agentId: string): boolean { + return this.runtime.canContinueAgentMemoryTask(agentId) + } + writeMemoriesSync(candidates: MemoryCandidate[], options: WriteMemoriesOptions): string[] { return this.writeCoordinator.writeMemoriesSync(candidates, options) } @@ -285,6 +293,10 @@ export class MemoryPresenter implements MemoryRuntimePort { return this.embedding.reindexEmbeddings(agentId, force) } + isReindexing(agentId: string): boolean { + return this.embedding.isReindexing(agentId) + } + backfillEmbeddings(agentId: string): Promise { return this.embedding.backfillEmbeddings(agentId) } diff --git a/src/main/presenter/memoryPresenter/infra/embeddingPipeline.ts b/src/main/presenter/memoryPresenter/infra/embeddingPipeline.ts index 18a4553cc..4931b6559 100644 --- a/src/main/presenter/memoryPresenter/infra/embeddingPipeline.ts +++ b/src/main/presenter/memoryPresenter/infra/embeddingPipeline.ts @@ -1,7 +1,11 @@ import logger from '@shared/logger' import { + ERROR_RETRY_BATCH_LIMIT, + ERROR_RETRY_COOLDOWN_MS, EMBEDDING_PREWARM_TEXT, + ORPHAN_RECONCILE_BATCH, + ORPHAN_RECONCILE_RETRY_COOLDOWN_MS, REINDEX_BATCH_SIZE, REINDEX_MAX_BATCHES, WARM_DIMENSION_FAILURE_COOLDOWN_MS @@ -26,6 +30,11 @@ interface EmbeddingPipelineRuntimeState { embeddingDrains: Map> reindexing: Map> backfilling: Map> + errorRetryAt: Map + errorRetryAfterId: Map + orphanVectorReconciles: Map> + orphanVectorReconciled: Set + orphanVectorReconcileRetryAt: Map } export class EmbeddingPipeline { @@ -35,6 +44,12 @@ export class EmbeddingPipeline { private readonly embeddingDrains = new Map>() private readonly reindexing = new Map>() private readonly backfilling = new Map>() + private readonly errorRetryAt = new Map() + private readonly errorRetryAfterId = new Map() + private readonly orphanVectorReconciles = new Map>() + private readonly orphanVectorReconcileTokens = new Map() + private readonly orphanVectorReconciled = new Set() + private readonly orphanVectorReconcileRetryAt = new Map() constructor( private readonly ctx: MemoryRuntimeContext, @@ -72,7 +87,40 @@ export class EmbeddingPipeline { private async drainPendingEmbeddings(agentId: string, limit: number): Promise { if (!this.ctx.canContinueAgentMemoryTask(agentId)) return const config = this.ctx.deps.resolveAgentConfig(agentId) - const pending = this.ctx.deps.repository.listPendingEmbedding(limit, agentId) + let pending = this.ctx.deps.repository.listPendingEmbedding(limit, agentId) + if (!pending.length) { + const lastRetryAt = this.errorRetryAt.get(agentId) ?? 0 + const now = Date.now() + if (now - lastRetryAt >= ERROR_RETRY_COOLDOWN_MS) { + let afterId = this.errorRetryAfterId.get(agentId) ?? null + let retryIds = this.ctx.deps.repository.listEmbeddingStatusIds( + agentId, + ['error'], + ERROR_RETRY_BATCH_LIMIT, + afterId + ) + if (!retryIds.length && afterId !== null) { + afterId = null + retryIds = this.ctx.deps.repository.listEmbeddingStatusIds( + agentId, + ['error'], + ERROR_RETRY_BATCH_LIMIT, + null + ) + } + if (retryIds.length) { + const requeued = this.ctx.deps.repository.requeueForEmbedding( + agentId, + ['error'], + retryIds.length, + afterId + ) + this.errorRetryAt.set(agentId, now) + this.errorRetryAfterId.set(agentId, retryIds[retryIds.length - 1]) + if (requeued > 0) pending = this.ctx.deps.repository.listPendingEmbedding(limit, agentId) + } + } + } if (!pending.length) return const embedding = config?.memoryEmbedding @@ -112,6 +160,7 @@ export class EmbeddingPipeline { this.rows.isPendingEmbeddableRow(agentId, this.ctx.deps.repository.getById(pending[i].id)) ) { this.ctx.deps.repository.updatePendingEmbeddingStatus(agentId, pending[i].id, 'error') + this.errorRetryAt.set(agentId, Date.now()) } } if (!records.length) return @@ -175,6 +224,7 @@ export class EmbeddingPipeline { } ) } else if (!outcome.usable) { + this.errorRetryAt.set(agentId, Date.now()) this.ctx.deps.repository.updatePendingEmbeddingStatus(agentId, record.memoryId, 'error') } } @@ -196,6 +246,7 @@ export class EmbeddingPipeline { const liveRows = pending.filter((row) => this.rows.isPendingEmbeddableRow(agentId, this.ctx.deps.repository.getById(row.id)) ) + if (liveRows.length) this.errorRetryAt.set(agentId, Date.now()) for (const row of liveRows) { this.ctx.deps.repository.updatePendingEmbeddingStatus(agentId, row.id, 'error') } @@ -209,6 +260,7 @@ export class EmbeddingPipeline { if (inflight) return inflight const tracked = this.runReindex(agentId, force).finally(() => { if (this.reindexing.get(agentId) === tracked) this.reindexing.delete(agentId) + if (!this.ctx.isDisposed) this.ctx.emitChanged(agentId, 'reindex') }) this.reindexing.set(agentId, tracked) return tracked @@ -230,6 +282,7 @@ export class EmbeddingPipeline { await locked.close() await this.ctx.deps.resetVectorStore(agentId) }) + this.clearOrphanReconcileMarks(agentId) if (!this.ctx.canContinueAgentMemoryTask(agentId)) return this.ctx.emitChanged(agentId, 'reindex') await this.drainUntilExhausted(agentId) @@ -337,6 +390,7 @@ export class EmbeddingPipeline { } this.vectorStore.markReady(agentId, embedding, dimensions) + this.scheduleOrphanVectorReconcile(agentId, embedding, dimensions, fingerprint) if (!this.reindexing.has(agentId)) { void this.ports.backfillEmbeddings(agentId).catch((error) => { logger.warn(`[Memory] backfill failed for ${agentId}: ${String(error)}`) @@ -344,6 +398,128 @@ export class EmbeddingPipeline { } } + private scheduleOrphanVectorReconcile( + agentId: string, + embedding: MemoryModelRef, + dimensions: number, + fingerprint: string + ): void { + const key = this.vectorStore.cacheKey(agentId, embedding, dimensions) + if (this.orphanVectorReconciled.has(key)) return + if (Date.now() < (this.orphanVectorReconcileRetryAt.get(key) ?? 0)) return + if (this.orphanVectorReconciles.has(key)) return + + const token = Symbol(key) + this.orphanVectorReconcileTokens.set(key, token) + const tracked = this.waitForBackgroundTick() + .then(() => + this.reconcileOrphanVectorsOnce(agentId, embedding, dimensions, fingerprint, key, token) + ) + .catch((error) => { + if (this.isCurrentOrphanReconcile(key, token)) { + this.orphanVectorReconcileRetryAt.set( + key, + Date.now() + ORPHAN_RECONCILE_RETRY_COOLDOWN_MS + ) + } + logger.warn(`[Memory] orphan vector reconcile failed for ${agentId}: ${String(error)}`) + }) + .finally(() => { + if (this.orphanVectorReconciles.get(key) === tracked) { + this.orphanVectorReconciles.delete(key) + if (this.isCurrentOrphanReconcile(key, token)) { + this.orphanVectorReconcileTokens.delete(key) + } + } + }) + this.orphanVectorReconciles.set(key, tracked) + } + + private isCurrentOrphanReconcile(key: string, token: symbol): boolean { + return this.orphanVectorReconcileTokens.get(key) === token + } + + private async reconcileOrphanVectorsOnce( + agentId: string, + embedding: MemoryModelRef, + dimensions: number, + fingerprint: string, + key: string, + token: symbol + ): Promise { + if (!this.isCurrentOrphanReconcile(key, token)) return + if (this.orphanVectorReconciled.has(key)) return + const retryAt = this.orphanVectorReconcileRetryAt.get(key) ?? 0 + if (Date.now() < retryAt) return + try { + const completed = await this.vectorStore.withAgentLock(agentId, async (locked) => { + if (!this.ctx.canUseCurrentMemoryEmbedding(agentId, embedding)) return false + const store = await locked.open(embedding, dimensions) + if (!store.isUsable()) { + return false + } + let afterId: string | null = null + for (let guard = 0; guard < REINDEX_MAX_BATCHES; guard += 1) { + const ids = await store.listMemoryIds(afterId, ORPHAN_RECONCILE_BATCH) + if (!ids.length) return true + afterId = ids[ids.length - 1] + const liveRows = this.ctx.deps.repository.listByIds(agentId, ids) + const liveIds = new Set(liveRows.map((row) => row.id)) + const missingIds = ids.filter((id) => !liveIds.has(id)) + if (missingIds.length) { + const prunableIds = [ + ...new Set( + this.ctx.deps.repository.filterPrunableVectorRefs( + agentId, + missingIds, + dimensions, + fingerprint + ) + ) + ] + for (let start = 0; start < prunableIds.length; start += ORPHAN_RECONCILE_BATCH) { + if (!this.ctx.canUseCurrentMemoryEmbedding(agentId, embedding)) return false + await store.deleteByMemoryIds( + prunableIds.slice(start, start + ORPHAN_RECONCILE_BATCH) + ) + } + } + if (ids.length < ORPHAN_RECONCILE_BATCH) { + return true + } + } + return false + }) + if (!this.isCurrentOrphanReconcile(key, token)) return + if (completed) { + this.orphanVectorReconciled.add(key) + this.orphanVectorReconcileRetryAt.delete(key) + } else { + this.orphanVectorReconcileRetryAt.set(key, Date.now() + ORPHAN_RECONCILE_RETRY_COOLDOWN_MS) + logger.warn( + `[Memory] orphan vector reconcile did not complete full scan for ${agentId}; retrying later` + ) + } + } catch (error) { + if (this.isCurrentOrphanReconcile(key, token)) { + this.orphanVectorReconcileRetryAt.set(key, Date.now() + ORPHAN_RECONCILE_RETRY_COOLDOWN_MS) + } + logger.warn(`[Memory] orphan vector reconcile failed for ${agentId}: ${String(error)}`) + } + } + + private clearOrphanReconcileMarks(agentId: string): void { + for (const key of this.orphanVectorReconciled) { + if (key.startsWith(`${agentId}::`)) this.orphanVectorReconciled.delete(key) + } + for (const key of this.orphanVectorReconcileRetryAt.keys()) { + if (key.startsWith(`${agentId}::`)) this.orphanVectorReconcileRetryAt.delete(key) + } + for (const key of this.orphanVectorReconcileTokens.keys()) { + if (key.startsWith(`${agentId}::`)) this.orphanVectorReconcileTokens.delete(key) + } + } + private async resolveWarmVectorDimensions( agentId: string, embedding: MemoryModelRef @@ -406,6 +582,7 @@ export class EmbeddingPipeline { ...this.reindexing.values(), ...this.backfilling.values(), ...this.embeddingDrains.values(), + ...this.orphanVectorReconciles.values(), ...this.vectorStoreWarmups.values(), ...this.embeddingWarmups.values() ] @@ -415,12 +592,14 @@ export class EmbeddingPipeline { const reindexing = this.reindexing.get(agentId) const backfilling = this.backfilling.get(agentId) const embeddingDrain = this.embeddingDrains.get(agentId) + const orphanReconciles = this.getAgentEntries(this.orphanVectorReconciles, agentId) const vectorWarmups = this.getAgentEntries(this.vectorStoreWarmups, agentId) const embeddingWarmups = this.getAgentEntries(this.embeddingWarmups, agentId) return [ reindexing, backfilling, embeddingDrain, + ...orphanReconciles.map(([, promise]) => promise), ...vectorWarmups.map(([, promise]) => promise), ...embeddingWarmups.map(([, promise]) => promise) ].filter((promise): promise is Promise => Boolean(promise)) @@ -440,6 +619,13 @@ export class EmbeddingPipeline { this.reindexing.delete(agentId) this.backfilling.delete(agentId) this.embeddingDrains.delete(agentId) + this.errorRetryAt.delete(agentId) + this.errorRetryAfterId.delete(agentId) + this.clearOrphanReconcileMarks(agentId) + for (const [key] of this.getAgentEntries(this.orphanVectorReconciles, agentId)) { + this.orphanVectorReconciles.delete(key) + this.orphanVectorReconcileTokens.delete(key) + } for (const [key] of this.getAgentEntries(this.vectorStoreWarmups, agentId)) { this.vectorStoreWarmups.delete(key) } @@ -458,6 +644,12 @@ export class EmbeddingPipeline { this.embeddingDrains.clear() this.reindexing.clear() this.backfilling.clear() + this.errorRetryAt.clear() + this.errorRetryAfterId.clear() + this.orphanVectorReconciles.clear() + this.orphanVectorReconcileTokens.clear() + this.orphanVectorReconciled.clear() + this.orphanVectorReconcileRetryAt.clear() } /** @internal Live mutable state for legacy facade-oracle tests only. */ @@ -468,7 +660,12 @@ export class EmbeddingPipeline { vectorStoreDimensionFailures: this.vectorStoreDimensionFailures, embeddingDrains: this.embeddingDrains, reindexing: this.reindexing, - backfilling: this.backfilling + backfilling: this.backfilling, + errorRetryAt: this.errorRetryAt, + errorRetryAfterId: this.errorRetryAfterId, + orphanVectorReconciles: this.orphanVectorReconciles, + orphanVectorReconciled: this.orphanVectorReconciled, + orphanVectorReconcileRetryAt: this.orphanVectorReconcileRetryAt } } } diff --git a/src/main/presenter/memoryPresenter/infra/memoryVectorStore.ts b/src/main/presenter/memoryPresenter/infra/memoryVectorStore.ts index c5a4b8e68..6f88ff42c 100644 --- a/src/main/presenter/memoryPresenter/infra/memoryVectorStore.ts +++ b/src/main/presenter/memoryPresenter/infra/memoryVectorStore.ts @@ -350,6 +350,28 @@ export class MemoryVectorStore implements IMemoryVectorStore { ) } + async listMemoryIds(afterId: string | null, limit: number): Promise { + const cappedLimit = Math.max(0, Math.floor(limit)) + if (cappedLimit === 0) return [] + const reader = afterId + ? await this.connection.runAndReadAll( + `SELECT memory_id + FROM ${this.vectorTable} + WHERE memory_id > ? + ORDER BY memory_id + LIMIT ?;`, + [afterId, cappedLimit] + ) + : await this.connection.runAndReadAll( + `SELECT memory_id + FROM ${this.vectorTable} + ORDER BY memory_id + LIMIT ?;`, + [cappedLimit] + ) + return reader.getRowObjectsJson().map((row: Record) => String(row.memory_id)) + } + /** * Delete an agent's store files from disk without needing an open instance (e.g. after * restart). `force` ignores missing files; a real failure (lock/permission) is thrown so @@ -374,7 +396,7 @@ export class MemoryVectorStore implements IMemoryVectorStore { if (this.connection) this.connection.closeSync() if (this.dbInstance) this.dbInstance.closeSync() } catch (error) { - console.error('[MemoryVectorStore] close error', error) + logger.warn(`[MemoryVectorStore] close error: ${String(error)}`) } } } diff --git a/src/main/presenter/memoryPresenter/infra/vectorStoreManager.ts b/src/main/presenter/memoryPresenter/infra/vectorStoreManager.ts index 1a71d0a47..9da642033 100644 --- a/src/main/presenter/memoryPresenter/infra/vectorStoreManager.ts +++ b/src/main/presenter/memoryPresenter/infra/vectorStoreManager.ts @@ -16,6 +16,23 @@ interface VectorStoreRuntimeState { vectorStoreLocks: Map> } +export type VectorDeleteResult = 'deleted' | 'skipped' | 'unusable' + +export interface VectorDeleteOptions { + embeddingModel?: string | null + embeddingDim?: number | null +} + +function embeddingFromFingerprint(fingerprint: string | null | undefined): MemoryModelRef | null { + if (!fingerprint) return null + const separator = fingerprint.indexOf(':') + if (separator <= 0 || separator === fingerprint.length - 1) return null + return { + providerId: fingerprint.slice(0, separator), + modelId: fingerprint.slice(separator + 1) + } +} + export class VectorStoreManager implements VectorStoreRetrievalPort { private readonly vectorStores = new Map>() private readonly vectorStoreIdentities = new Map() @@ -158,16 +175,71 @@ export class VectorStoreManager implements VectorStoreRetrievalPort { }) } - async deleteVectorsForMemoryIds(agentId: string, memoryIds: string[]): Promise { - if (!memoryIds.length) return - await this.withAgentLock(agentId, async () => { + async deleteVectorsForMemoryIdsOpening( + agentId: string, + memoryIds: string[], + options: VectorDeleteOptions = {} + ): Promise { + if (!memoryIds.length) return 'skipped' + let result: VectorDeleteResult = 'skipped' + await this.withAgentLock(agentId, async (locked) => { if (this.ctx.isDisposed) return - const store = await this.vectorStoreForAgent(agentId) - if (!store) return - await store.deleteByMemoryIds(memoryIds).catch((error) => { + let targetEmbedding = embeddingFromFingerprint(options.embeddingModel) + let targetDimensions = + typeof options.embeddingDim === 'number' && + Number.isFinite(options.embeddingDim) && + options.embeddingDim > 0 + ? Math.floor(options.embeddingDim) + : null + + if (!targetEmbedding) { + const embedding = this.ctx.deps.resolveAgentConfig(agentId)?.memoryEmbedding + if (!embedding?.providerId || !embedding?.modelId) { + logger.debug(`[Memory] vector delete skipped for ${agentId}: embedding is not configured`) + return + } + targetEmbedding = { providerId: embedding.providerId, modelId: embedding.modelId } + } + if (targetDimensions === null) { + const fingerprint = embeddingFingerprint( + targetEmbedding.providerId, + targetEmbedding.modelId + ) + targetDimensions = + this.getWarmVectorStoreDimension(agentId, targetEmbedding) ?? + this.ctx.deps.repository.getCurrentEmbeddingDimension(agentId, fingerprint) + if (targetDimensions === null) { + logger.debug(`[Memory] vector delete deferred for ${agentId}: dimension is unknown`) + return + } + } + + const targetIdentity = this.cacheKey(agentId, targetEmbedding, targetDimensions) + let store = + this.vectorStoreIdentities.get(agentId) === targetIdentity + ? await this.vectorStoreForAgent(agentId) + : null + if (!store) { + try { + store = await locked.open(targetEmbedding, targetDimensions) + } catch (error) { + logger.warn(`[Memory] vector delete open failed for ${agentId}: ${String(error)}`) + return + } + } + if (!store.isUsable()) { + this.clearReady(agentId) + result = 'unusable' + return + } + try { + await store.deleteByMemoryIds(memoryIds) + result = 'deleted' + } catch (error) { logger.warn(`[Memory] vector delete failed: ${String(error)}`) - }) + } }) + return result } async deletePrunableVectorsForMemoryIds( @@ -202,15 +274,6 @@ export class VectorStoreManager implements VectorStoreRetrievalPort { }) } - async deleteVectorsForMemoryIdsForEmbedding( - agentId: string, - embedding: MemoryModelRef, - dimensions: number, - memoryIds: string[] - ): Promise { - return this.deletePrunableVectorsForMemoryIds(agentId, embedding, dimensions, memoryIds) - } - async queryNeighborsByMemoryId( agentId: string, embedding: MemoryModelRef, diff --git a/src/main/presenter/memoryPresenter/runtimeConstants.ts b/src/main/presenter/memoryPresenter/runtimeConstants.ts index bda8d2a28..f1881bb9a 100644 --- a/src/main/presenter/memoryPresenter/runtimeConstants.ts +++ b/src/main/presenter/memoryPresenter/runtimeConstants.ts @@ -11,9 +11,15 @@ export const PERSONA_MEMORY_LIMIT = 20 export const WORKING_BLOB_TOKEN_LIMIT = 400 export const WORKING_PROVENANCE_SEED = 'session-working-blob' +export const WORKING_CANDIDATE_PAGE_LIMIT = 64 +export const WORKING_CANDIDATE_SCAN_LIMIT = 512 export const REINDEX_BATCH_SIZE = 50 export const REINDEX_MAX_BATCHES = 200 +export const ERROR_RETRY_COOLDOWN_MS = 10 * 60 * 1000 +export const ERROR_RETRY_BATCH_LIMIT = 50 +export const ORPHAN_RECONCILE_BATCH = 512 +export const ORPHAN_RECONCILE_RETRY_COOLDOWN_MS = 5 * 60 * 1000 export const DECISION_NEIGHBOR_TOP_S = 10 diff --git a/src/main/presenter/memoryPresenter/services/conflictService.ts b/src/main/presenter/memoryPresenter/services/conflictService.ts index 074811c29..f5c9b2d08 100644 --- a/src/main/presenter/memoryPresenter/services/conflictService.ts +++ b/src/main/presenter/memoryPresenter/services/conflictService.ts @@ -67,7 +67,9 @@ export class ConflictService { ) if (!pair) return false if (!this.ctx.canWriteAgentMemory(agentId)) return false - this.applyConflictResolution(agentId, pair, outcome, options) + this.ctx.deps.repository.runInTransaction(() => { + this.applyConflictResolution(agentId, pair, outcome, options) + }) this.ports.syncWorkingMemoryAfterMutation(agentId) this.ctx.writeAudit(agentId, { eventType: 'memory/challenge_resolved', diff --git a/src/main/presenter/memoryPresenter/services/maintenanceService.ts b/src/main/presenter/memoryPresenter/services/maintenanceService.ts index 19b658ab8..9e93f072c 100644 --- a/src/main/presenter/memoryPresenter/services/maintenanceService.ts +++ b/src/main/presenter/memoryPresenter/services/maintenanceService.ts @@ -2,7 +2,7 @@ import logger from '@shared/logger' import { isAgentMemoryCategory } from '@shared/types/agent-memory' import { ARCHIVE_AGE_MS, ARCHIVE_DECAY_THRESHOLD } from '../core/lifecycle' -import { decayScore, distanceToSimilarity } from '../core/scoring' +import { distanceToSimilarity } from '../core/scoring' import { ADD_DECISION, buildDecisionPrompt, @@ -29,6 +29,7 @@ import { isSafeAgentId, type AgentMemoryRow, type ConsolidationScanCursor, + FORGET_HALF_LIFE_MS, type MemoryPersonaDraftResult, type MemoryReflectionResult } from '../types' @@ -439,13 +440,21 @@ export class MaintenanceService { const secondaryCategory = isAgentMemoryCategory(secondary.category) ? secondary.category : null - const survivorId = this.rows.applyContentUpdate( + const update = this.rows.applyContentUpdate( agentId, primary, mergedContent, now, secondaryCategory ) + if (update.action === 'suppressed') { + this.ctx.deps.repository.setLastConsolidatedAt(source.id, now) + this.ctx.deps.repository.setLastConsolidatedAt(neighbor.id, now) + merged.add(source.id) + merged.add(neighbor.id) + continue + } + const survivorId = update.id this.rows.bumpConfidence(survivorId) this.ctx.deps.repository.setImportance(survivorId, secondary.importance) this.ctx.deps.repository.updateStatus(survivorId, 'pending_embedding') @@ -553,17 +562,11 @@ export class MaintenanceService { } private stampConsolidation(agentId: string, now: number): void { - for (const row of this.ctx.deps.repository.listByAgent(agentId)) { - if (row.kind === 'persona') continue - this.ctx.deps.repository.setLastConsolidatedAt(row.id, now) - } + this.ctx.deps.repository.stampConsolidationForAgent(agentId, now) } refreshDecayScores(agentId: string, now: number): void { - for (const row of this.ctx.deps.repository.listByAgent(agentId)) { - if (row.kind === 'persona') continue - this.ctx.deps.repository.updateDecayScore(row.id, decayScore(row, now), null) - } + this.ctx.deps.repository.refreshDecayScoresForAgent(agentId, now, FORGET_HALF_LIFE_MS) } archiveStale(agentId: string, now: number = Date.now()): number { diff --git a/src/main/presenter/memoryPresenter/services/managementService.ts b/src/main/presenter/memoryPresenter/services/managementService.ts index 698785c4c..d861f6746 100644 --- a/src/main/presenter/memoryPresenter/services/managementService.ts +++ b/src/main/presenter/memoryPresenter/services/managementService.ts @@ -40,13 +40,20 @@ function isInternalMemoryKind(row: AgentMemoryRow): boolean { return row.kind === 'persona' || row.kind === 'working' } +type DeleteVectorResult = 'deleted' | 'skipped' | 'unusable' + export class ManagementService { constructor( private readonly ctx: MemoryRuntimeContext, private readonly ports: { - deleteVectorsForMemoryIds: (agentId: string, memoryIds: string[]) => Promise + deleteVectorsForDeletedMemory: ( + agentId: string, + memoryIds: string[], + embedding: { embeddingModel: string | null; embeddingDim: number | null } + ) => Promise resetAgentStore: (agentId: string) => Promise isReindexing: (agentId: string) => boolean + reindexEmbeddings: (agentId: string, force?: boolean) => Promise syncWorkingMemoryAfterMutation: (agentId: string) => void triggerEmbedding: (agentId: string) => Promise clearConsolidationCooldown: (agentId: string) => void @@ -60,8 +67,17 @@ export class ManagementService { const row = this.ctx.deps.repository.getById(memoryId) if (!row || row.agent_id !== agentId || row.status !== 'archived') return false if (isInternalMemoryKind(row)) return false - this.ctx.deps.repository.updateStatus(memoryId, 'pending_embedding') - this.ports.syncWorkingMemoryAfterMutation(agentId) + this.ctx.deps.repository.runInTransaction(() => { + this.ctx.deps.repository.updateStatus(memoryId, 'pending_embedding') + this.ports.syncWorkingMemoryAfterMutation(agentId) + this.ctx.writeAudit(agentId, { + eventType: 'memory/restore', + actorType: 'user', + status: 'completed', + inputRefs: { memoryId }, + outputRefs: { action: 'restored', memoryId } + }) + }) void this.ports.triggerEmbedding(agentId).catch((error) => { logger.warn(`[Memory] background embedding failed: ${String(error)}`) }) @@ -76,11 +92,24 @@ export class ManagementService { const row = this.ctx.deps.repository.getById(memoryId) if (!row || row.agent_id !== agentId) return false if (isInternalMemoryKind(row)) return false - if (row.status === 'archived') return true - this.ctx.deps.repository.archive(row.id, Date.now()) - if (this.ctx.isDisposed) return true - this.ports.syncWorkingMemoryAfterMutation(agentId) - this.ctx.emitChanged(agentId, 'extract') + const alreadyArchived = row.status === 'archived' + this.ctx.deps.repository.runInTransaction(() => { + if (!alreadyArchived) { + this.ctx.deps.repository.archive(row.id, Date.now()) + this.ports.syncWorkingMemoryAfterMutation(agentId) + } + this.ctx.writeAudit(agentId, { + eventType: 'memory/forget', + actorType: 'runtime', + status: 'completed', + reason: alreadyArchived ? 'already_archived' : null, + inputRefs: { memoryId }, + outputRefs: { action: alreadyArchived ? 'already_archived' : 'archived', memoryId } + }) + }) + if (!alreadyArchived) { + this.ctx.emitChanged(agentId, 'extract') + } return true } @@ -92,22 +121,23 @@ export class ManagementService { if (!row || row.agent_id !== agentId) return false if (isInternalMemoryKind(row)) return false const alreadyArchived = row.status === 'archived' - if (!alreadyArchived) { - this.ctx.deps.repository.archive(row.id, Date.now()) - if (!this.ctx.isDisposed) { + this.ctx.deps.repository.runInTransaction(() => { + if (!alreadyArchived) { + this.ctx.deps.repository.archive(row.id, Date.now()) this.ports.syncWorkingMemoryAfterMutation(agentId) - this.ctx.emitChanged(agentId, 'extract') } - } - if (this.ctx.isDisposed) return true - this.ctx.writeAudit(agentId, { - eventType: 'memory/archive', - actorType: 'user', - status: 'completed', - reason: alreadyArchived ? 'already_archived' : null, - inputRefs: { memoryId }, - outputRefs: { action: alreadyArchived ? 'already_archived' : 'archived', memoryId } + this.ctx.writeAudit(agentId, { + eventType: 'memory/archive', + actorType: 'user', + status: 'completed', + reason: alreadyArchived ? 'already_archived' : null, + inputRefs: { memoryId }, + outputRefs: { action: alreadyArchived ? 'already_archived' : 'archived', memoryId } + }) }) + if (!alreadyArchived) { + this.ctx.emitChanged(agentId, 'extract') + } return true } @@ -274,7 +304,15 @@ export class ManagementService { if (row.kind !== 'working') { this.ports.syncWorkingMemoryAfterMutation(agentId) } - await this.ports.deleteVectorsForMemoryIds(agentId, [memoryId]) + const deleteResult = await this.ports.deleteVectorsForDeletedMemory(agentId, [memoryId], { + embeddingModel: row.embedding_model, + embeddingDim: row.embedding_dim + }) + if (deleteResult === 'unusable' && !this.ports.isReindexing(agentId)) { + void this.ports.reindexEmbeddings(agentId, true).catch((error) => { + logger.warn(`[Memory] store rebuild after delete failed for ${agentId}: ${String(error)}`) + }) + } if (this.ctx.isDisposed) return true this.ctx.emitChanged(agentId, 'delete', { memoryId }) return true @@ -305,10 +343,10 @@ export class ManagementService { if (!this.ctx.isManagedAgent(agentId)) { return { total: 0, pendingEmbedding: 0, hasPersona: false } } - const all = this.ctx.deps.repository.listByAgent(agentId, { includeSuperseded: true }) + const counts = this.ctx.deps.repository.countStatusView(agentId) return { - total: all.length, - pendingEmbedding: all.filter((row) => row.status === 'pending_embedding').length, + total: counts.total, + pendingEmbedding: counts.pendingEmbedding, hasPersona: this.ctx.deps.repository.getActivePersona(agentId) !== undefined, reindexing: this.ports.isReindexing(agentId) } diff --git a/src/main/presenter/memoryPresenter/services/retrievalService.ts b/src/main/presenter/memoryPresenter/services/retrievalService.ts index a890c0552..6e44cadc7 100644 --- a/src/main/presenter/memoryPresenter/services/retrievalService.ts +++ b/src/main/presenter/memoryPresenter/services/retrievalService.ts @@ -110,6 +110,18 @@ export class RetrievalService { }) } + async retrieveForDecision( + agentId: string, + query: string, + now: number + ): Promise { + return this.retrieve(agentId, query, now, false, { + keywordQuery: this.buildAgentFacingRecallKeywordQuery(agentId, query), + keywordMatchMode: 'any', + enableInlinePrune: false + }) + } + private buildAgentFacingRecallKeywordQuery(agentId: string, query: string): string { const candidates = extractRecallKeywordCandidates(query) if (!candidates.length) return '' @@ -352,7 +364,7 @@ export class RetrievalService { keywordQuery: this.buildAgentFacingRecallKeywordQuery(agentId, query), keywordMatchMode: 'any' }) - : await this.recall(agentId, query) + : [] if (!persona && !working && recalled.length === 0) return null const tokenBudget = resolveInjectionTokenBudget(config?.memoryInjectionTokenBudget) const payload: MemoryInjectionPayload = { diff --git a/src/main/presenter/memoryPresenter/services/rowMutations.ts b/src/main/presenter/memoryPresenter/services/rowMutations.ts index 24d62b427..f41eb14c4 100644 --- a/src/main/presenter/memoryPresenter/services/rowMutations.ts +++ b/src/main/presenter/memoryPresenter/services/rowMutations.ts @@ -16,6 +16,16 @@ function canCarryCategory(kind: AgentMemoryRow['kind']): boolean { return kind === 'episodic' || kind === 'semantic' } +export type ProvenanceHitResult = + | { action: 'absorbed' } + | { action: 'continue' } + | { action: 'noop'; reason: string } + +export type ContentUpdateResult = + | { action: 'updated'; id: string } + | { action: 'folded'; id: string } + | { action: 'suppressed'; id: string; reason: string } + export class MemoryRowMutations { constructor(private readonly ctx: MemoryRuntimeContext) {} @@ -107,28 +117,45 @@ export class MemoryRowMutations { content: string, now: number, category?: AgentMemoryCategory | null - ): string { + ): ContentUpdateResult { const newKey = buildMemoryProvenanceKey(agentId, row.kind, content) const nextCategory = canCarryCategory(row.kind) ? (row.category ?? category ?? null) : undefined if (newKey !== row.provenance_key) { const owner = this.ctx.deps.repository.getByProvenanceKey(agentId, newKey) if (owner && owner.id !== row.id) { - this.absorbProvenanceHit(agentId, owner) - if (canCarryCategory(owner.kind) && owner.category === null && nextCategory != null) { - this.ctx.deps.repository.updateContent( - owner.id, - owner.content, - owner.provenance_key, - now, - nextCategory - ) + const hit = this.handleProvenanceHit(agentId, owner, { + allowDecisionForSuperseded: true + }) + if (hit.action === 'noop' && (owner.status === 'archived' || owner.superseded_by)) { + return { + action: 'suppressed', + id: owner.id, + reason: hit.reason + } } - this.ctx.deps.repository.markSuperseded(row.id, owner.id) - return owner.id + this.ctx.deps.repository.runInTransaction(() => { + if (hit.action === 'absorbed') { + this.ctx.deps.repository.updateStatus(owner.id, 'pending_embedding') + } + if (hit.action === 'continue') { + this.reviveSupersededAfterDecision(agentId, owner) + } + if (canCarryCategory(owner.kind) && owner.category === null && nextCategory != null) { + this.ctx.deps.repository.updateContent( + owner.id, + owner.content, + owner.provenance_key, + now, + nextCategory + ) + } + this.ctx.deps.repository.markSuperseded(row.id, owner.id) + }) + return { action: 'folded', id: owner.id } } } this.ctx.deps.repository.updateContent(row.id, content, newKey, now, nextCategory) - return row.id + return { action: 'updated', id: row.id } } supersedeHead(agentId: string, row: AgentMemoryRow): AgentMemoryRow { @@ -143,19 +170,50 @@ export class MemoryRowMutations { return current } - absorbProvenanceHit(agentId: string, existing: AgentMemoryRow): boolean { + handleProvenanceHit( + agentId: string, + existing: AgentMemoryRow, + options: { allowDecisionForSuperseded?: boolean } = {} + ): ProvenanceHitResult { const archived = existing.status === 'archived' const superseded = existing.superseded_by !== null - if (!archived && !superseded) return false + if (!archived && !superseded) return { action: 'noop', reason: 'duplicate' } - if (superseded) { - const head = this.supersedeHead(agentId, existing) - this.ctx.deps.repository.markSuperseded(existing.id, null) - if (head.id !== existing.id && head.superseded_by === null && head.status !== 'archived') { - this.ctx.deps.repository.markSuperseded(head.id, existing.id) - } + if (archived && this.ctx.deps.auditRepository?.hasForgetEvent(agentId, existing.id)) { + return { action: 'noop', reason: 'suppressed-user-forget' } + } + + if (archived && superseded) { + return { action: 'noop', reason: 'suppressed-conflict-loser' } + } + + if (superseded && !archived && options.allowDecisionForSuperseded) { + return { action: 'continue' } + } + + if (superseded && !archived) { + return { action: 'noop', reason: 'duplicate' } } + + return { action: 'absorbed' } + } + + reviveSupersededAfterDecision( + agentId: string, + existing: AgentMemoryRow + ): { retiredHeadId: string | null } { + if (existing.status === 'archived' || existing.superseded_by === null) { + return { retiredHeadId: null } + } + + const head = this.supersedeHead(agentId, existing) + this.ctx.deps.repository.markSuperseded(existing.id, null) this.ctx.deps.repository.updateStatus(existing.id, 'pending_embedding') - return true + + if (head.id !== existing.id && head.status !== 'archived' && head.superseded_by === null) { + this.ctx.deps.repository.markSuperseded(head.id, existing.id) + return { retiredHeadId: head.id } + } + return { retiredHeadId: null } } } diff --git a/src/main/presenter/memoryPresenter/services/workingMemoryService.ts b/src/main/presenter/memoryPresenter/services/workingMemoryService.ts index c7fb17d44..8aace0269 100644 --- a/src/main/presenter/memoryPresenter/services/workingMemoryService.ts +++ b/src/main/presenter/memoryPresenter/services/workingMemoryService.ts @@ -3,9 +3,15 @@ import { nanoid } from 'nanoid' import { buildMemoryProvenanceKey } from '../core/scoring' import { estimateTokens } from '../core/injectionPort' -import { WORKING_BLOB_TOKEN_LIMIT, WORKING_PROVENANCE_SEED } from '../runtimeConstants' +import { + WORKING_BLOB_TOKEN_LIMIT, + WORKING_CANDIDATE_PAGE_LIMIT, + WORKING_CANDIDATE_SCAN_LIMIT, + WORKING_PROVENANCE_SEED +} from '../runtimeConstants' import { isUniqueConstraintError, type MemoryRuntimeContext } from '../context' import type { WorkingMemoryReadPort } from '../ports' +import type { AgentMemoryWorkingCandidateCursor } from '../types' export class WorkingMemoryService implements WorkingMemoryReadPort { private readonly workingRefreshInFlight = new Set() @@ -86,25 +92,36 @@ export class WorkingMemoryService implements WorkingMemoryReadPort { } buildWorkingBlob(agentId: string): string { - const units = this.ctx.deps.repository - .listByAgent(agentId, { kinds: ['semantic', 'reflection', 'episodic'] }) - .slice() - .sort( - (a, b) => - b.importance - a.importance || - b.access_count - a.access_count || - b.created_at - a.created_at - ) const lines: string[] = [] let tokens = 0 - for (const unit of units) { - const content = unit.content.trim() - if (!content) continue - const line = `- ${content}` - const cost = estimateTokens(line) - if (tokens + cost > WORKING_BLOB_TOKEN_LIMIT) continue - lines.push(line) - tokens += cost + let scanned = 0 + let cursor: AgentMemoryWorkingCandidateCursor | undefined + while (scanned < WORKING_CANDIDATE_SCAN_LIMIT && tokens < WORKING_BLOB_TOKEN_LIMIT) { + const pageLimit = Math.min( + WORKING_CANDIDATE_PAGE_LIMIT, + WORKING_CANDIDATE_SCAN_LIMIT - scanned + ) + const units = this.ctx.deps.repository.listWorkingCandidates(agentId, pageLimit, cursor) + if (!units.length) break + scanned += units.length + for (const unit of units) { + const content = unit.content.trim() + if (!content) continue + const line = `- ${content}` + const cost = estimateTokens(line) + if (tokens + cost > WORKING_BLOB_TOKEN_LIMIT) continue + lines.push(line) + tokens += cost + if (tokens >= WORKING_BLOB_TOKEN_LIMIT) break + } + const last = units[units.length - 1] + cursor = { + importance: last.importance, + accessCount: last.access_count, + createdAt: last.created_at, + id: last.id + } + if (units.length < pageLimit) break } return lines.join('\n').trim() } diff --git a/src/main/presenter/memoryPresenter/services/writeCoordinator.ts b/src/main/presenter/memoryPresenter/services/writeCoordinator.ts index 78e59484f..0cb0e1614 100644 --- a/src/main/presenter/memoryPresenter/services/writeCoordinator.ts +++ b/src/main/presenter/memoryPresenter/services/writeCoordinator.ts @@ -17,6 +17,7 @@ import { buildMemoryProvenanceKey } from '../core/scoring' import { DECISION_NEIGHBOR_TOP_S, MEMORY_CREATED_IDS_EVENT_LIMIT } from '../runtimeConstants' import type { MemoryRowMutations } from './rowMutations' import type { + AgentMemoryRow, MemoryCandidate, MemoryExtractionInput, MemoryExtractionResult, @@ -104,16 +105,49 @@ function userAddAuditFromOutcome(outcome: MemoryWriteOutcome): { } } +function isLiveDecisionTarget( + agentId: string, + row: AgentMemoryRow | undefined +): row is AgentMemoryRow { + return ( + !!row && + row.agent_id === agentId && + row.superseded_by === null && + row.status !== 'archived' && + row.status !== 'conflicted' + ) +} + +function recallItemFromRow(row: AgentMemoryRow): MemoryRecallItem { + return { + id: row.id, + kind: row.kind, + content: row.content, + score: 1, + importance: row.importance, + sources: { fts: true }, + sourceSession: row.source_session, + sourceEntryIds: null, + breakdown: { + similarity: 0, + recency: row.last_accessed ?? row.created_at, + importance: row.importance, + confidence: row.confidence ?? 0, + rrf: 1, + final: 1 + } + } +} + export class WriteCoordinator { constructor( private readonly ctx: MemoryRuntimeContext, private readonly rows: MemoryRowMutations, private readonly ports: { - retrieve: ( + retrieveForDecision: ( agentId: string, query: string, - now: number, - recordAccessHits: boolean + now: number ) => Promise syncWorkingMemoryAfterMutation: (agentId: string) => void triggerEmbedding: (agentId: string) => Promise @@ -131,7 +165,17 @@ export class WriteCoordinator { const provenanceKey = buildMemoryProvenanceKey(options.agentId, normalized.kind, content) const duplicate = this.ctx.deps.repository.getByProvenanceKey(options.agentId, provenanceKey) if (duplicate) { - if (this.rows.absorbProvenanceHit(options.agentId, duplicate)) created.push(duplicate.id) + const hit = this.rows.handleProvenanceHit(options.agentId, duplicate, { + allowDecisionForSuperseded: true + }) + if (hit.action === 'absorbed') { + this.absorbArchivedProvenanceOwner(duplicate) + created.push(duplicate.id) + } + if (hit.action === 'continue') { + this.reviveProvenanceOwner(options.agentId, duplicate, Date.now(), normalized.category) + created.push(duplicate.id) + } continue } const id = this.rows.insertMemory( @@ -233,22 +277,38 @@ export class WriteCoordinator { const provenanceKey = buildMemoryProvenanceKey(agentId, normalized.kind, content) const duplicate = this.ctx.deps.repository.getByProvenanceKey(agentId, provenanceKey) + let decisionHead: AgentMemoryRow | null = null + let supersededDuplicate: AgentMemoryRow | null = null if (duplicate) { - const touched = this.rows.absorbProvenanceHit(agentId, duplicate) - return touched - ? { action: 'updated', id: duplicate.id } - : { action: 'noop', reason: 'duplicate', id: duplicate.id } + const hit = this.rows.handleProvenanceHit(agentId, duplicate, { + allowDecisionForSuperseded: true + }) + if (hit.action === 'absorbed') { + this.absorbArchivedProvenanceOwner(duplicate) + return { action: 'updated', id: duplicate.id } + } + if (hit.action === 'noop') return { action: 'noop', reason: hit.reason, id: duplicate.id } + supersededDuplicate = duplicate + const head = this.rows.supersedeHead(agentId, duplicate) + if (isLiveDecisionTarget(agentId, head)) decisionHead = head } let neighbors: MemoryRecallItem[] = [] try { - const hits = await this.ports.retrieve(agentId, content, now, false) + const hits = await this.ports.retrieveForDecision(agentId, content, now) neighbors = hits.slice(0, DECISION_NEIGHBOR_TOP_S) + if (decisionHead && !neighbors.some((neighbor) => neighbor.id === decisionHead.id)) { + neighbors.unshift(recallItemFromRow(decisionHead)) + neighbors = neighbors.slice(0, DECISION_NEIGHBOR_TOP_S) + } } catch (error) { logger.warn(`[Memory] decision neighbor recall failed, adding: ${String(error)}`) } if (!this.ctx.canWriteAgentMemory(agentId)) return { action: 'noop', reason: 'disposed' } if (!neighbors.length) { + if (supersededDuplicate) { + return this.reviveProvenanceOwner(agentId, supersededDuplicate, now, normalized.category) + } const id = this.rows.insertMemory(agentId, normalized, content, provenanceKey, options) return id ? { action: 'created', id } : { action: 'noop', reason: 'insert-skipped' } } @@ -276,15 +336,19 @@ export class WriteCoordinator { case 'UPDATE': if (target) { const targetRow = this.ctx.deps.repository.getById(target.id) - if (targetRow) { + if (isLiveDecisionTarget(agentId, targetRow)) { const merged = decision.mergedContent ?? content - const survivorId = this.rows.applyContentUpdate( + const update = this.rows.applyContentUpdate( agentId, targetRow, merged, now, normalized.category ) + if (update.action === 'suppressed') { + return { action: 'noop', reason: update.reason, id: update.id } + } + const survivorId = update.id this.rows.bumpConfidence(survivorId) this.ctx.deps.repository.updateStatus(survivorId, 'pending_embedding') return { action: 'updated', id: survivorId } @@ -293,32 +357,33 @@ export class WriteCoordinator { break case 'SUPERSEDE': if (target) { + const targetRow = this.ctx.deps.repository.getById(target.id) + if (!isLiveDecisionTarget(agentId, targetRow)) break const merged = decision.mergedContent ?? content const mergedKey = buildMemoryProvenanceKey(agentId, normalized.kind, merged) - const newId = this.rows.insertMemory(agentId, normalized, merged, mergedKey, options) + let newId: string | null = null + this.ctx.deps.repository.runInTransaction(() => { + newId = this.rows.insertMemory(agentId, normalized, merged, mergedKey, options) + if (newId) this.ctx.deps.repository.markSuperseded(target.id, newId) + }) if (newId) { - this.ctx.deps.repository.markSuperseded(target.id, newId) return { action: 'superseded', id: newId, supersededId: target.id, created: true } } const existing = this.ctx.deps.repository.getByProvenanceKey(agentId, mergedKey) if (existing && existing.id !== target.id) { - this.rows.absorbProvenanceHit(agentId, existing) - if (existing.category === null && normalized.category !== null) { - this.ctx.deps.repository.updateContent( - existing.id, - existing.content, - existing.provenance_key, - now, - normalized.category - ) - } - this.ctx.deps.repository.markSuperseded(target.id, existing.id) - return { - action: 'superseded', - id: existing.id, - supersededId: target.id, - created: false + const hit = this.rows.handleProvenanceHit(agentId, existing, { + allowDecisionForSuperseded: true + }) + if (hit.action === 'noop' && hit.reason !== 'duplicate') { + return { action: 'noop', reason: hit.reason, id: existing.id } } + return this.reviveProvenanceOwner( + agentId, + existing, + now, + normalized.category, + target.id + ) } return { action: 'noop', reason: 'supersede-collided', id: target.id } } @@ -348,11 +413,24 @@ export class WriteCoordinator { this.ctx.deps.repository.updateStatus(challengerId, 'pending_embedding') return { action: 'created', id: challengerId } } + if (supersededDuplicate) { + const currentTarget = this.ctx.deps.repository.getById(target.id) + return this.reviveProvenanceOwner( + agentId, + supersededDuplicate, + now, + normalized.category, + isLiveDecisionTarget(agentId, currentTarget) ? currentTarget.id : undefined + ) + } return { action: 'noop', reason: 'challenge-insert-skipped', id: target.id } } break } const id = this.rows.insertMemory(agentId, normalized, content, provenanceKey, options) + if (!id && supersededDuplicate) { + return this.reviveProvenanceOwner(agentId, supersededDuplicate, now, normalized.category) + } return id ? { action: 'created', id } : { action: 'noop', reason: 'insert-skipped' } } @@ -392,15 +470,60 @@ export class WriteCoordinator { const provenanceKey = buildMemoryProvenanceKey(agentId, normalized.kind, content) const duplicate = this.ctx.deps.repository.getByProvenanceKey(agentId, provenanceKey) if (duplicate) { - const touched = this.rows.absorbProvenanceHit(agentId, duplicate) - return touched - ? { action: 'updated', id: duplicate.id } - : { action: 'noop', reason: 'duplicate', id: duplicate.id } + const hit = this.rows.handleProvenanceHit(agentId, duplicate, { + allowDecisionForSuperseded: true + }) + if (hit.action === 'absorbed') { + this.absorbArchivedProvenanceOwner(duplicate) + return { action: 'updated', id: duplicate.id } + } + if (hit.action === 'continue') { + return this.reviveProvenanceOwner(agentId, duplicate, Date.now(), normalized.category) + } + return { action: 'noop', reason: hit.reason, id: duplicate.id } } const id = this.rows.insertMemory(agentId, normalized, content, provenanceKey, options) return id ? { action: 'created', id } : { action: 'noop', reason: 'insert-skipped' } } + private absorbArchivedProvenanceOwner(existing: AgentMemoryRow): void { + this.ctx.deps.repository.runInTransaction(() => { + this.ctx.deps.repository.updateStatus(existing.id, 'pending_embedding') + }) + } + + private reviveProvenanceOwner( + agentId: string, + existing: AgentMemoryRow, + now: number, + category: AgentMemoryRow['category'], + foldTargetId?: string + ): MemoryWriteOutcome { + this.ctx.deps.repository.runInTransaction(() => { + if (existing.status === 'archived' && existing.superseded_by === null) { + this.ctx.deps.repository.updateStatus(existing.id, 'pending_embedding') + } + this.rows.reviveSupersededAfterDecision(agentId, existing) + if (existing.category === null && category !== null) { + this.ctx.deps.repository.updateContent( + existing.id, + existing.content, + existing.provenance_key, + now, + category + ) + } + if (foldTargetId && foldTargetId !== existing.id) { + this.ctx.deps.repository.markSuperseded(foldTargetId, existing.id) + } + }) + + if (foldTargetId && foldTargetId !== existing.id) { + return { action: 'superseded', id: existing.id, supersededId: foldTargetId, created: false } + } + return { action: 'updated', id: existing.id } + } + async addUserMemory( agentId: string, input: { diff --git a/src/main/presenter/memoryPresenter/types.ts b/src/main/presenter/memoryPresenter/types.ts index 5aa34a680..21af17702 100644 --- a/src/main/presenter/memoryPresenter/types.ts +++ b/src/main/presenter/memoryPresenter/types.ts @@ -1,6 +1,7 @@ import type { AgentMemoryKind, AgentMemoryLifecycleRow, + AgentMemoryWorkingCandidateCursor, AgentMemoryRow, AgentMemoryStatus, AgentMemoryConflictState, @@ -26,6 +27,7 @@ import type { LLM_EMBEDDING_ATTRS } from '@shared/presenter' export type { AgentMemoryKind, AgentMemoryLifecycleRow, + AgentMemoryWorkingCandidateCursor, AgentMemoryRow, AgentMemoryStatus, AgentMemoryConflictState, @@ -78,11 +80,24 @@ export interface MemoryRepositoryPort { // Bulk-resets the embedding state of an agent's non-superseded rows in the given statuses back // to pending_embedding (one SQL UPDATE), returning how many rows changed. Used by reindex and // backfill so the requeue never loops per row on the caller's stack. - requeueForEmbedding(agentId: string, statuses: AgentMemoryStatus[]): number + requeueForEmbedding( + agentId: string, + statuses: AgentMemoryStatus[], + limit?: number, + afterId?: string | null + ): number + listEmbeddingStatusIds( + agentId: string, + statuses: AgentMemoryStatus[], + limit: number, + afterId?: string | null + ): string[] markSuperseded(id: string, supersededBy: string | null): void recordAccess(id: string, accessedAt?: number): void recordAccessBatch(ids: string[], accessedAt?: number): void updateDecayScore(id: string, decayScore: number | null, consolidatedAt?: number | null): void + refreshDecayScoresForAgent(agentId: string, now: number, halfLifeMs: number): void + stampConsolidationForAgent(agentId: string, at: number): void updateContent( id: string, content: string, @@ -112,7 +127,14 @@ export interface MemoryRepositoryPort { delete(id: string): void clearByAgent(agentId: string): number countByAgent(agentId: string): number + countStatusView(agentId: string): { total: number; pendingEmbedding: number } hasActiveMemory(agentId: string): boolean + runInTransaction(fn: () => T): T + listWorkingCandidates( + agentId: string, + limit: number, + after?: AgentMemoryWorkingCandidateCursor + ): AgentMemoryRow[] listAgentIdsWithMemories(): string[] listConsolidationScanRows( agentId: string, @@ -152,6 +174,7 @@ export interface MemoryAuditRepositoryPort { insert(input: AgentMemoryAuditInsertInput): AgentMemoryAuditRow listByAgent(agentId: string, options?: number | MemoryAuditListOptions): AgentMemoryAuditRow[] getLatestCompletedEventAt(agentId: string, eventType: string): number | null + hasForgetEvent(agentId: string, memoryId: string): boolean getHealthAuditStats( agentId: string, scanLimit: number, @@ -208,6 +231,7 @@ export interface IMemoryVectorStore { query(embedding: number[], options: MemoryVectorQueryOptions): Promise queryByMemoryId(memoryId: string, options: MemoryVectorQueryOptions): Promise deleteByMemoryIds(memoryIds: string[]): Promise + listMemoryIds(afterId: string | null, limit: number): Promise close(): Promise isUsable(): boolean } diff --git a/src/main/presenter/sqlitePresenter/tables/agentMemory.ts b/src/main/presenter/sqlitePresenter/tables/agentMemory.ts index cd7905b48..eba614d7e 100644 --- a/src/main/presenter/sqlitePresenter/tables/agentMemory.ts +++ b/src/main/presenter/sqlitePresenter/tables/agentMemory.ts @@ -50,6 +50,13 @@ export interface AgentMemoryRow { persona_state: string | null } +export interface AgentMemoryWorkingCandidateCursor { + importance: number + accessCount: number + createdAt: number + id: string +} + export type AgentMemoryLifecycleRow = Pick< AgentMemoryRow, | 'id' @@ -114,6 +121,7 @@ const AGENT_MEMORY_FTS_META_KEY = 'agent_memory_fts' const AGENT_MEMORY_FTS_META_VERSION = 1 type FtsCapability = { available: boolean; tokenizer: 'trigram' | 'unicode61' } +type MathFunctionCapability = { available: boolean } type SearchMatchMode = 'all' | 'any' type RecallKeywordTermStat = { term: string; hitCount: number; totalRows: number } @@ -153,6 +161,13 @@ function readAggregateNullableNumber(value: unknown): number | null { return typeof value === 'number' && Number.isFinite(value) ? value : null } +function calculateDecayScoreForRow(row: AgentMemoryRow, now: number, halfLifeMs: number): number { + const anchor = row.last_accessed ?? row.created_at + const age = Math.max(0, now - anchor) + const importance = Math.min(1, Math.max(0, row.importance)) + return Math.pow(0.5, age / (halfLifeMs * (1 + importance))) +} + function sqlLiteral(value: string): string { return `'${value.replace(/'/g, "''")}'` } @@ -189,6 +204,7 @@ function readAggregateRecord( export class AgentMemoryTable extends BaseTable { private ftsCapability: FtsCapability | undefined + private mathFunctionCapability: MathFunctionCapability | undefined private ftsReady = false constructor(db: Database.Database) { @@ -294,6 +310,19 @@ export class AgentMemoryTable extends BaseTable { return this.ftsCapability } + private detectMathFunctionCapability(): MathFunctionCapability { + if (this.mathFunctionCapability) return this.mathFunctionCapability + try { + const row = this.db.prepare('SELECT pow(2.0, 2.0) AS value').get() as + | { value: number } + | undefined + this.mathFunctionCapability = { available: row?.value === 4 } + } catch { + this.mathFunctionCapability = { available: false } + } + return this.mathFunctionCapability + } + private ftsTableExists(): boolean { const row = this.db .prepare(`SELECT name FROM sqlite_master WHERE type='table' AND name='agent_memory_fts'`) @@ -819,9 +848,43 @@ export class AgentMemoryTable extends BaseTable { // in pending_embedding forever, since listPendingEmbedding never returns those kinds. Status // changes do not touch content, so the FTS triggers (UPDATE OF content) never fire here. // Returns the number of rows changed. - requeueForEmbedding(agentId: string, statuses: AgentMemoryStatus[]): number { + requeueForEmbedding( + agentId: string, + statuses: AgentMemoryStatus[], + limit?: number, + afterId?: string | null + ): number { if (!statuses.length) return 0 const placeholders = statuses.map(() => '?').join(', ') + if (limit !== undefined) { + const cappedLimit = Math.max(0, Math.floor(limit)) + if (cappedLimit === 0) return 0 + const afterSql = afterId ? 'AND id > ?' : '' + const params: unknown[] = [agentId, ...statuses] + if (afterId) params.push(afterId) + params.push(cappedLimit) + const result = this.db + .prepare( + `UPDATE agent_memory + SET status = 'pending_embedding', + embedding_id = NULL, + embedding_dim = NULL, + embedding_model = NULL + WHERE id IN ( + SELECT id + FROM agent_memory + WHERE agent_id = ? + AND superseded_by IS NULL + AND kind NOT IN ('persona', 'working') + AND status IN (${placeholders}) + ${afterSql} + ORDER BY id ASC + LIMIT ? + )` + ) + .run(...params) + return result.changes + } const result = this.db .prepare( `UPDATE agent_memory @@ -838,6 +901,36 @@ export class AgentMemoryTable extends BaseTable { return result.changes } + listEmbeddingStatusIds( + agentId: string, + statuses: AgentMemoryStatus[], + limit: number, + afterId?: string | null + ): string[] { + if (!statuses.length) return [] + const cappedLimit = Math.max(0, Math.floor(limit)) + if (cappedLimit === 0) return [] + const placeholders = statuses.map(() => '?').join(', ') + const afterSql = afterId ? 'AND id > ?' : '' + const params: unknown[] = [agentId, ...statuses] + if (afterId) params.push(afterId) + params.push(cappedLimit) + const rows = this.db + .prepare( + `SELECT id + FROM agent_memory + WHERE agent_id = ? + AND superseded_by IS NULL + AND kind NOT IN ('persona', 'working') + AND status IN (${placeholders}) + ${afterSql} + ORDER BY id ASC + LIMIT ?` + ) + .all(...params) as Array<{ id: string }> + return rows.map((row) => row.id) + } + markSuperseded(id: string, supersededBy: string | null): void { this.db.prepare('UPDATE agent_memory SET superseded_by = ? WHERE id = ?').run(supersededBy, id) } @@ -881,6 +974,48 @@ export class AgentMemoryTable extends BaseTable { .run(decayScore, consolidatedAt, id) } + refreshDecayScoresForAgent(agentId: string, now: number, halfLifeMs: number): void { + if (this.detectMathFunctionCapability().available) { + this.db + .prepare( + `UPDATE agent_memory + SET decay_score = pow( + 0.5, + max(0, ? - COALESCE(last_accessed, created_at)) / + (? * (1 + min(1.0, max(0.0, importance)))) + ) + WHERE agent_id = ? + AND kind != 'persona' + AND kind != 'working' + AND superseded_by IS NULL + AND status NOT IN ('archived', 'conflicted')` + ) + .run(now, halfLifeMs, agentId) + return + } + + const rows = this.listByAgent(agentId).filter((row) => row.kind !== 'persona') + this.runInTransaction(() => { + for (const row of rows) { + this.updateDecayScore(row.id, calculateDecayScoreForRow(row, now, halfLifeMs), null) + } + }) + } + + stampConsolidationForAgent(agentId: string, at: number): void { + this.db + .prepare( + `UPDATE agent_memory + SET last_consolidated_at = ? + WHERE agent_id = ? + AND kind != 'persona' + AND kind != 'working' + AND superseded_by IS NULL + AND status NOT IN ('archived', 'conflicted')` + ) + .run(at, agentId) + } + // Refreshes a row's content in place (UPDATE/merge decision), keeping its provenance_key in sync // with the new content so the idempotent dedup short-circuit keeps matching. last_accessed is // re-anchored too so a rewritten row's forgetting clock resets — a just-merged current-truth row @@ -1191,6 +1326,75 @@ export class AgentMemoryTable extends BaseTable { return row?.count ?? 0 } + countStatusView(agentId: string): { total: number; pendingEmbedding: number } { + const row = this.db + .prepare( + `SELECT COUNT(*) AS total, + SUM(CASE WHEN status = 'pending_embedding' THEN 1 ELSE 0 END) AS pendingEmbedding + FROM agent_memory + WHERE agent_id = ? + AND status != 'archived' + AND status != 'conflicted' + AND kind != 'working'` + ) + .get(agentId) as { total: number; pendingEmbedding: number | null } | undefined + return { + total: row?.total ?? 0, + pendingEmbedding: row?.pendingEmbedding ?? 0 + } + } + + runInTransaction(fn: () => T): T { + return this.db.transaction(fn)() + } + + listWorkingCandidates( + agentId: string, + limit: number, + after?: AgentMemoryWorkingCandidateCursor + ): AgentMemoryRow[] { + const cappedLimit = Math.max(0, Math.floor(limit)) + if (cappedLimit === 0) return [] + const cursorSql = after + ? `AND ( + importance < ? + OR (importance = ? AND access_count < ?) + OR (importance = ? AND access_count = ? AND created_at < ?) + OR (importance = ? AND access_count = ? AND created_at = ? AND id < ?) + )` + : '' + const params: unknown[] = [agentId] + if (after) { + params.push( + after.importance, + after.importance, + after.accessCount, + after.importance, + after.accessCount, + after.createdAt, + after.importance, + after.accessCount, + after.createdAt, + after.id + ) + } + params.push(cappedLimit) + return this.db + .prepare( + `SELECT * + FROM agent_memory + WHERE agent_id = ? + AND superseded_by IS NULL + AND status != 'archived' + AND status != 'conflicted' + AND kind IN ('semantic', 'reflection', 'episodic') + ${cursorSql} + ORDER BY importance DESC, access_count DESC, created_at DESC, id DESC + LIMIT ?` + ) + .all(...params) as AgentMemoryRow[] + } + hasActiveMemory(agentId: string): boolean { const row = this.db .prepare( @@ -1311,24 +1515,44 @@ export class AgentMemoryTable extends BaseTable { const uniqueIds = [...new Set(ids.filter((id) => id.trim()))] if (!uniqueIds.length) return [] const placeholders = uniqueIds.map(() => '?').join(', ') - const rows = this.db + const existingRows = this.db .prepare( - `SELECT id + `SELECT id, + embedding_id, + embedding_dim, + embedding_model, + kind, + superseded_by, + status FROM agent_memory WHERE agent_id = ? - AND id IN (${placeholders}) - AND embedding_id IS NOT NULL - AND embedding_dim = ? - AND embedding_model = ? - AND ( - kind IN ('persona', 'working') OR - superseded_by IS NOT NULL OR - status = 'archived' - )` + AND id IN (${placeholders})` ) - .all(agentId, ...uniqueIds, embeddingDim, embeddingModel) as Array<{ id: string }> - const liveIds = new Set(rows.map((row) => row.id)) - return uniqueIds.filter((id) => liveIds.has(id)) + .all(agentId, ...uniqueIds) as Array<{ + id: string + embedding_id: string | null + embedding_dim: number | null + embedding_model: string | null + kind: AgentMemoryKind + superseded_by: string | null + status: AgentMemoryStatus + }> + const existingIds = new Set(existingRows.map((row) => row.id)) + const prunableIds = new Set( + existingRows + .filter( + (row) => + row.embedding_id !== null && + row.embedding_dim === embeddingDim && + row.embedding_model === embeddingModel && + (row.kind === 'persona' || + row.kind === 'working' || + row.superseded_by !== null || + row.status === 'archived') + ) + .map((row) => row.id) + ) + return uniqueIds.filter((id) => !existingIds.has(id) || prunableIds.has(id)) } clearPrunableEmbeddingRefs( diff --git a/src/main/presenter/sqlitePresenter/tables/agentMemoryAudit.ts b/src/main/presenter/sqlitePresenter/tables/agentMemoryAudit.ts index 71ce4c6bf..d9da24de2 100644 --- a/src/main/presenter/sqlitePresenter/tables/agentMemoryAudit.ts +++ b/src/main/presenter/sqlitePresenter/tables/agentMemoryAudit.ts @@ -71,6 +71,15 @@ function stringifyMetadata(value: Record | undefined): string { return JSON.stringify(value ?? {}) } +function metadataReferencesMemoryId(metadataJson: string, memoryId: string): boolean { + try { + const metadata = JSON.parse(metadataJson) as Record + return metadata.memoryId === memoryId + } catch { + return false + } +} + export class AgentMemoryAuditTable extends BaseTable { constructor(db: Database.Database) { super(db, 'agent_memory_audit') @@ -224,6 +233,42 @@ export class AgentMemoryAuditTable extends BaseTable { return row?.at ?? null } + hasForgetEvent(agentId: string, memoryId: string): boolean { + const rows = this.db + .prepare( + `SELECT event_type, + actor_type, + input_refs_json, + output_refs_json + FROM agent_memory_audit + WHERE agent_id = ? + AND status = 'completed' + AND ( + (event_type = 'memory/forget' AND actor_type = 'runtime') + OR (event_type = 'memory/archive' AND actor_type = 'user') + OR event_type = 'memory/restore' + ) + ORDER BY created_at DESC, id DESC` + ) + .all(agentId) as Array<{ + event_type: string + actor_type: AgentMemoryAuditActorType + input_refs_json: string + output_refs_json: string + }> + for (const row of rows) { + if ( + !metadataReferencesMemoryId(row.input_refs_json, memoryId) && + !metadataReferencesMemoryId(row.output_refs_json, memoryId) + ) { + continue + } + if (row.event_type === 'memory/restore') return false + return true + } + return false + } + getHealthAuditStats( agentId: string, scanLimit: number, diff --git a/src/main/routes/index.ts b/src/main/routes/index.ts index 38a45d745..f3e82e770 100644 --- a/src/main/routes/index.ts +++ b/src/main/routes/index.ts @@ -105,6 +105,7 @@ import { memoryListRoute, memoryListViewManifestsRoute, memoryRejectPersonaDraftRoute, + memoryReindexRoute, memoryResolveConflictRoute, memoryRestoreRoute, memoryRollbackPersonaRoute, @@ -2401,6 +2402,19 @@ export async function dispatchDeepchatRoute( }) } + case memoryReindexRoute.name: { + const input = memoryReindexRoute.input.parse(rawInput) + const agentType = await runtime.configPresenter.getAgentType(input.agentId) + if (agentType !== 'deepchat' || !runtime.memoryPresenter.canReindex(input.agentId)) { + return memoryReindexRoute.output.parse({ started: false }) + } + const already = runtime.memoryPresenter.isReindexing(input.agentId) + void runtime.memoryPresenter.reindexEmbeddings(input.agentId, true).catch((error) => { + console.warn(`[Memory] manual reindex failed for ${input.agentId}: ${String(error)}`) + }) + return memoryReindexRoute.output.parse({ started: !already }) + } + case memoryGetLifecycleRoute.name: { const input = memoryGetLifecycleRoute.input.parse(rawInput) const agentType = await runtime.configPresenter.getAgentType(input.agentId) diff --git a/src/renderer/api/MemoryClient.ts b/src/renderer/api/MemoryClient.ts index 0a910093c..6ef786615 100644 --- a/src/renderer/api/MemoryClient.ts +++ b/src/renderer/api/MemoryClient.ts @@ -18,6 +18,7 @@ import { memoryListRoute, memoryListViewManifestsRoute, memoryRejectPersonaDraftRoute, + memoryReindexRoute, memoryResolveConflictRoute, memoryRestoreRoute, memoryRollbackPersonaRoute, @@ -179,6 +180,10 @@ export function createMemoryClient(bridge: DeepchatBridge = getDeepchatBridge()) return result.ok } + async function reindex(agentId: string): Promise<{ started: boolean }> { + return bridge.invoke(memoryReindexRoute.name, { agentId }) + } + async function getSourceSpan(agentId: string, memoryId: string): Promise { const result = await bridge.invoke(memoryGetSourceSpanRoute.name, { agentId, memoryId }) return result.span @@ -259,6 +264,7 @@ export function createMemoryClient(bridge: DeepchatBridge = getDeepchatBridge()) archive, clear, restore, + reindex, getSourceSpan, listConflicts, resolveConflict, diff --git a/src/renderer/settings/components/MemoryHealthSection.vue b/src/renderer/settings/components/MemoryHealthSection.vue index d9b7e1b2c..25e4c9ddb 100644 --- a/src/renderer/settings/components/MemoryHealthSection.vue +++ b/src/renderer/settings/components/MemoryHealthSection.vue @@ -138,8 +138,30 @@
-
- {{ t('settings.deepchatAgents.memoryManager.health.pipeline') }} +
+
+ {{ t('settings.deepchatAgents.memoryManager.health.pipeline') }} +
+
import { computed, defineComponent, h } from 'vue' import { useI18n } from 'vue-i18n' +import { Icon } from '@iconify/vue' import { Badge } from '@shadcn/components/ui/badge' +import { Button } from '@shadcn/components/ui/button' import type { MemoryArchiveCandidateLifecyclePreview, MemoryHealthDto, @@ -284,14 +308,20 @@ const props = withDefaults( archiveCandidateLifecyclePreview?: MemoryArchiveCandidateLifecyclePreview | null archiveCandidateLifecyclePreviewLoading?: boolean archiveCandidateLifecyclePreviewError?: string | null + reindexing?: boolean }>(), { archiveCandidateLifecyclePreview: null, archiveCandidateLifecyclePreviewLoading: false, - archiveCandidateLifecyclePreviewError: null + archiveCandidateLifecyclePreviewError: null, + reindexing: false } ) +const emit = defineEmits<{ + reindex: [] +}>() + const { t, locale } = useI18n() const dash = '—' diff --git a/src/renderer/settings/components/MemoryManagerPanel.vue b/src/renderer/settings/components/MemoryManagerPanel.vue index d2de9df85..0b1d5d187 100644 --- a/src/renderer/settings/components/MemoryManagerPanel.vue +++ b/src/renderer/settings/components/MemoryManagerPanel.vue @@ -393,6 +393,8 @@ :archive-candidate-lifecycle-preview="archiveCandidateLifecyclePreview" :archive-candidate-lifecycle-preview-loading="archiveCandidateLifecyclePreviewLoading" :archive-candidate-lifecycle-preview-error="archiveCandidateLifecyclePreviewError" + :reindexing="isReindexing" + @reindex="handleReindex" /> @@ -785,6 +787,7 @@ const auditEvents = ref([]) const viewManifests = ref([]) const status = ref(null) const health = ref(null) +const reindexPendingAgentId = ref(null) const healthDirty = ref(true) const archiveCandidateLifecyclePreview = ref(null) const archiveCandidateLifecyclePreviewLoading = ref(false) @@ -905,6 +908,16 @@ function markHealthDirty(): void { archiveCandidateLifecyclePreviewLoading.value = false } +const isReindexing = computed( + () => reindexPendingAgentId.value === props.agentId || status.value?.reindexing === true +) + +function settleReindexPending(agentId: string): void { + if (reindexPendingAgentId.value === agentId && status.value?.reindexing !== true) { + reindexPendingAgentId.value = null + } +} + function refreshHealthIfActive(): void { if (activeTab.value === 'health') void ensureHealthFresh() } @@ -1389,9 +1402,36 @@ async function handleRestore(memoryId: string): Promise { } } +async function handleReindex(): Promise { + if (!props.agentId || isReindexing.value) return + const agentId = props.agentId + reindexPendingAgentId.value = agentId + try { + const result = await memoryClient.reindex(agentId) + if (props.agentId === agentId && status.value) { + status.value = { ...status.value, reindexing: result.started || status.value.reindexing } + } + if (!result.started && reindexPendingAgentId.value === agentId) { + reindexPendingAgentId.value = null + } + if (props.agentId === agentId) { + await Promise.all([refresh(), refreshHealth(agentId)]) + settleReindexPending(agentId) + } + } catch (e) { + if (reindexPendingAgentId.value === agentId) { + reindexPendingAgentId.value = null + } + notifyActionFailed(e) + } +} + watch( () => props.agentId, - () => { + (agentId) => { + if (reindexPendingAgentId.value !== agentId) { + reindexPendingAgentId.value = null + } activeTab.value = 'memories' categoryFilter.value = 'all' markHealthDirty() @@ -1412,7 +1452,14 @@ onMounted(() => { if (payload.agentId === props.agentId) { memoryUpdateVersion += 1 markHealthDirtyFromEvent(payload.reason) - void refresh() + const refreshTask = refresh() + if (payload.reason === 'reindex') { + void refreshTask.then(() => { + settleReindexPending(payload.agentId) + }) + } else { + void refreshTask + } } }) }) diff --git a/src/renderer/src/i18n/da-DK/settings.json b/src/renderer/src/i18n/da-DK/settings.json index e97c57f52..8fdf48907 100644 --- a/src/renderer/src/i18n/da-DK/settings.json +++ b/src/renderer/src/i18n/da-DK/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "Efter kategori", "byStatus": "Efter status", "pipeline": "Pipeline", + "reindex": "Genindekser", + "reindexing": "Genindekserer", "quality": "Kvalitet", "topAccessed": "Mest brugt", "noTopAccessed": "Ingen brugte minder endnu.", diff --git a/src/renderer/src/i18n/de-DE/settings.json b/src/renderer/src/i18n/de-DE/settings.json index a5bda0d86..4eb856a92 100644 --- a/src/renderer/src/i18n/de-DE/settings.json +++ b/src/renderer/src/i18n/de-DE/settings.json @@ -250,6 +250,8 @@ "byCategory": "Nach Kategorie", "byStatus": "Nach Status", "pipeline": "Pipeline", + "reindex": "Neu indizieren", + "reindexing": "Indizierung läuft", "quality": "Qualität", "topAccessed": "Am häufigsten genutzt", "noTopAccessed": "Noch keine genutzten Erinnerungen.", diff --git a/src/renderer/src/i18n/en-US/settings.json b/src/renderer/src/i18n/en-US/settings.json index 6e96a6d55..f676fb1a5 100644 --- a/src/renderer/src/i18n/en-US/settings.json +++ b/src/renderer/src/i18n/en-US/settings.json @@ -247,6 +247,8 @@ "byCategory": "By category", "byStatus": "By status", "pipeline": "Pipeline", + "reindex": "Reindex", + "reindexing": "Reindexing", "quality": "Quality", "topAccessed": "Top accessed", "noTopAccessed": "No accessed memories yet.", diff --git a/src/renderer/src/i18n/es-ES/settings.json b/src/renderer/src/i18n/es-ES/settings.json index 494dc2ae9..b4b51dadc 100644 --- a/src/renderer/src/i18n/es-ES/settings.json +++ b/src/renderer/src/i18n/es-ES/settings.json @@ -250,6 +250,8 @@ "byCategory": "Por categoría", "byStatus": "Por estado", "pipeline": "Canalización", + "reindex": "Reindexar", + "reindexing": "Reindexando", "quality": "Calidad", "topAccessed": "Más usadas", "noTopAccessed": "Aún no hay memorias usadas.", diff --git a/src/renderer/src/i18n/fa-IR/settings.json b/src/renderer/src/i18n/fa-IR/settings.json index 0d2ccd3ba..8bd18b96b 100644 --- a/src/renderer/src/i18n/fa-IR/settings.json +++ b/src/renderer/src/i18n/fa-IR/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "بر اساس دسته", "byStatus": "بر اساس وضعیت", "pipeline": "خط پردازش", + "reindex": "بازنمایه‌سازی", + "reindexing": "در حال بازنمایه‌سازی", "quality": "کیفیت", "topAccessed": "بیشترین استفاده", "noTopAccessed": "هنوز حافظه استفاده‌شده‌ای نیست.", diff --git a/src/renderer/src/i18n/fr-FR/settings.json b/src/renderer/src/i18n/fr-FR/settings.json index c971a686a..e7e50f8a4 100644 --- a/src/renderer/src/i18n/fr-FR/settings.json +++ b/src/renderer/src/i18n/fr-FR/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "Par catégorie", "byStatus": "Par statut", "pipeline": "Pipeline", + "reindex": "Réindexer", + "reindexing": "Réindexation en cours", "quality": "Qualité", "topAccessed": "Les plus consultées", "noTopAccessed": "Aucune mémoire consultée pour le moment.", diff --git a/src/renderer/src/i18n/he-IL/settings.json b/src/renderer/src/i18n/he-IL/settings.json index 4798ce422..9f3ca45d1 100644 --- a/src/renderer/src/i18n/he-IL/settings.json +++ b/src/renderer/src/i18n/he-IL/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "לפי קטגוריה", "byStatus": "לפי סטטוס", "pipeline": "צינור עיבוד", + "reindex": "בנה אינדקס מחדש", + "reindexing": "בונה אינדקס מחדש", "quality": "איכות", "topAccessed": "הכי נגישים", "noTopAccessed": "אין עדיין זיכרונות שנעשה בהם שימוש.", diff --git a/src/renderer/src/i18n/id-ID/settings.json b/src/renderer/src/i18n/id-ID/settings.json index 5511cd7e1..9e94b4186 100644 --- a/src/renderer/src/i18n/id-ID/settings.json +++ b/src/renderer/src/i18n/id-ID/settings.json @@ -250,6 +250,8 @@ "byCategory": "Menurut kategori", "byStatus": "Menurut status", "pipeline": "Pipeline", + "reindex": "Indeks ulang", + "reindexing": "Mengindeks ulang", "quality": "Kualitas", "topAccessed": "Paling sering diakses", "noTopAccessed": "Belum ada memori yang diakses.", diff --git a/src/renderer/src/i18n/it-IT/settings.json b/src/renderer/src/i18n/it-IT/settings.json index 6d78e0d54..04f4b2da0 100644 --- a/src/renderer/src/i18n/it-IT/settings.json +++ b/src/renderer/src/i18n/it-IT/settings.json @@ -250,6 +250,8 @@ "byCategory": "Per categoria", "byStatus": "Per stato", "pipeline": "Pipeline", + "reindex": "Reindicizza", + "reindexing": "Reindicizzazione", "quality": "Qualità", "topAccessed": "Più consultate", "noTopAccessed": "Nessuna memoria consultata finora.", diff --git a/src/renderer/src/i18n/ja-JP/settings.json b/src/renderer/src/i18n/ja-JP/settings.json index 6f2a7aa2c..1f47baa61 100644 --- a/src/renderer/src/i18n/ja-JP/settings.json +++ b/src/renderer/src/i18n/ja-JP/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "カテゴリ別", "byStatus": "状態別", "pipeline": "パイプライン", + "reindex": "再インデックス", + "reindexing": "再インデックス中", "quality": "品質", "topAccessed": "アクセス上位", "noTopAccessed": "アクセスされたメモリーはまだありません。", diff --git a/src/renderer/src/i18n/ko-KR/settings.json b/src/renderer/src/i18n/ko-KR/settings.json index 66c327657..6e32ded8c 100644 --- a/src/renderer/src/i18n/ko-KR/settings.json +++ b/src/renderer/src/i18n/ko-KR/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "카테고리별", "byStatus": "상태별", "pipeline": "파이프라인", + "reindex": "다시 색인", + "reindexing": "다시 색인 중", "quality": "품질", "topAccessed": "최다 사용", "noTopAccessed": "아직 사용된 메모리가 없습니다.", diff --git a/src/renderer/src/i18n/ms-MY/settings.json b/src/renderer/src/i18n/ms-MY/settings.json index 4fd423df6..7538d7abf 100644 --- a/src/renderer/src/i18n/ms-MY/settings.json +++ b/src/renderer/src/i18n/ms-MY/settings.json @@ -250,6 +250,8 @@ "byCategory": "Mengikut kategori", "byStatus": "Mengikut status", "pipeline": "Saluran", + "reindex": "Indeks semula", + "reindexing": "Sedang mengindeks semula", "quality": "Kualiti", "topAccessed": "Paling kerap diakses", "noTopAccessed": "Belum ada memori yang diakses.", diff --git a/src/renderer/src/i18n/pl-PL/settings.json b/src/renderer/src/i18n/pl-PL/settings.json index 0f7ed20a1..674962c58 100644 --- a/src/renderer/src/i18n/pl-PL/settings.json +++ b/src/renderer/src/i18n/pl-PL/settings.json @@ -250,6 +250,8 @@ "byCategory": "Według kategorii", "byStatus": "Według statusu", "pipeline": "Pipeline", + "reindex": "Przeindeksuj", + "reindexing": "Ponowne indeksowanie", "quality": "Jakość", "topAccessed": "Najczęściej używane", "noTopAccessed": "Brak użytych wspomnień.", diff --git a/src/renderer/src/i18n/pt-BR/settings.json b/src/renderer/src/i18n/pt-BR/settings.json index 745a6aa60..9cea20818 100644 --- a/src/renderer/src/i18n/pt-BR/settings.json +++ b/src/renderer/src/i18n/pt-BR/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "Por categoria", "byStatus": "Por status", "pipeline": "Pipeline", + "reindex": "Reindexar", + "reindexing": "Reindexando", "quality": "Qualidade", "topAccessed": "Mais acessadas", "noTopAccessed": "Nenhuma memória acessada ainda.", diff --git a/src/renderer/src/i18n/ru-RU/settings.json b/src/renderer/src/i18n/ru-RU/settings.json index 17f49d7ef..1d02249d0 100644 --- a/src/renderer/src/i18n/ru-RU/settings.json +++ b/src/renderer/src/i18n/ru-RU/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "По категории", "byStatus": "По статусу", "pipeline": "Конвейер", + "reindex": "Переиндексировать", + "reindexing": "Переиндексация", "quality": "Качество", "topAccessed": "Чаще всего использовались", "noTopAccessed": "Использованных воспоминаний пока нет.", diff --git a/src/renderer/src/i18n/tr-TR/settings.json b/src/renderer/src/i18n/tr-TR/settings.json index 4c4762a9e..0da4cdb15 100644 --- a/src/renderer/src/i18n/tr-TR/settings.json +++ b/src/renderer/src/i18n/tr-TR/settings.json @@ -250,6 +250,8 @@ "byCategory": "Kategoriye göre", "byStatus": "Duruma göre", "pipeline": "İş hattı", + "reindex": "Yeniden indeksle", + "reindexing": "Yeniden indeksleniyor", "quality": "Kalite", "topAccessed": "En çok erişilen", "noTopAccessed": "Henüz erişilmiş bellek yok.", diff --git a/src/renderer/src/i18n/vi-VN/settings.json b/src/renderer/src/i18n/vi-VN/settings.json index eaf2eccf8..33f2185c3 100644 --- a/src/renderer/src/i18n/vi-VN/settings.json +++ b/src/renderer/src/i18n/vi-VN/settings.json @@ -250,6 +250,8 @@ "byCategory": "Theo danh mục", "byStatus": "Theo trạng thái", "pipeline": "Pipeline", + "reindex": "Lập chỉ mục lại", + "reindexing": "Đang lập chỉ mục lại", "quality": "Chất lượng", "topAccessed": "Truy cập nhiều nhất", "noTopAccessed": "Chưa có bộ nhớ nào được truy cập.", diff --git a/src/renderer/src/i18n/zh-CN/settings.json b/src/renderer/src/i18n/zh-CN/settings.json index 6f9fcb0d0..58768bcc9 100644 --- a/src/renderer/src/i18n/zh-CN/settings.json +++ b/src/renderer/src/i18n/zh-CN/settings.json @@ -247,6 +247,8 @@ "byCategory": "按分类", "byStatus": "按状态", "pipeline": "管线", + "reindex": "重建索引", + "reindexing": "重建中", "quality": "质量", "topAccessed": "访问最多", "noTopAccessed": "暂无被访问过的记忆。", diff --git a/src/renderer/src/i18n/zh-HK/settings.json b/src/renderer/src/i18n/zh-HK/settings.json index 399dd139a..15eecad2b 100644 --- a/src/renderer/src/i18n/zh-HK/settings.json +++ b/src/renderer/src/i18n/zh-HK/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "按分類", "byStatus": "按狀態", "pipeline": "管線", + "reindex": "重建索引", + "reindexing": "重建中", "quality": "質素", "topAccessed": "存取最多", "noTopAccessed": "暫無被存取過的記憶。", diff --git a/src/renderer/src/i18n/zh-TW/settings.json b/src/renderer/src/i18n/zh-TW/settings.json index 5ec9c5865..33ebf791a 100644 --- a/src/renderer/src/i18n/zh-TW/settings.json +++ b/src/renderer/src/i18n/zh-TW/settings.json @@ -2645,6 +2645,8 @@ "byCategory": "依分類", "byStatus": "依狀態", "pipeline": "管線", + "reindex": "重建索引", + "reindexing": "重建中", "quality": "品質", "topAccessed": "存取最多", "noTopAccessed": "尚無被存取過的記憶。", diff --git a/src/shared/contracts/routes.ts b/src/shared/contracts/routes.ts index f32bc0ac9..5a9398fb1 100644 --- a/src/shared/contracts/routes.ts +++ b/src/shared/contracts/routes.ts @@ -46,6 +46,7 @@ import { memoryListRoute, memoryListViewManifestsRoute, memoryRejectPersonaDraftRoute, + memoryReindexRoute, memoryResolveConflictRoute, memoryRestoreRoute, memoryRollbackPersonaRoute, @@ -873,6 +874,7 @@ const DEEPCHAT_ROUTE_CATALOG_PART_5 = { [memoryGetByIdsRoute.name]: memoryGetByIdsRoute, [memoryGetStatusRoute.name]: memoryGetStatusRoute, [memoryGetHealthRoute.name]: memoryGetHealthRoute, + [memoryReindexRoute.name]: memoryReindexRoute, [memoryGetLifecycleRoute.name]: memoryGetLifecycleRoute, [memoryGetArchiveCandidateLifecyclePreviewRoute.name]: memoryGetArchiveCandidateLifecyclePreviewRoute, diff --git a/src/shared/contracts/routes/memory.routes.ts b/src/shared/contracts/routes/memory.routes.ts index c65269a37..1d051c76a 100644 --- a/src/shared/contracts/routes/memory.routes.ts +++ b/src/shared/contracts/routes/memory.routes.ts @@ -331,6 +331,12 @@ export const memoryGetHealthRoute = defineRouteContract({ output: z.object({ health: MemoryHealthSchema }) }) +export const memoryReindexRoute = defineRouteContract({ + name: 'memory.reindex', + input: z.object({ agentId: AgentIdSchema }), + output: z.object({ started: z.boolean() }) +}) + export const memoryGetLifecycleRoute = defineRouteContract({ name: 'memory.getLifecycle', input: z.object({ agentId: AgentIdSchema, memoryId: z.string().min(1) }), diff --git a/test/main/presenter/agentMemoryTable.test.ts b/test/main/presenter/agentMemoryTable.test.ts index c4ee7f1e6..b35c74c07 100644 --- a/test/main/presenter/agentMemoryTable.test.ts +++ b/test/main/presenter/agentMemoryTable.test.ts @@ -831,6 +831,81 @@ describeIfSqlite('AgentMemoryTable', () => { } }) + it('listWorkingCandidates pages by the stable working-blob ordering cursor', () => { + const db = new DatabaseCtor(':memory:') + try { + const table = new AgentMemoryTableCtor(db) + table.createTable() + table.insert({ + id: 'm1', + agentId: 'a', + kind: 'semantic', + content: 'first', + importance: 0.4, + createdAt: 1000 + }) + table.insert({ + id: 'm2', + agentId: 'a', + kind: 'episodic', + content: 'second', + importance: 0.8, + createdAt: 2000 + }) + table.insert({ + id: 'm3', + agentId: 'a', + kind: 'semantic', + content: 'third', + importance: 0.9, + createdAt: 3000 + }) + table.insert({ + id: 'm4', + agentId: 'a', + kind: 'semantic', + content: 'fourth', + importance: 0.9, + createdAt: 4000 + }) + table.recordAccess('m3', 5000) + table.recordAccess('m4', 5000) + table.recordAccess('m4', 6000) + const archived = table.insert({ + id: 'archived', + agentId: 'a', + kind: 'semantic', + content: 'archived', + importance: 1, + createdAt: 9000 + }) + table.archive(archived.id) + table.insert({ id: 'working', agentId: 'a', kind: 'working', content: 'working' }) + const superseded = table.insert({ + id: 'superseded', + agentId: 'a', + kind: 'semantic', + content: 'superseded', + importance: 1, + createdAt: 8000 + }) + table.markSuperseded(superseded.id, 'm4') + + const firstPage = table.listWorkingCandidates('a', 2) + expect(firstPage.map((row) => row.id)).toEqual(['m4', 'm3']) + const cursorRow = firstPage[1] + const secondPage = table.listWorkingCandidates('a', 2, { + importance: cursorRow.importance, + accessCount: cursorRow.access_count, + createdAt: cursorRow.created_at, + id: cursorRow.id + }) + expect(secondPage.map((row) => row.id)).toEqual(['m2', 'm1']) + } finally { + db.close() + } + }) + it('lists archive candidate lifecycle projections without content payloads', () => { const db = new DatabaseCtor(':memory:') try { @@ -1213,6 +1288,29 @@ describeIfSqlite('AgentMemoryTable FTS5 + migration', () => { } }) + it('requeueForEmbedding supports a bounded id cursor for fair error retry', () => { + const db = new DatabaseCtor(':memory:') + try { + const table = new AgentMemoryTableCtor(db) + table.createTable() + for (const id of ['err-01', 'err-02', 'err-03']) { + table.insert({ id, agentId: 'a', kind: 'semantic', content: id, status: 'error' }) + } + + expect(table.listEmbeddingStatusIds('a', ['error'], 2, 'err-01')).toEqual([ + 'err-02', + 'err-03' + ]) + expect(table.requeueForEmbedding('a', ['error'], 1, 'err-01')).toBe(1) + + expect(table.getById('err-01')?.status).toBe('error') + expect(table.getById('err-02')?.status).toBe('pending_embedding') + expect(table.getById('err-03')?.status).toBe('error') + } finally { + db.close() + } + }) + it('listPendingEmbedding never returns persona rows even if one is marked pending', () => { const db = new DatabaseCtor(':memory:') try { @@ -1382,6 +1480,9 @@ describeIfSqlite('AgentMemoryTable FTS5 + migration', () => { expect( table.filterPrunableVectorRefs('a', ['active', 'archived', 'superseded'], 3, 'p:m') ).toEqual(['archived', 'superseded']) + expect(table.filterPrunableVectorRefs('a', ['missing-row'], 3, 'p:m')).toEqual([ + 'missing-row' + ]) expect( table.clearPrunableEmbeddingRefs('a', ['active', 'archived', 'superseded'], 3, 'p:m') ).toBe(2) @@ -1820,6 +1921,122 @@ describeIfSqlite('AgentMemoryTable FTS5 + migration', () => { } }) + it('agent memory audit hasForgetEvent honors restore ordering', () => { + const db = new DatabaseCtor(':memory:') + try { + const table = new AgentMemoryAuditTableCtor(db) + table.createTable() + table.insert({ + id: 'forget-1', + agentId: 'a', + eventType: 'memory/forget', + actorType: 'runtime', + status: 'completed', + inputRefs: { memoryId: 'm1' }, + outputRefs: { memoryId: 'm1' }, + createdAt: 100 + }) + expect(table.hasForgetEvent('a', 'm1')).toBe(true) + + table.insert({ + id: 'restore-1', + agentId: 'a', + eventType: 'memory/restore', + actorType: 'user', + status: 'completed', + inputRefs: { memoryId: 'm1' }, + outputRefs: { memoryId: 'm1' }, + createdAt: 200 + }) + expect(table.hasForgetEvent('a', 'm1')).toBe(false) + + table.insert({ + id: 'archive-1', + agentId: 'a', + eventType: 'memory/archive', + actorType: 'user', + status: 'completed', + inputRefs: { memoryId: 'm1' }, + outputRefs: { memoryId: 'm1' }, + createdAt: 300 + }) + expect(table.hasForgetEvent('a', 'm1')).toBe(true) + + table.insert({ + id: 'restore-failed', + agentId: 'a', + eventType: 'memory/restore', + actorType: 'user', + status: 'failed', + inputRefs: { memoryId: 'm1' }, + outputRefs: { memoryId: 'm1' }, + createdAt: 400 + }) + expect(table.hasForgetEvent('a', 'm1')).toBe(true) + expect(table.hasForgetEvent('a', 'other')).toBe(false) + } finally { + db.close() + } + }) + + it('agent memory audit hasForgetEvent does not miss older memory refs behind newer events', () => { + const db = new DatabaseCtor(':memory:') + try { + const table = new AgentMemoryAuditTableCtor(db) + table.createTable() + table.insert({ + id: 'old-forget', + agentId: 'a', + eventType: 'memory/forget', + actorType: 'runtime', + status: 'completed', + inputRefs: { memoryId: 'm-old' }, + outputRefs: { memoryId: 'm-old' }, + createdAt: 1 + }) + for (let index = 0; index < 205; index += 1) { + table.insert({ + id: `newer-other-${index}`, + agentId: 'a', + eventType: 'memory/restore', + actorType: 'user', + status: 'completed', + inputRefs: { memoryId: `other-${index}` }, + outputRefs: { memoryId: `other-${index}` }, + createdAt: 1000 + index + }) + } + + expect(table.hasForgetEvent('a', 'm-old')).toBe(true) + + table.insert({ + id: 'new-restore', + agentId: 'a', + eventType: 'memory/restore', + actorType: 'user', + status: 'completed', + inputRefs: { memoryId: 'm-old' }, + outputRefs: { memoryId: 'm-old' }, + createdAt: 2000 + }) + expect(table.hasForgetEvent('a', 'm-old')).toBe(false) + + table.insert({ + id: 'new-archive', + agentId: 'a', + eventType: 'memory/archive', + actorType: 'user', + status: 'completed', + inputRefs: { memoryId: 'm-old' }, + outputRefs: { memoryId: 'm-old' }, + createdAt: 2001 + }) + expect(table.hasForgetEvent('a', 'm-old')).toBe(true) + } finally { + db.close() + } + }) + it('agent memory audit computes bounded health status counts and recent failures', () => { const db = new DatabaseCtor(':memory:') try { diff --git a/test/main/presenter/fakes/memoryFakes.ts b/test/main/presenter/fakes/memoryFakes.ts index 412ffb1bf..70de655e7 100644 --- a/test/main/presenter/fakes/memoryFakes.ts +++ b/test/main/presenter/fakes/memoryFakes.ts @@ -15,6 +15,7 @@ import type { AgentMemoryInsertInput, AgentMemoryLifecycleRow, AgentMemoryRow, + AgentMemoryWorkingCandidateCursor, IMemoryVectorStore, MemoryAuditListOptions, MemoryAuditRepositoryPort, @@ -46,6 +47,7 @@ function toLifecycleRow(row: AgentMemoryRow): AgentMemoryLifecycleRow { // exercise the presenter without a native database. export class FakeRepository implements MemoryRepositoryPort { rows = new Map() + transactionCalls = 0 insert(input: AgentMemoryInsertInput): AgentMemoryRow { if (input.provenanceKey) { @@ -267,9 +269,26 @@ export class FakeRepository implements MemoryRepositoryPort { return true } - requeueForEmbedding(agentId: string, statuses: AgentMemoryRow['status'][]) { + requeueForEmbedding( + agentId: string, + statuses: AgentMemoryRow['status'][], + limit?: number, + afterId?: string | null + ) { let changed = 0 - for (const row of this.rows.values()) { + const candidates = [...this.rows.values()] + .filter( + (row) => + row.agent_id === agentId && + !row.superseded_by && + row.kind !== 'persona' && + row.kind !== 'working' && + statuses.includes(row.status) && + (!afterId || row.id > afterId) + ) + .sort((a, b) => a.id.localeCompare(b.id)) + .slice(0, limit === undefined ? undefined : Math.max(0, Math.floor(limit))) + for (const row of candidates) { if ( row.agent_id !== agentId || row.superseded_by || @@ -287,6 +306,27 @@ export class FakeRepository implements MemoryRepositoryPort { return changed } + listEmbeddingStatusIds( + agentId: string, + statuses: AgentMemoryRow['status'][], + limit: number, + afterId?: string | null + ) { + return [...this.rows.values()] + .filter( + (row) => + row.agent_id === agentId && + !row.superseded_by && + row.kind !== 'persona' && + row.kind !== 'working' && + statuses.includes(row.status) && + (!afterId || row.id > afterId) + ) + .sort((a, b) => a.id.localeCompare(b.id)) + .slice(0, Math.max(0, Math.floor(limit))) + .map((row) => row.id) + } + markSuperseded(id: string, supersededBy: string | null) { const row = this.rows.get(id) if (row) row.superseded_by = supersededBy @@ -314,6 +354,23 @@ export class FakeRepository implements MemoryRepositoryPort { } } + refreshDecayScoresForAgent(agentId: string, now: number, halfLifeMs: number) { + for (const row of this.listByAgent(agentId)) { + if (row.kind === 'persona') continue + const anchor = row.last_accessed ?? row.created_at + const age = Math.max(0, now - anchor) + const importance = Math.min(1, Math.max(0, row.importance)) + row.decay_score = Math.pow(0.5, age / (halfLifeMs * (1 + importance))) + } + } + + stampConsolidationForAgent(agentId: string, at: number) { + for (const row of this.listByAgent(agentId)) { + if (row.kind === 'persona') continue + row.last_consolidated_at = at + } + } + updateContent( id: string, content: string, @@ -531,6 +588,67 @@ export class FakeRepository implements MemoryRepositoryPort { return this.listByAgent(agentId, { includeSuperseded: true }).length } + countStatusView(agentId: string) { + const rows = [...this.rows.values()].filter( + (row) => + row.agent_id === agentId && + row.status !== 'archived' && + row.status !== 'conflicted' && + row.kind !== 'working' + ) + return { + total: rows.length, + pendingEmbedding: rows.filter((row) => row.status === 'pending_embedding').length + } + } + + runInTransaction(fn: () => T): T { + this.transactionCalls += 1 + const snapshot = new Map([...this.rows.entries()].map(([id, row]) => [id, { ...row }])) + try { + return fn() + } catch (error) { + this.rows = snapshot + throw error + } + } + + listWorkingCandidates(agentId: string, limit: number, after?: AgentMemoryWorkingCandidateCursor) { + const isAfterCursor = (row: AgentMemoryRow): boolean => { + if (!after) return true + return ( + row.importance < after.importance || + (row.importance === after.importance && row.access_count < after.accessCount) || + (row.importance === after.importance && + row.access_count === after.accessCount && + row.created_at < after.createdAt) || + (row.importance === after.importance && + row.access_count === after.accessCount && + row.created_at === after.createdAt && + row.id < after.id) + ) + } + + return [...this.rows.values()] + .filter( + (row) => + row.agent_id === agentId && + row.superseded_by === null && + row.status !== 'archived' && + row.status !== 'conflicted' && + (row.kind === 'semantic' || row.kind === 'reflection' || row.kind === 'episodic') && + isAfterCursor(row) + ) + .sort( + (a, b) => + b.importance - a.importance || + b.access_count - a.access_count || + b.created_at - a.created_at || + b.id.localeCompare(a.id) + ) + .slice(0, Math.max(0, Math.floor(limit))) + } + hasActiveMemory(agentId: string) { return [...this.rows.values()].some( (row) => row.agent_id === agentId && row.status !== 'archived' @@ -631,9 +749,10 @@ export class FakeRepository implements MemoryRepositoryPort { embeddingModel: string ) { const uniqueIds = [...new Set(ids.filter((id) => id.trim()))] - return uniqueIds.filter((id) => - this.isPrunableVectorRow(agentId, this.rows.get(id), embeddingDim, embeddingModel) - ) + return uniqueIds.filter((id) => { + const row = this.rows.get(id) + return !row || this.isPrunableVectorRow(agentId, row, embeddingDim, embeddingModel) + }) } clearPrunableEmbeddingRefs( @@ -714,6 +833,26 @@ export class FakeAuditRepository implements MemoryAuditRepositoryPort { return latest } + hasForgetEvent(agentId: string, memoryId: string): boolean { + const rows = this.rows + .filter((row) => { + if (row.agent_id !== agentId || row.status !== 'completed') return false + return ( + (row.event_type === 'memory/forget' && row.actor_type === 'runtime') || + (row.event_type === 'memory/archive' && row.actor_type === 'user') || + row.event_type === 'memory/restore' + ) + }) + .sort((a, b) => b.created_at - a.created_at || b.id.localeCompare(a.id)) + for (const row of rows) { + const input = JSON.parse(row.input_refs_json) as Record + const output = JSON.parse(row.output_refs_json) as Record + if (input.memoryId !== memoryId && output.memoryId !== memoryId) continue + return row.event_type !== 'memory/restore' + } + return false + } + getHealthAuditStats( agentId: string, scanLimit: number, @@ -773,6 +912,13 @@ export class FakeVectorStore implements IMemoryVectorStore { for (const id of memoryIds) this.vectors.delete(id) } + async listMemoryIds(afterId: string | null, limit: number) { + return [...this.vectors.keys()] + .filter((id) => afterId === null || id > afterId) + .sort((a, b) => a.localeCompare(b)) + .slice(0, Math.max(0, Math.floor(limit))) + } + async close() { this.closeCount += 1 } diff --git a/test/main/presenter/memoryExtraction.test.ts b/test/main/presenter/memoryExtraction.test.ts index 25704fcf8..4ae8a3fc2 100644 --- a/test/main/presenter/memoryExtraction.test.ts +++ b/test/main/presenter/memoryExtraction.test.ts @@ -158,8 +158,10 @@ describe('MemoryPresenter.extractAndStore', () => { query: async () => [], queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], clear: async () => {}, - close: async () => {} + close: async () => {}, + isUsable: () => true }) }) @@ -213,8 +215,10 @@ describe('MemoryPresenter.extractAndStore', () => { query: async () => [], queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], clear: async () => {}, - close: async () => {} + close: async () => {}, + isUsable: () => true }) }) @@ -249,8 +253,10 @@ describe('MemoryPresenter.extractAndStore', () => { query: async () => [], queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], clear: async () => {}, - close: async () => {} + close: async () => {}, + isUsable: () => true }) }) @@ -303,6 +309,7 @@ describe('MemoryPresenter.extractAndStore triage gate, cheap model, lineage', () query: async () => [], queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], close: async () => {}, isUsable: () => true }), @@ -436,6 +443,7 @@ describe('MemoryPresenter.maybeReflect cheap model', () => { query: async () => [], queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], close: async () => {}, isUsable: () => true }), @@ -558,7 +566,14 @@ function makeFakeRepo() { last_accessed: null, access_count: 0, decay_score: null, - source_entry_ids: input.sourceEntryIds?.length ? JSON.stringify(input.sourceEntryIds) : null + source_entry_ids: input.sourceEntryIds?.length + ? JSON.stringify(input.sourceEntryIds) + : null, + confidence: null, + last_consolidated_at: null, + conflict_state: null, + conflict_with: input.conflictWith ?? null, + persona_state: input.personaState ?? null } rows.set(row.id, row) return row @@ -583,10 +598,69 @@ function makeFakeRepo() { [...rows.values()] .filter((r) => r.status === 'pending_embedding' && (!agentId || r.agent_id === agentId)) .slice(0, limit), + updatePendingEmbeddingStatus: ( + agentId: string, + id: string, + status: string, + embedding?: { + embeddingId?: string | null + embeddingDim?: number | null + embeddingModel?: string | null + } + ) => { + const r = rows.get(id) + if (!r || r.agent_id !== agentId || r.status !== 'pending_embedding') return false + r.status = status + r.embedding_id = embedding?.embeddingId ?? null + r.embedding_dim = embedding?.embeddingDim ?? null + r.embedding_model = embedding?.embeddingModel ?? null + return true + }, updateStatus: (id: string, status: string) => { const r = rows.get(id) if (r) r.status = status }, + requeueForEmbedding: ( + agentId: string, + statuses: string[], + limit?: number, + afterId?: string | null + ) => { + const candidates = [...rows.values()] + .filter( + (r) => + r.agent_id === agentId && + !r.superseded_by && + statuses.includes(r.status) && + (!afterId || r.id > afterId) + ) + .sort((a, b) => a.id.localeCompare(b.id)) + .slice(0, limit === undefined ? undefined : Math.max(0, Math.floor(limit))) + for (const r of candidates) { + r.status = 'pending_embedding' + r.embedding_id = null + r.embedding_dim = null + r.embedding_model = null + } + return candidates.length + }, + listEmbeddingStatusIds: ( + agentId: string, + statuses: string[], + limit: number, + afterId?: string | null + ) => + [...rows.values()] + .filter( + (r) => + r.agent_id === agentId && + !r.superseded_by && + statuses.includes(r.status) && + (!afterId || r.id > afterId) + ) + .sort((a, b) => a.id.localeCompare(b.id)) + .slice(0, Math.max(0, Math.floor(limit))) + .map((row) => row.id), updateContent: ( id: string, content: string, @@ -603,6 +677,53 @@ function makeFakeRepo() { }, markSuperseded: () => {}, recordAccess: () => {}, + recordAccessBatch: () => {}, + setPersonaState: () => {}, + setAnchor: () => {}, + getDraftPersona: () => undefined, + setConfidence: () => {}, + setImportance: () => {}, + markConflict: () => {}, + setConflictWith: () => {}, + setLastConsolidatedAt: () => {}, + getLastConsolidatedAt: () => null, + getCurrentEmbeddingDimension: () => null, + getHealthStats: () => ({ + totalRows: 0, + byKind: { semantic: 0, episodic: 0, reflection: 0, persona: 0, working: 0 }, + byCategory: { + user_preference: 0, + project_fact: 0, + task_outcome: 0, + heuristic: 0, + anti_pattern: 0, + uncategorized: 0 + }, + byStatus: { + pending_embedding: 0, + embedded: 0, + fts_only: 0, + error: 0, + archived: 0, + conflicted: 0 + }, + neverAccessed: 0, + importanceAvg: null, + importanceMedian: null, + confidenceAvg: null, + conflicted: 0, + challenged: 0 + }), + hasStaleEmbeddings: () => false, + countStaleEmbeddings: () => 0, + archive: (id: string) => { + const r = rows.get(id) + if (r) r.status = 'archived' + }, + listArchiveCandidates: () => [], + listArchiveCandidateLifecycleRows: () => [], + countArchiveCandidates: () => 0, + listTopAccessed: () => [], delete: (id: string) => rows.delete(id), clearByAgent: (agentId: string) => { let n = 0 @@ -611,7 +732,67 @@ function makeFakeRepo() { }, countByAgent: (agentId: string) => [...rows.values()].filter((r) => r.agent_id === agentId).length, + countStatusView: (agentId: string) => { + const view = [...rows.values()].filter( + (r) => r.agent_id === agentId && r.status !== 'archived' && r.status !== 'conflicted' + ) + return { + total: view.length, + pendingEmbedding: view.filter((r) => r.status === 'pending_embedding').length + } + }, + hasActiveMemory: () => false, + listAgentIdsWithMemories: () => [], + runInTransaction: (fn: () => T): T => { + const snapshot = new Map([...rows.entries()].map(([id, row]) => [id, { ...row }])) + try { + return fn() + } catch (error) { + rows.clear() + for (const [id, row] of snapshot) rows.set(id, row) + throw error + } + }, + listWorkingCandidates: ( + agentId: string, + limit: number, + after?: { importance: number; accessCount: number; createdAt: number; id: string } + ) => + [...rows.values()] + .filter((r) => { + if ( + r.agent_id !== agentId || + r.superseded_by !== null || + r.status === 'archived' || + r.status === 'conflicted' || + !['semantic', 'reflection', 'episodic'].includes(r.kind) + ) { + return false + } + if (!after) return true + return ( + r.importance < after.importance || + (r.importance === after.importance && r.access_count < after.accessCount) || + (r.importance === after.importance && + r.access_count === after.accessCount && + r.created_at < after.createdAt) || + (r.importance === after.importance && + r.access_count === after.accessCount && + r.created_at === after.createdAt && + r.id < after.id) + ) + }) + .sort( + (a, b) => + b.importance - a.importance || + b.access_count - a.access_count || + b.created_at - a.created_at || + b.id.localeCompare(a.id) + ) + .slice(0, Math.max(0, Math.floor(limit))), listConsolidationScanRows: () => [], + refreshDecayScoresForAgent: () => {}, + stampConsolidationForAgent: () => {}, repairInternalKindStatuses: () => 0, listPrunableVectorRefs: () => [], filterPrunableVectorRefs: () => [], diff --git a/test/main/presenter/memoryPresenter.test.ts b/test/main/presenter/memoryPresenter.test.ts index 35359833d..3223afbdb 100644 --- a/test/main/presenter/memoryPresenter.test.ts +++ b/test/main/presenter/memoryPresenter.test.ts @@ -494,6 +494,36 @@ describe('working-memory L1 (T5)', () => { expect(working?.content).not.toContain('x'.repeat(2000)) }) + it('continues past the first oversized working candidate page', () => { + const { presenter, repo } = makePresenter(enabledConfig) + const listWorkingCandidatesSpy = vi.spyOn(repo, 'listWorkingCandidates') + for (let index = 0; index < 64; index += 1) { + repo.insert({ + id: `long-${String(index).padStart(3, '0')}`, + agentId: 'deepchat', + kind: 'semantic', + content: `oversized ${index} ${'x'.repeat(2000)}`, + importance: 1, + createdAt: 10_000 - index + }) + } + repo.insert({ + id: 'short-after-page', + agentId: 'deepchat', + kind: 'semantic', + content: 'short fact after long candidates', + importance: 0.9, + createdAt: 1 + }) + + presenter.refreshWorkingMemory('deepchat') + + const working = [...repo.rows.values()].find((row) => row.kind === 'working') + expect(working?.content).toContain('short fact after long candidates') + expect(listWorkingCandidatesSpy.mock.calls.length).toBeGreaterThan(1) + expect(listWorkingCandidatesSpy.mock.calls[1][2]).toMatchObject({ id: 'long-063' }) + }) + it('falls back to recall when no working blob exists', async () => { const { presenter, repo } = makePresenter(enabledConfig) repo.insert({ @@ -595,22 +625,16 @@ describe('working-memory L1 (T5)', () => { it('coalesces concurrent cold-start misses into a single refresh', async () => { const { presenter, repo } = makePresenter(enabledConfig) - const listByAgentSpy = vi.spyOn(repo, 'listByAgent') + const listWorkingCandidatesSpy = vi.spyOn(repo, 'listWorkingCandidates') // Two opens race before either refresh microtask runs; the in-flight flag collapses them to one. await Promise.all([ presenter.buildInjection('deepchat', 'q'), presenter.buildInjection('deepchat', 'q') ]) await flushMicrotasks() - const workingRefreshScans = listByAgentSpy.mock.calls.filter(([agentId, options]) => { - const kinds = options?.kinds ?? [] - return ( - agentId === 'deepchat' && - kinds.includes('semantic') && - kinds.includes('reflection') && - kinds.includes('episodic') - ) - }) + const workingRefreshScans = listWorkingCandidatesSpy.mock.calls.filter( + ([agentId]) => agentId === 'deepchat' + ) expect(workingRefreshScans).toHaveLength(1) }) @@ -2173,7 +2197,7 @@ describe('MemoryPresenter management', () => { resolveCreate = () => resolve(store) }) ) - const newContent = 'user likes redis preference' + const newContent = 'postgres preference' const embeddingCalls: Array<{ texts: string[]; rowExists: boolean }> = [] const getEmbeddings = vi.fn(async (_p: string, _m: string, texts: string[]) => { embeddingCalls.push({ @@ -2905,6 +2929,7 @@ describe('MemoryPresenter.processPendingEmbeddings (batch + fairness)', () => { query: async () => [], queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], close: async () => {}, isUsable: () => true } @@ -2941,6 +2966,7 @@ describe('MemoryPresenter.processPendingEmbeddings (batch + fairness)', () => { query: async () => [], queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], close: async () => {}, isUsable: () => true } @@ -2984,6 +3010,7 @@ describe('MemoryPresenter.processPendingEmbeddings (batch + fairness)', () => { query: async () => [], queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], close: async () => {}, isUsable: () => false } @@ -3113,6 +3140,124 @@ describe('MemoryPresenter.processPendingEmbeddings (batch + fairness)', () => { expect(repo.getById('m1')?.status).toBe('embedded') expect(repo.getById('m2')?.status).toBe('embedded') }) + + it('does not issue an empty error requeue when there are no error rows', async () => { + const { presenter, repo } = makePresenter(enabledConfig) + const requeueSpy = vi.spyOn(repo, 'requeueForEmbedding') + + await presenter.processPendingEmbeddings('a') + + expect(requeueSpy).not.toHaveBeenCalled() + }) + + it('cools down error retry even when a bounded requeue races to zero changes', async () => { + const { presenter, repo } = makePresenter(enabledConfig) + repo.insert({ + id: 'err-01', + agentId: 'a', + kind: 'semantic', + content: 'retry race', + status: 'error' + }) + const requeueSpy = vi.spyOn(repo, 'requeueForEmbedding') + requeueSpy.mockReturnValueOnce(0) + const embeddingState = ( + presenter as unknown as { + embedding: { + getMutableRuntimeStateForTests(): { + errorRetryAt: Map + errorRetryAfterId: Map + } + } + } + ).embedding.getMutableRuntimeStateForTests() + + await presenter.processPendingEmbeddings('a') + + expect(requeueSpy).toHaveBeenCalledTimes(1) + expect(embeddingState.errorRetryAt.has('a')).toBe(true) + expect(embeddingState.errorRetryAfterId.get('a')).toBe('err-01') + + await presenter.processPendingEmbeddings('a') + + expect(requeueSpy).toHaveBeenCalledTimes(1) + }) + + it('advances error retry fairly instead of starving rows after the first batch', async () => { + const { presenter, repo } = makePresenter(enabledConfig) + for (let index = 0; index < 51; index += 1) { + repo.insert({ + id: `err-${String(index).padStart(2, '0')}`, + agentId: 'a', + kind: 'semantic', + content: `retry ${index}`, + status: 'error' + }) + } + const embeddingState = ( + presenter as unknown as { + embedding: { + getMutableRuntimeStateForTests(): { + errorRetryAt: Map + errorRetryAfterId: Map + } + } + } + ).embedding.getMutableRuntimeStateForTests() + + await presenter.processPendingEmbeddings('a') + expect(repo.getById('err-49')?.status).toBe('embedded') + expect(repo.getById('err-50')?.status).toBe('error') + expect(embeddingState.errorRetryAfterId.get('a')).toBe('err-49') + + embeddingState.errorRetryAt.set('a', 0) + await presenter.processPendingEmbeddings('a') + + expect(repo.getById('err-50')?.status).toBe('embedded') + }) + + it('sets an error retry cooldown when embedding vectors have mismatched dimensions', async () => { + const repo = new FakeRepository() + const presenter = new MemoryPresenter({ + repository: repo, + resolveAgentConfig: () => enabledConfig, + getEmbeddings: async () => [ + [1, 2, 3, 4], + [1, 2] + ], + createVectorStore: async () => new FakeVectorStore(), + resetVectorStore: async () => undefined + }) + repo.insert({ + id: 'ok', + agentId: 'a', + kind: 'semantic', + content: 'ok', + status: 'pending_embedding' + }) + repo.insert({ + id: 'bad', + agentId: 'a', + kind: 'semantic', + content: 'bad', + status: 'pending_embedding' + }) + const embeddingState = ( + presenter as unknown as { + embedding: { + getMutableRuntimeStateForTests(): { + errorRetryAt: Map + } + } + } + ).embedding.getMutableRuntimeStateForTests() + + await presenter.processPendingEmbeddings('a') + + expect(repo.getById('ok')?.status).toBe('embedded') + expect(repo.getById('bad')?.status).toBe('error') + expect(embeddingState.errorRetryAt.has('a')).toBe(true) + }) }) describe('MemoryPresenter change events (onMemoryChanged)', () => { @@ -3154,6 +3299,14 @@ describe('MemoryPresenter change events (onMemoryChanged)', () => { expect(onMemoryChanged).not.toHaveBeenCalled() }) + it('emits "reindex" when a manual reindex returns early with no rows', async () => { + const { presenter, onMemoryChanged } = makeWithSpy() + + await presenter.reindexEmbeddings('a') + + expect(onMemoryChanged).toHaveBeenCalledWith('a', 'reindex') + }) + it('emits persona reasons through the draft / approve / rollback lifecycle', async () => { const { presenter, onMemoryChanged } = makeWithSpy() const v1 = presenter.evolvePersona('a', 'v1', null) @@ -3646,6 +3799,104 @@ describe('writeMemoriesSync insert error classification (C2, AC-2.2)', () => { }) describe('MemoryPresenter embedding reindex (T5, AC-3.x)', () => { + it('deletes orphan vectors in bounded batches during warm reconcile', async () => { + const { presenter, store } = makePresenter(enabledConfig) + for (let index = 0; index < 601; index += 1) { + store.vectors.set(`orphan-${String(index).padStart(4, '0')}`, textToVector('orphan redis')) + } + const deleteSpy = vi.spyOn(store, 'deleteByMemoryIds') + + await presenter.recall('a', 'redis') + await waitForMemoryCondition(() => store.vectors.size === 0, 'orphan vectors were not deleted') + + expect(deleteSpy).toHaveBeenCalledTimes(2) + expect(deleteSpy.mock.calls.map(([ids]) => ids.length)).toEqual([512, 89]) + expect(deleteSpy.mock.calls.every(([ids]) => ids.length <= 512)).toBe(true) + }) + + it('keeps vector recall ready and cools down after orphan reconcile delete failure', async () => { + const { presenter, store } = makePresenter(enabledConfig) + store.vectors.set('orphan-0001', textToVector('orphan redis')) + const deleteSpy = vi.spyOn(store, 'deleteByMemoryIds') + deleteSpy.mockRejectedValueOnce(new Error('delete failed')) + const embeddingState = ( + presenter as unknown as { + embedding: { + getMutableRuntimeStateForTests(): { + orphanVectorReconcileRetryAt: Map + } + } + } + ).embedding.getMutableRuntimeStateForTests() + + await presenter.recall('a', 'redis') + await waitForMemoryCondition(() => deleteSpy.mock.calls.length === 1) + expect(store.vectors.has('orphan-0001')).toBe(true) + expect( + (presenter as unknown as { vectorStoreReady: Map }).vectorStoreReady.has('a') + ).toBe(true) + + ;( + presenter as unknown as { clearVectorStoreReady(agentId: string): void } + ).clearVectorStoreReady('a') + await presenter.recall('a', 'redis') + await flushMicrotasks() + expect(deleteSpy).toHaveBeenCalledTimes(1) + + for (const key of embeddingState.orphanVectorReconcileRetryAt.keys()) { + embeddingState.orphanVectorReconcileRetryAt.set(key, 0) + } + ;( + presenter as unknown as { clearVectorStoreReady(agentId: string): void } + ).clearVectorStoreReady('a') + await presenter.recall('a', 'redis') + await waitForMemoryCondition( + () => deleteSpy.mock.calls.length >= 2 && !store.vectors.has('orphan-0001'), + 'orphan reconcile did not retry after failure' + ) + }) + + it('cleanupDeletedAgentResources waits for in-flight orphan reconcile before clearing marks', async () => { + const { presenter, store } = makePresenter(enabledConfig) + let releaseList!: () => void + vi.spyOn(store, 'listMemoryIds').mockImplementationOnce( + async () => + new Promise((resolve) => { + releaseList = () => resolve([]) + }) + ) + const embeddingState = ( + presenter as unknown as { + embedding: { + getMutableRuntimeStateForTests(): { + orphanVectorReconciles: Map> + orphanVectorReconciled: Set + } + } + } + ).embedding.getMutableRuntimeStateForTests() + + await presenter.recall('a', 'redis') + await waitForMemoryCondition( + () => embeddingState.orphanVectorReconciles.size === 1 && typeof releaseList === 'function', + 'orphan reconcile did not enter in-flight tracking' + ) + + let cleanupDone = false + const cleanup = presenter.cleanupDeletedAgentResources('a').then(() => { + cleanupDone = true + }) + await flushMicrotasks() + expect(cleanupDone).toBe(false) + + releaseList() + await cleanup + + expect(cleanupDone).toBe(true) + expect(embeddingState.orphanVectorReconciles.size).toBe(0) + expect(embeddingState.orphanVectorReconciled.size).toBe(0) + }) + it('reindexEmbeddings re-queues, rebuilds the store, and re-embeds with the new fingerprint', async () => { const repo = new FakeRepository() let config: DeepChatAgentConfig = { @@ -3938,6 +4189,7 @@ describe('MemoryPresenter embedding reindex (T5, AC-3.x)', () => { query: async () => [], queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], close: async () => {}, isUsable: () => false } @@ -5643,9 +5895,10 @@ describe('MemoryPresenter lifecycle revival (SDD-8)', () => { expect(recalled.some((m) => m.id === id)).toBe(true) }) - it('re-stating a superseded preference revives it and retires the contradicting head (AC-1.2)', async () => { + it('re-stating a superseded preference can revive it after a SUPERSEDE decision (AC-1.2)', async () => { const generateText = routedLLM({ - extraction: '[{"kind":"semantic","content":"user likes redis","importance":0.8}]' + extraction: '[{"kind":"semantic","content":"user likes redis","importance":0.8}]', + decision: '{"decision":"SUPERSEDE","targetIndex":0,"mergedContent":"user likes redis"}' }) const { presenter, repo } = makeLLMPresenter(generateText) const aId = await seedEmbedded(presenter, 'user likes redis') @@ -5667,6 +5920,88 @@ describe('MemoryPresenter lifecycle revival (SDD-8)', () => { expect(recalled.some((m) => m.id === bId)).toBe(false) }) + it('ADD fallback revives a superseded provenance owner instead of dropping the fact', async () => { + const generateText = routedLLM({ + extraction: '[{"kind":"semantic","content":"user likes redis","importance":0.8}]', + decision: '{"decision":"ADD","targetIndex":null,"mergedContent":null}' + }) + const { presenter, repo } = makeLLMPresenter(generateText) + const ownerId = await seedEmbedded(presenter, 'user likes redis') + const headId = await seedEmbedded(presenter, 'user dislikes redis') + repo.markSuperseded(ownerId, headId) + + const result = await presenter.extractAndStore({ + agentId: 'a', + spanText: 'User: I like redis again', + model: { providerId: 'main', modelId: 'main' } + }) + + if (!result.ok) throw new Error('expected ok') + expect(result.createdIds).toHaveLength(0) + expect(repo.getById(ownerId)).toMatchObject({ + status: 'pending_embedding', + superseded_by: null + }) + expect(repo.getById(headId)?.superseded_by).toBe(ownerId) + }) + + it('CHALLENGE fallback revives a superseded provenance owner and retires the head', async () => { + const generateText = routedLLM({ + extraction: '[{"kind":"semantic","content":"user likes redis","importance":0.8}]', + decision: '{"decision":"CHALLENGE","targetIndex":0,"mergedContent":null}' + }) + const { presenter, repo } = makeLLMPresenter(generateText) + const ownerId = await seedEmbedded(presenter, 'user likes redis') + const headId = await seedEmbedded(presenter, 'user dislikes redis') + repo.markSuperseded(ownerId, headId) + + const result = await presenter.extractAndStore({ + agentId: 'a', + spanText: 'User: I like redis again', + model: { providerId: 'main', modelId: 'main' } + }) + + if (!result.ok) throw new Error('expected ok') + expect(result.createdIds).toHaveLength(0) + expect(repo.getById(ownerId)?.superseded_by).toBeNull() + expect(repo.getById(headId)?.superseded_by).toBe(ownerId) + expect(repo.listByAgent('a', { statuses: ['conflicted'], includeSuperseded: true })).toEqual([]) + }) + + it('CHALLENGE fallback does not fold a target invalidated before the collision path', async () => { + const repo = new FakeRepository() + let headId = '' + const generateText = vi.fn(async (_p: string, _m: string, prompt: string) => { + if (prompt.includes('KEEP or SKIP')) return 'KEEP' + if (prompt.includes('JSON array')) { + return '[{"kind":"semantic","content":"user likes redis","importance":0.8}]' + } + if (prompt.includes('Choose exactly ONE decision')) { + if (headId) repo.archive(headId, Date.now()) + return '{"decision":"CHALLENGE","targetIndex":0,"mergedContent":null}' + } + return '' + }) + const { presenter } = makeLLMPresenter(generateText, embeddingConfig, repo) + const ownerId = await seedEmbedded(presenter, 'user likes redis') + headId = await seedEmbedded(presenter, 'user dislikes redis') + repo.markSuperseded(ownerId, headId) + + const result = await presenter.extractAndStore({ + agentId: 'a', + spanText: 'User: I like redis again', + model: { providerId: 'main', modelId: 'main' } + }) + + if (!result.ok) throw new Error('expected ok') + expect(repo.getById(ownerId)?.superseded_by).toBeNull() + expect(repo.getById(headId)).toMatchObject({ + status: 'archived', + superseded_by: null + }) + expect(repo.listByAgent('a', { statuses: ['conflicted'], includeSuperseded: true })).toEqual([]) + }) + it('SUPERSEDE whose merged wording collides with an archived row revives it and folds the target in (AC-1.4)', async () => { const generateText = routedLLM({ extraction: '[{"kind":"semantic","content":"user now hates redis","importance":0.8}]', @@ -5815,6 +6150,157 @@ describe('MemoryPresenter lifecycle revival (SDD-8)', () => { expect(repo.listByAgent('a')[0].id).toBe(ownerId) }) + it('rolls back an archived provenance revive when the fold transaction fails', async () => { + const generateText = routedLLM({ + extraction: '[{"kind":"semantic","content":"user enjoys redis","importance":0.8}]', + decision: '{"decision":"UPDATE","targetIndex":0,"mergedContent":"user prefers vue"}' + }) + const { presenter, repo } = makeLLMPresenter(generateText) + const ownerId = await seedEmbedded(presenter, 'user prefers vue') + const targetId = await seedEmbedded(presenter, 'user likes redis') + repo.archive(ownerId, 1) + const markSuperseded = vi.spyOn(repo, 'markSuperseded') + markSuperseded.mockImplementationOnce(() => { + throw new Error('fold failed') + }) + + const result = await presenter.extractAndStore({ + agentId: 'a', + spanText: 'User: I enjoy redis', + model: { providerId: 'main', modelId: 'main' } + }) + + expect(result.ok).toBe(false) + expect(repo.getById(ownerId)?.status).toBe('archived') + expect(repo.getById(targetId)?.superseded_by).toBeNull() + }) + + it('an UPDATE whose merged content collides with a superseded row revives the owner', async () => { + const generateText = routedLLM({ + extraction: '[{"kind":"semantic","content":"user enjoys redis","importance":0.8}]', + decision: '{"decision":"UPDATE","targetIndex":0,"mergedContent":"user prefers vue"}' + }) + const { presenter, repo } = makeLLMPresenter(generateText) + const targetId = await seedEmbedded(presenter, 'user likes redis') + const ownerId = await seedEmbedded(presenter, 'user prefers vue') + const headId = await seedEmbedded(presenter, 'user prefers react') + repo.markSuperseded(ownerId, headId) + + const result = await presenter.extractAndStore({ + agentId: 'a', + spanText: 'User: I enjoy redis', + model: { providerId: 'main', modelId: 'main' } + }) + + if (!result.ok) throw new Error('expected ok') + expect(result.createdIds).toHaveLength(0) + expect(repo.getById(ownerId)).toMatchObject({ + status: 'pending_embedding', + superseded_by: null + }) + expect(repo.getById(headId)?.superseded_by).toBe(ownerId) + expect(repo.getById(targetId)?.superseded_by).toBe(ownerId) + }) + + it('an UPDATE collision with user-forgotten provenance is a true noop', async () => { + const generateText = routedLLM({ + extraction: '[{"kind":"semantic","content":"user enjoys redis","importance":0.8}]', + decision: '{"decision":"UPDATE","targetIndex":0,"mergedContent":"user prefers vue"}' + }) + const { presenter, repo } = makeLLMPresenter(generateText) + const ownerId = await seedEmbedded(presenter, 'user prefers vue') + const targetId = await seedEmbedded(presenter, 'user likes redis') + expect(await presenter.archiveUserMemory('a', ownerId)).toBe(true) + const targetBefore = repo.getById(targetId)! + const beforeSnapshot = { + content: targetBefore.content, + provenanceKey: targetBefore.provenance_key, + confidence: targetBefore.confidence + } + + const result = await presenter.extractAndStore({ + agentId: 'a', + spanText: 'User: I enjoy redis', + model: { providerId: 'main', modelId: 'main' } + }) + + if (!result.ok) throw new Error('expected ok') + const targetAfter = repo.getById(targetId)! + expect(result.createdIds).toHaveLength(0) + expect(repo.getById(ownerId)?.status).toBe('archived') + expect(targetAfter.content).toBe(beforeSnapshot.content) + expect(targetAfter.provenance_key).toBe(beforeSnapshot.provenanceKey) + expect(targetAfter.status).toBe('embedded') + expect(targetAfter.confidence).toBe(beforeSnapshot.confidence) + expect(targetAfter.superseded_by).toBeNull() + }) + + it('restore audit offsets a prior forget so a decay-archived memory can be learned again', async () => { + vi.useFakeTimers() + try { + const generateText = routedLLM({ + extraction: '[{"kind":"semantic","content":"user likes redis","importance":0.8}]' + }) + const { presenter, repo, auditRepo } = makeLLMPresenter(generateText) + vi.setSystemTime(1000) + const id = await seedEmbedded(presenter, 'user likes redis') + vi.setSystemTime(2000) + expect(await presenter.forgetMemory('a', id)).toBe(true) + vi.setSystemTime(3000) + expect(presenter.restoreMemory('a', id)).toBe(true) + expect(auditRepo.hasForgetEvent('a', id)).toBe(false) + vi.setSystemTime(4000) + repo.archive(id, Date.now()) + + vi.setSystemTime(5000) + const result = await presenter.extractAndStore({ + agentId: 'a', + spanText: 'User: I like redis', + model: { providerId: 'main', modelId: 'main' } + }) + + if (!result.ok) throw new Error('expected ok') + expect(result.createdIds).toHaveLength(0) + expect(repo.getById(id)?.status).toBe('pending_embedding') + } finally { + vi.useRealTimers() + } + }) + + it('a forget after restore remains the latest provenance tombstone', async () => { + vi.useFakeTimers() + try { + const generateText = routedLLM({ + extraction: '[{"kind":"semantic","content":"user likes redis","importance":0.8}]' + }) + const { presenter, repo, auditRepo } = makeLLMPresenter(generateText) + vi.setSystemTime(1000) + const id = await seedEmbedded(presenter, 'user likes redis') + vi.setSystemTime(2000) + expect(await presenter.forgetMemory('a', id)).toBe(true) + vi.setSystemTime(3000) + expect(presenter.restoreMemory('a', id)).toBe(true) + vi.setSystemTime(4000) + repo.archive(id, Date.now()) + vi.setSystemTime(5000) + expect(await presenter.forgetMemory('a', id)).toBe(true) + expect(auditRepo.hasForgetEvent('a', id)).toBe(true) + + vi.setSystemTime(6000) + const result = await presenter.extractAndStore({ + agentId: 'a', + spanText: 'User: I like redis', + model: { providerId: 'main', modelId: 'main' } + }) + + if (!result.ok) throw new Error('expected ok') + expect(result.createdIds).toHaveLength(0) + expect(repo.getById(id)?.status).toBe('archived') + } finally { + vi.useRealTimers() + } + }) + it('a consolidation pass interrupted by dispose writes nothing to the repository (AC-3.1)', async () => { let resolveLLM = (): void => {} const llmGate = new Promise((resolve) => { @@ -6161,6 +6647,7 @@ describe('MemoryPresenter lifecycle revival (SDD-8)', () => { }), queryByMemoryId: async () => [], deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], close: async () => {}, isUsable: () => true } @@ -6246,6 +6733,131 @@ describe('MemoryPresenter lifecycle revival (SDD-8)', () => { expect(deleteSpy).not.toHaveBeenCalled() // no write against the store dispose just closed }) + it('deletes vectors from the row snapshot embedding store even after config changes', async () => { + const repo = new FakeRepository() + const store = new FakeVectorStore() + const createVectorStore = vi.fn(async () => store) + const presenter = new MemoryPresenter({ + repository: repo, + resolveAgentConfig: () => ({ + memoryEnabled: true, + memoryEmbedding: { providerId: 'p', modelId: 'current' } + }), + getEmbeddings: async (_p, _m, texts) => texts.map((text) => textToVector(text)), + createVectorStore, + resetVectorStore: async () => undefined + }) + repo.insert({ id: 'm1', agentId: 'a', kind: 'semantic', content: 'redis fact' }) + repo.updateStatus('m1', 'embedded', { + embeddingId: 'm1', + embeddingDim: 4, + embeddingModel: 'p:legacy' + }) + store.vectors.set('m1', textToVector('redis fact')) + const deleteSpy = vi.spyOn(store, 'deleteByMemoryIds') + + expect(await presenter.deleteMemory('a', 'm1')).toBe(true) + + expect(createVectorStore).toHaveBeenCalledWith('a', { providerId: 'p', modelId: 'legacy' }, 4) + expect(deleteSpy).toHaveBeenCalledWith(['m1']) + expect(repo.getById('m1')).toBeUndefined() + }) + + it('keeps the row snapshot embedding model when delete must infer the dimension', async () => { + const repo = new FakeRepository() + const store = new FakeVectorStore() + const createVectorStore = vi.fn(async () => store) + const presenter = new MemoryPresenter({ + repository: repo, + resolveAgentConfig: () => ({ + memoryEnabled: true, + memoryEmbedding: { providerId: 'p', modelId: 'current' } + }), + getEmbeddings: async (_p, _m, texts) => texts.map((text) => textToVector(text)), + createVectorStore, + resetVectorStore: async () => undefined + }) + repo.insert({ id: 'legacy-ref', agentId: 'a', kind: 'semantic', content: 'legacy ref' }) + repo.updateStatus('legacy-ref', 'embedded', { + embeddingId: 'legacy-ref', + embeddingDim: 4, + embeddingModel: 'p:legacy' + }) + repo.insert({ id: 'current-ref', agentId: 'a', kind: 'semantic', content: 'current ref' }) + repo.updateStatus('current-ref', 'embedded', { + embeddingId: 'current-ref', + embeddingDim: 6, + embeddingModel: 'p:current' + }) + repo.insert({ id: 'm1', agentId: 'a', kind: 'semantic', content: 'redis fact' }) + repo.updateStatus('m1', 'embedded', { + embeddingId: 'm1', + embeddingDim: null, + embeddingModel: 'p:legacy' + }) + store.vectors.set('m1', textToVector('redis fact')) + const deleteSpy = vi.spyOn(store, 'deleteByMemoryIds') + + expect(await presenter.deleteMemory('a', 'm1')).toBe(true) + + expect(createVectorStore).toHaveBeenCalledWith('a', { providerId: 'p', modelId: 'legacy' }, 4) + expect(deleteSpy).toHaveBeenCalledWith(['m1']) + }) + + it('does not fail the SQLite delete when opening the vector store fails', async () => { + const repo = new FakeRepository() + const presenter = new MemoryPresenter({ + repository: repo, + resolveAgentConfig: () => enabledConfig, + getEmbeddings: async (_p, _m, texts) => texts.map((text) => textToVector(text)), + createVectorStore: async () => { + throw new Error('open failed') + }, + resetVectorStore: async () => undefined + }) + repo.insert({ id: 'm1', agentId: 'a', kind: 'semantic', content: 'redis fact' }) + repo.updateStatus('m1', 'embedded', { + embeddingId: 'm1', + embeddingDim: 4, + embeddingModel: 'p:m' + }) + + await expect(presenter.deleteMemory('a', 'm1')).resolves.toBe(true) + expect(repo.getById('m1')).toBeUndefined() + }) + + it('schedules a forced reindex when delete finds an unusable vector store', async () => { + const repo = new FakeRepository() + const unusableStore: IMemoryVectorStore = { + upsert: async () => {}, + query: async () => [], + queryByMemoryId: async () => [], + deleteByMemoryIds: async () => {}, + listMemoryIds: async () => [], + close: async () => {}, + isUsable: () => false + } + const presenter = new MemoryPresenter({ + repository: repo, + resolveAgentConfig: () => enabledConfig, + getEmbeddings: async (_p, _m, texts) => texts.map((text) => textToVector(text)), + createVectorStore: async () => unusableStore, + resetVectorStore: async () => undefined + }) + repo.insert({ id: 'm1', agentId: 'a', kind: 'semantic', content: 'redis fact' }) + repo.updateStatus('m1', 'embedded', { + embeddingId: 'm1', + embeddingDim: 4, + embeddingModel: 'p:m' + }) + const reindexSpy = vi.spyOn(presenter, 'reindexEmbeddings').mockResolvedValue() + + expect(await presenter.deleteMemory('a', 'm1')).toBe(true) + + expect(reindexSpy).toHaveBeenCalledWith('a', true) + expect(repo.getById('m1')).toBeUndefined() + }) + it('dispose waits for an in-flight vector delete before closing the store (AC-3.11)', async () => { const { presenter, repo, store } = makeLLMPresenter(routedLLM({})) const id = await seedEmbedded(presenter, 'user likes redis') diff --git a/test/main/presenter/memoryVectorStore.test.ts b/test/main/presenter/memoryVectorStore.test.ts index 6b333d582..13631add1 100644 --- a/test/main/presenter/memoryVectorStore.test.ts +++ b/test/main/presenter/memoryVectorStore.test.ts @@ -35,6 +35,12 @@ interface QueryableStore { ): Promise> } +interface ListableStore { + connection: { runAndReadAll: ReturnType } + vectorTable: string + listMemoryIds(afterId: string | null, limit: number): Promise +} + function makeStore(onRun: (sql: string) => void = () => {}) { const calls: string[] = [] const connection = { @@ -127,6 +133,38 @@ describe('MemoryVectorStore.queryByMemoryId', () => { }) }) +describe('MemoryVectorStore.listMemoryIds', () => { + it('uses keyset pagination and a bounded limit', async () => { + const connection = { + runAndReadAll: vi.fn(async () => ({ + getRowObjectsJson: () => [{ memory_id: 'm2' }, { memory_id: 'm3' }] + })) + } + const store = Object.create(MemoryVectorStore.prototype) as ListableStore + store.connection = connection + store.vectorTable = 'memory_vector' + + await expect(store.listMemoryIds('m1', 2)).resolves.toEqual(['m2', 'm3']) + + expect(connection.runAndReadAll).toHaveBeenCalledWith( + expect.stringContaining('memory_id > ?'), + ['m1', 2] + ) + }) + + it('returns early for a zero limit', async () => { + const connection = { + runAndReadAll: vi.fn() + } + const store = Object.create(MemoryVectorStore.prototype) as ListableStore + store.connection = connection + store.vectorTable = 'memory_vector' + + await expect(store.listMemoryIds(null, 0)).resolves.toEqual([]) + expect(connection.runAndReadAll).not.toHaveBeenCalled() + }) +}) + interface EmbeddingMeta { provider: string model: string diff --git a/test/main/routes/dispatcher.test.ts b/test/main/routes/dispatcher.test.ts index 0620b20ee..df4f1c2ce 100644 --- a/test/main/routes/dispatcher.test.ts +++ b/test/main/routes/dispatcher.test.ts @@ -2127,6 +2127,72 @@ describe('dispatchDeepchatRoute', () => { expect(archiveUserMemory).toHaveBeenCalledWith('deepchat', 'm1') }) + it('dispatches memory.reindex only when a new managed task can start', async () => { + const { runtime, configPresenter } = createRuntime() + const canReindex = vi.fn(() => true) + const isReindexing = vi.fn(() => false) + const reindexEmbeddings = vi.fn().mockResolvedValue(undefined) + ;(runtime as any).memoryPresenter = { canReindex, isReindexing, reindexEmbeddings } + + await expect( + dispatchDeepchatRoute( + runtime, + 'memory.reindex', + { agentId: 'other' }, + { webContentsId: 42, windowId: 7 } + ) + ).resolves.toEqual({ started: false }) + expect(canReindex).not.toHaveBeenCalled() + expect(reindexEmbeddings).not.toHaveBeenCalled() + + canReindex.mockReturnValueOnce(false) + await expect( + dispatchDeepchatRoute( + runtime, + 'memory.reindex', + { agentId: 'deepchat' }, + { webContentsId: 42, windowId: 7 } + ) + ).resolves.toEqual({ started: false }) + expect(canReindex).toHaveBeenCalledWith('deepchat') + expect(reindexEmbeddings).not.toHaveBeenCalled() + + canReindex.mockReturnValue(true) + isReindexing.mockReturnValueOnce(true) + await expect( + dispatchDeepchatRoute( + runtime, + 'memory.reindex', + { agentId: 'deepchat' }, + { webContentsId: 42, windowId: 7 } + ) + ).resolves.toEqual({ started: false }) + expect(reindexEmbeddings).toHaveBeenCalledTimes(1) + expect(reindexEmbeddings).toHaveBeenLastCalledWith('deepchat', true) + + isReindexing.mockReturnValueOnce(false) + await expect( + dispatchDeepchatRoute( + runtime, + 'memory.reindex', + { agentId: 'deepchat' }, + { webContentsId: 42, windowId: 7 } + ) + ).resolves.toEqual({ started: true }) + expect(reindexEmbeddings).toHaveBeenCalledTimes(2) + + vi.mocked(configPresenter.getAgentType).mockResolvedValueOnce('acp') + await expect( + dispatchDeepchatRoute( + runtime, + 'memory.reindex', + { agentId: 'deepchat' }, + { webContentsId: 42, windowId: 7 } + ) + ).resolves.toEqual({ started: false }) + expect(reindexEmbeddings).toHaveBeenCalledTimes(2) + }) + it('returns no memory view manifests for missing or non-DeepChat agents', async () => { const { runtime, configPresenter } = createRuntime() const listMemoryViewManifestAnchorsByAgent = vi.fn() diff --git a/test/main/routes/memoryDto.test.ts b/test/main/routes/memoryDto.test.ts index 9be698ad9..5a068c270 100644 --- a/test/main/routes/memoryDto.test.ts +++ b/test/main/routes/memoryDto.test.ts @@ -10,6 +10,7 @@ import { memoryGetHealthRoute, memoryGetLifecycleRoute, memoryListRoute, + memoryReindexRoute, memoryRestoreRoute, memorySearchRoute } from '@shared/contracts/routes' @@ -185,6 +186,16 @@ describe('memory.restore route contract round-trip', () => { }) }) +describe('memory.reindex route contract', () => { + it('round-trips the reindex request and started response', () => { + expect(memoryReindexRoute.input.parse({ agentId: 'deepchat-abc123' })).toEqual({ + agentId: 'deepchat-abc123' + }) + expect(memoryReindexRoute.output.parse({ started: true })).toEqual({ started: true }) + expect(memoryReindexRoute.output.parse({ started: false })).toEqual({ started: false }) + }) +}) + describe('memory.getHealth route contract', () => { it('round-trips the empty health DTO and a populated bounded preview', () => { expect(memoryGetHealthRoute.input.parse({ agentId: 'deepchat' })).toEqual({ diff --git a/test/renderer/api/clients.test.ts b/test/renderer/api/clients.test.ts index ce740e633..0eeee1a54 100644 --- a/test/renderer/api/clients.test.ts +++ b/test/renderer/api/clients.test.ts @@ -1007,6 +1007,8 @@ describe('renderer api clients', () => { } case 'memory.archive': return { ok: true } + case 'memory.reindex': + return { started: true } case 'memory.getHealth': return { health: { @@ -1417,6 +1419,7 @@ describe('renderer api clients', () => { await memoryClient.add('agent-1', { content: 'plain note' }) const selectedMemories = await memoryClient.getByIds('agent-1', ['mem-archived', 'mem-added']) const archived = await memoryClient.archive('agent-1', 'mem-added') + const reindex = await memoryClient.reindex('agent-1') const health = await memoryClient.getHealth('agent-1') const lifecycle = await memoryClient.getLifecycle('agent-1', 'mem-1') const archiveCandidatePreview = @@ -1485,15 +1488,17 @@ describe('renderer api clients', () => { memoryId: 'mem-added' }) expect(archived).toBe(true) - expect(bridge.invoke).toHaveBeenNthCalledWith(17, 'memory.getHealth', { agentId: 'agent-1' }) + expect(bridge.invoke).toHaveBeenNthCalledWith(17, 'memory.reindex', { agentId: 'agent-1' }) + expect(reindex.started).toBe(true) + expect(bridge.invoke).toHaveBeenNthCalledWith(18, 'memory.getHealth', { agentId: 'agent-1' }) expect(health.totalRows).toBe(1) - expect(bridge.invoke).toHaveBeenNthCalledWith(18, 'memory.getLifecycle', { + expect(bridge.invoke).toHaveBeenNthCalledWith(19, 'memory.getLifecycle', { agentId: 'agent-1', memoryId: 'mem-1' }) expect(lifecycle?.memoryId).toBe('mem-1') expect(bridge.invoke).toHaveBeenNthCalledWith( - 19, + 20, 'memory.getArchiveCandidateLifecyclePreview', { agentId: 'agent-1' diff --git a/test/renderer/components/MemoryHealthSection.test.ts b/test/renderer/components/MemoryHealthSection.test.ts index e4b37d07a..9f8f68615 100644 --- a/test/renderer/components/MemoryHealthSection.test.ts +++ b/test/renderer/components/MemoryHealthSection.test.ts @@ -23,6 +23,21 @@ vi.mock('@shadcn/components/ui/badge', () => ({ } })) +vi.mock('@shadcn/components/ui/button', () => ({ + Button: { + name: 'Button', + template: '' + } +})) + +vi.mock('@iconify/vue', () => ({ + addCollection: vi.fn(), + Icon: { + name: 'Icon', + template: '' + } +})) + interface SettingsJson { deepchatAgents?: { memoryManager?: { @@ -150,6 +165,7 @@ function mountSection(props: { archiveCandidateLifecyclePreview?: MemoryArchiveCandidateLifecyclePreview | null archiveCandidateLifecyclePreviewLoading?: boolean archiveCandidateLifecyclePreviewError?: string | null + reindexing?: boolean }) { return mount(MemoryHealthSection, { props: { @@ -158,7 +174,8 @@ function mountSection(props: { error: props.error ?? null, archiveCandidateLifecyclePreview: props.archiveCandidateLifecyclePreview, archiveCandidateLifecyclePreviewLoading: props.archiveCandidateLifecyclePreviewLoading, - archiveCandidateLifecyclePreviewError: props.archiveCandidateLifecyclePreviewError + archiveCandidateLifecyclePreviewError: props.archiveCandidateLifecyclePreviewError ?? null, + reindexing: props.reindexing ?? false } }) } @@ -206,7 +223,22 @@ describe('MemoryHealthSection', () => { expect(wrapper.text()).toContain('memory/maintenance_llm') expect(wrapper.text()).toContain('model unavailable') expect(wrapper.text()).toContain('—') - expect(wrapper.find('button').exists()).toBe(false) + expect(wrapper.find('button').text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindex' + ) + }) + + it('emits reindex and renders the in-progress label', async () => { + const ready = mountSection({ health: loadedHealth }) + await ready.find('button').trigger('click') + expect(ready.emitted('reindex')).toHaveLength(1) + expect(ready.find('button').attributes('aria-busy')).toBeUndefined() + + const running = mountSection({ health: loadedHealth, reindexing: true }) + const button = running.find('button') + expect(button.text()).toContain('settings.deepchatAgents.memoryManager.health.reindexing') + expect(button.attributes('disabled')).toBeDefined() + expect(button.attributes('aria-busy')).toBe('true') }) it('renders archive candidate lifecycle preview states without memory content', () => { diff --git a/test/renderer/components/MemoryManagerDialog.test.ts b/test/renderer/components/MemoryManagerDialog.test.ts index 511094daf..3ae5bdc3b 100644 --- a/test/renderer/components/MemoryManagerDialog.test.ts +++ b/test/renderer/components/MemoryManagerDialog.test.ts @@ -243,6 +243,7 @@ async function setup( manifestPromise?: Promise auditReject?: boolean manifestReject?: boolean + reindexPromise?: Promise<{ started: boolean }> } = {} ) { vi.resetModules() @@ -314,6 +315,9 @@ async function setup( rejectPersonaDraft: vi.fn().mockResolvedValue(overrides.reject ?? true), setPersonaAnchor: vi.fn().mockResolvedValue(overrides.anchor ?? true), resolveConflict: vi.fn().mockResolvedValue(true), + reindex: overrides.reindexPromise + ? vi.fn().mockReturnValue(overrides.reindexPromise) + : vi.fn().mockResolvedValue({ started: true }), onUpdated: vi.fn().mockImplementation((listener) => { updateListener = listener return dispose @@ -390,8 +394,11 @@ async function setup( memoryClient, toast, dispose, - emitMemoryUpdated: (reason: MemoryUpdatedPayload['reason'] = 'extract') => - updateListener?.({ agentId: 'a', reason, version: 1 }) + emitMemoryUpdated: ( + reason: MemoryUpdatedPayload['reason'] = 'extract', + agentId = 'a', + version = 1 + ) => updateListener?.({ agentId, reason, version }) } } @@ -445,6 +452,14 @@ async function deactivateHealthTab( await clickTab(wrapper, 'memories') } +function findReindexButton(wrapper: Awaited>['wrapper']) { + const button = wrapper + .findAllComponents(ButtonStub) + .find((item) => item.text().includes('settings.deepchatAgents.memoryManager.health.reindex')) + if (!button) throw new Error('Missing reindex button') + return button +} + function findSelectByText(wrapper: Awaited>['wrapper'], text: string) { const select = wrapper .findAllComponents({ name: 'Select' }) @@ -979,6 +994,8 @@ describe('MemoryManagerDialog memory health', () => { }) })) vi.doMock('@shadcn/components/ui/badge', () => ({ Badge: passStub('Badge') })) + vi.doMock('@shadcn/components/ui/button', () => ({ Button: passStub('Button') })) + vi.doMock('@iconify/vue', () => ({ addCollection: vi.fn(), Icon: passStub('Icon') })) const MemoryHealthSection = ( await import('../../../src/renderer/settings/components/MemoryHealthSection.vue') ).default @@ -996,7 +1013,7 @@ describe('MemoryManagerDialog memory health', () => { expect(wrapper.text()).toContain('memory/maintenance_llm') expect(wrapper.text()).toContain('model unavailable') expect(wrapper.text()).toContain('—') - expect(wrapper.find('button').exists()).toBe(false) + expect(wrapper.text()).toContain('settings.deepchatAgents.memoryManager.health.reindex') }) it('renders zero as a valid last accessed timestamp', async () => { @@ -1042,6 +1059,112 @@ describe('MemoryManagerDialog memory health', () => { expect(memoryClient.getArchiveCandidateLifecyclePreview).toHaveBeenCalledTimes(1) }) + it('does not carry local reindex pending state across agents', async () => { + const reindex = deferred<{ started: boolean }>() + const { wrapper, memoryClient } = await setup({ reindexPromise: reindex.promise }) + + await activateHealthTab(wrapper) + await findReindexButton(wrapper).trigger('click') + await nextTick() + + expect(memoryClient.reindex).toHaveBeenCalledWith('a') + expect(findReindexButton(wrapper).text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindexing' + ) + + await wrapper.setProps({ agentId: 'b' }) + await flushPromises() + await activateHealthTab(wrapper) + + expect(memoryClient.getStatus).toHaveBeenLastCalledWith('b') + expect(findReindexButton(wrapper).text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindex' + ) + expect(findReindexButton(wrapper).text()).not.toContain( + 'settings.deepchatAgents.memoryManager.health.reindexing' + ) + + reindex.resolve({ started: true }) + await flushPromises() + }) + + it('settles local reindex pending from matching-agent reindex status refreshes only', async () => { + const reindex = deferred<{ started: boolean }>() + const { wrapper, memoryClient, emitMemoryUpdated } = await setup({ + reindexPromise: reindex.promise + }) + + await activateHealthTab(wrapper) + await findReindexButton(wrapper).trigger('click') + await nextTick() + expect(findReindexButton(wrapper).text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindexing' + ) + + emitMemoryUpdated('reindex', 'b') + await flushPromises() + expect(findReindexButton(wrapper).text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindexing' + ) + + memoryClient.getStatus.mockResolvedValueOnce({ ...status, reindexing: true }) + emitMemoryUpdated('reindex', 'a') + await flushPromises() + expect(findReindexButton(wrapper).text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindexing' + ) + + memoryClient.getStatus.mockResolvedValueOnce(status) + emitMemoryUpdated('reindex', 'a') + await flushPromises() + expect(findReindexButton(wrapper).text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindex' + ) + + reindex.resolve({ started: true }) + await flushPromises() + }) + + it('clears local reindex pending when the request returns started=false', async () => { + const { wrapper, memoryClient } = await setup({ + reindexPromise: Promise.resolve({ started: false }) + }) + + await activateHealthTab(wrapper) + await findReindexButton(wrapper).trigger('click') + await flushPromises() + + expect(memoryClient.reindex).toHaveBeenCalledWith('a') + expect(findReindexButton(wrapper).text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindex' + ) + expect(findReindexButton(wrapper).text()).not.toContain( + 'settings.deepchatAgents.memoryManager.health.reindexing' + ) + }) + + it('clears local reindex pending after a started request refreshes to reindexing=false', async () => { + const reindex = deferred<{ started: boolean }>() + const { wrapper } = await setup({ reindexPromise: reindex.promise }) + + await activateHealthTab(wrapper) + await findReindexButton(wrapper).trigger('click') + await nextTick() + expect(findReindexButton(wrapper).text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindexing' + ) + + reindex.resolve({ started: true }) + await flushPromises() + + expect(findReindexButton(wrapper).text()).toContain( + 'settings.deepchatAgents.memoryManager.health.reindex' + ) + expect(findReindexButton(wrapper).text()).not.toContain( + 'settings.deepchatAgents.memoryManager.health.reindexing' + ) + }) + it('clears the Health badge after an inactive memory update until health is refreshed', async () => { const loadedHealth = { ...health, totalRows: 987 } const refreshedHealth = { ...health, totalRows: 654 }