[bug] Memory consolidation near-duplicate pass makes unbounded embedding calls and starves foreground vector recall

[yyhhyyyyyy]

Summary

mergeNearDuplicates (src/main/presenter/memoryPresenter/services/maintenanceService.ts:350) loops over every active memory row and calls retrieve() per row. Each retrieve fires one query-embedding API call plus an FTS query and a backfill kick. The consolidation budget (CONSOLIDATION_MAX_LLM_CALLS = 8, CONSOLIDATION_MAX_INPUT_TOKENS = 24000) only counts decision-LLM calls, which increment only after a >= 0.85 neighbor is found — a corpus with no near-duplicates is scanned end to end.

For N active memories this is up to N sequential embedding API calls + N SQL searches per LLM-tier pass (every 6h cooldown + idle triggers). Cost grows linearly with corpus size, unbounded.

Foreground interaction: while each row's query embedding is in flight, the recall hot-path dedupe (retrievalService.ts:96-101, keyed by agentId::model only) makes concurrent foreground recalls skip the vector path entirely. During a consolidation pass, user-facing memory injection degrades to FTS-only for most turns, silently.

Severity

P1 — cost/performance defect that violates the pass's own budget design and degrades the primary read path during routine background maintenance.

Fix direction

• Use stored vectors (rows are already embedded) for the near-dup neighbor scan instead of re-embedding row content, and cap scanned rows per pass with a resume cursor.
• Isolate background retrieves from the foreground query-embedding dedupe.

SDD doc: docs/issues/memory-audit-hardening/ (spec.md + plan.md + tasks.md)

[yyhhyyyyyy]

#1889