Improve search relevance and harden collector and Web workflows

MEMO.md

@@ -2,6 +2,62 @@
 
 ## Changelog Memo
 
+### 2026-10-25
+
+- [修复] 修正 CI 的时间敏感测试：摘要保留 10 秒 CPU 门槛，服务停止先等待夹具就绪；两项负对照均被拒绝，恢复后 70 项测试全过。仅测试与记录变更，新提交 hosted CI 待验证，详见 CHANGELOG。
+- [验证] Daily 采集器完成 30 分钟只读验收：平均 CPU 1.921%、最高 RSS 58.5 MiB，均达标；61 个样本中 16 个目录始终收敛。关闭空闲窗口证据缺口，持续写入与启动性能仍待验收。详见 CHANGELOG。
+- [排查] 603 个待上传条目均由隐私策略保留；275 个重试项分别为 14 个已捕获后缺失文件、6 个空 Kimi 文件和 255 个待核实 Copilot 条目。本轮未改代码、配置或清理数据；上一提交 14 项 CI 全过。
+- [部署] 菜单栏修复 `a9e59d50` 在 14 项 CI 全过后已升级 App 至 1.0.5 build 1782；安装字节、进程、IPC 与 HQ 健康读回通过，Service 保持原 PID，旧版 1779 可回滚。详见 CHANGELOG。
+- [修复] 菜单栏近期会话因旧会话恢复出现重复日期 ID；改用每组首个会话的稳定标识，保留排序与日期显示。回归先失败、修复后 46 项测试通过；真实隔离夹具旧版复现警告、新版观察 92 秒未出现。
+- [验证] Hosted UI smoke 14 项及截图对比通过；本机截图受屏幕录制权限限制，完整 UI 与回滚演练未做。详见 CHANGELOG。
+- [部署] 本机 App 已升级并打开为 1.0.5（build 1779），配套 Service 同步升级；正式签名、真实搜索／价值筛选、进程和安装文件读回通过，旧版与数据库备份保留。详见 CHANGELOG。
+- [修复] 首次启动暴露临时签名的框架加载问题，先恢复旧服务，再用 Daily 上现有 Developer ID 统一签名；已证明程序代码未变，并通过实际加载检查。当前 App 显示欢迎向导，完整 UI 和 30 分钟性能验收仍未执行。详见 CHANGELOG。
+
+### 2026-10-24
+
+- [部署] PR #496 的 `fb9538a` 经 14 项 CI 成功后，已升级 HQ 索引／Web 和 Daily 采集器；真实价值筛选、双入口健康、Web 字节匹配及采集上传进展通过。旧包与数据库备份保留，详见 CHANGELOG。
+- [未完成] 本机 App／配套 Service 因系统管理员认证超时仍保持旧版，新版 build 1779 已备妥；Daily 已运行近 3 小时，16/16 个根目录收敛且无新错误；30 分钟 CPU/RSS 仍待验收。详见 CHANGELOG。
+
+- [修复] PR #496 的两项 CI 失败同源于安全门禁仍匹配旧别名删除结构；更新精确白名单并补上身份绕过、未归属载荷及额外删除的拒绝测试。先复现失败，再通过 65 项测试及脚本/typecheck/lint；新提交的 hosted CI 待验证。详见 CHANGELOG。
+
+### 2026-10-23
+
+- [收口] 按反馈结束本批范围扩张：16 个本地提交已完成开发盘点、可靠性/Web 修复、搜索与价值可见性及测量工具；准备送审，尚未推送部署。语义替代涉及异步写入与 Web 权限，保留待决策，不为清空台账直接启用。详见 CHANGELOG。
+
+- [新增] 补齐只读数据库测量命令，显式传入路径即可输出存储／FTS 占比、tier 数量与评分分位数；6 项 CLI 测试和仓库夹具读回通过。未读取生产库、未调整评分阈值。详见 CHANGELOG。
+
+- [新增] 搜索可按高／中／低／未评分筛选，Service 与离线查询均在结果截断前过滤；短词候选上限也先过滤，避免旧高价值会话漏掉。先红 4 条 IPC 断言，再通过 193 项 App＋66 项 Service 测试。需配套升级 App/Service，未部署。详见 CHANGELOG。
+
+- [新增] 会话卡片和列表标题显示现有高／中／低价值标签，未评分不显示；复用既有评分与阈值，中英文及明暗主题组件截图检查完成，33 项测试通过。用户覆盖仍未实现，未部署。详见 CHANGELOG。
+
+- [修复] 搜索现综合全部关键词相关度，短词放首位不再吞掉后续长词排序；App 离线中文/日文/韩文检索与 Service 使用一致路由，平分按时间/ID 稳定排序。72 条行为断言先红，修复后 163 项 App＋63 项 Service 测试通过。详见 CHANGELOG。
+- [盘点] Lifecycle 3.4 的高亮、基础筛选、Service CJK 排序已有实现，台账改为部分完成；价值/生命周期筛选及加权仍单列。未推送部署、未测生产语料质量/时延，下一批核对价值分和使用信号。详见 CHANGELOG。
+
+- [排查] UI 测试隔离环境与 27 处 fail-fast 策略已存在，2 项源码约束测试通过；前台故障注入被系统认证状态阻止，测试体未执行，不能宣称行为通过。临时 bundle ID/测试已逐字节恢复，残项收窄为前台验收，继续搜索开发。详见 CHANGELOG。
+
+- [新增] Web 别名增删现与操作者审计同事务提交，审计/裁剪失败回滚；仅存公开项目标识，保留最近 10,000 条实际变更。身份由登录会话绑定，浏览器不能伪造，旧客户端回退 web-editor。79 项定向测试通过；须先升级 Service 再升级 RemoteServer，未部署。详见 CHANGELOG。
+
+- [优化] 隔离 Release 测量定位 Cursor 观察路径的 SHA-256 十六进制格式化开销，改为字节查表；2048 工作区/64 项分页中位 CPU 6.82→5.24 ms（约 -23%），文件安全检查保持不变。290 项采集/归档测试通过；这是合成样本收益，Daily CPU 目标仍未验收。详见 CHANGELOG。
+
+- [验证] `r11-dropped-2` 已用正式 Router 的可控断流/超限请求替代源码字符串检查；覆盖 503/413、MCP 错误体、失败上传不落盘/覆盖、上限成功和鉴权先于读体。旧映射负对照出现 5 条预期失败，恢复后 28 项测试全绿；未做真实网络断流或生产验收。详见 CHANGELOG，下一批测量 Cursor 观察路径成本。
+
+- [修复] 挂载真实 SessionDetailView 复现旧分页清掉新会话忙碌状态，以及任务启动前重复点击发起双请求；page/all/copy 现在同步标忙并按独立任务 token 清理，A→B→A 也不会误解锁。旧初始结果/错误与索引应用的取消守卫经验证有效。详见 `CHANGELOG.md`。
+- [验证] 观察器改为读取挂载 State 后，5 项用例连续 5 轮通过；撤掉修复的负对照出现 17 个预期失败，恢复后 72 项转录测试全绿，lint/invariants/XcodeGen drift 通过。假转录、隔离 home、复制输出注入；未推送部署。该残项本地关闭，下一项核验传输错误映射。
+- [验证] Release 的 arm64/x86_64 构建通过；Debug 动态库可见测试钩子，Release 可执行文件无对应符号。构建日志、产物哈希和 77 个本地文档链接检查见 `CHANGELOG.md`。
+- [盘点] 凭据登录全局五次/分钟限制由当前源码及 47 项认证测试确认；现行规格明确暂缓按客户端限流，且凭据模式尚无可信客户端地址契约，保留设计项，不改信任转发头。下一项为转录快速切换的挂载视图延迟回归。详见 `CHANGELOG.md`。
+- [修复] Web AI 修改地址不能继续复用服务端旧密钥：覆盖共享 Embedding 回退、旧标题字段、禁用凭据、环境变量、sidecar 与 Keychain；无法确认无凭据时拒绝整笔保存。无凭据地址编辑和普通模型/提示词编辑保留，页面明确说明未保存并允许修正草稿。详见 `CHANGELOG.md`。
+- [验证] 先复现地址重定向与页面误报，再通过 26 项 Service、24 项 HTTP/IPC/UI 路由、181 项 Web 行为测试及 lint/invariants；真实 socket 请求拒绝后文件逐字节不变。均为本地假密钥夹具，未读真实 Keychain、未推送部署。下一项核实凭据登录全局限流。
+- [修复] 持续复核发现上一批 Web actor 漏了服务总入口字段白名单：此前传输与业务方法分开通过，组合仍会被拒绝。补齐四条 IPC 白名单，把审计回归改为真实 socket→总入口→数据库链路；先复现 invalid，再通过 18 项相关测试。HTTP 仍拒绝浏览器自带 actor。详见 `CHANGELOG.md`。
+- [修复] Web 项目 move/archive/undo/batch 的审计身份从登录会话贯穿到 IPC 与 migration log；共享口令模式记为 web-editor，拒绝浏览器伪造 actor，不同 actor 不能复用同一操作 ID 的缓存结果。无 schema 迁移，原生 CLI/MCP 默认行为保留。详见 `CHANGELOG.md`。
+- [验证] HTTP 8 个、服务 8 个预期断言先失败；修复后 Web/认证/IPC 60 项、服务/注册表 29 项、底层 batch/store 34 项通过，lint/invariants 通过。尚未推送或部署；别名没有现成审计存储，单独保留设计项，Web 限流与 endpoint/key 风险仍待核实。
+- [盘点] 发现 Round-12 台账漏记后续修复：密钥文件/保存结果/测试路径隔离、导航与收藏 generation 已有实现；核对当前调用链并通过隔离 home 下 210 项 App 测试后，本地关闭这些已描述范围，未重复改生产代码。转录 apply 的快速切换场景仍未验证，单独保留。下一项为 Web 审计 actor 透传。详见 `CHANGELOG.md`。
+- [修复] blocked-root 重试改为逐根单调时钟退避（1 秒起、最高 60 秒）；冷却期间不触发捕获空转，健康根与独立上传继续，扫描完成后重置、重启后立即尝试。该 followup 本地关闭。详见 `CHANGELOG.md`。
+- [验证] 先复现 24 个行为断言失败，再通过最终 87 项运行时测试；移除调度过滤的负对照确实失败，恢复后全绿。lint/invariants 通过。尚未推送或部署；权限恢复可能等待剩余退避时间，不能据此宣称 Daily CPU 达标。下一批核实现有安全/UI 残项。
+- [盘点] 建立持续开发目标与隔离工作树 `codex/leader-loop-20261023`；盘点 21 份实施计划、24 份规格、27 份设计文档，校正 ROADMAP/TODO/followups。397 个未勾选标记不是 397 个待开发功能；产品台账现为 12 个开放/条件性方向、1 个已实现方向。完整路由见 `docs/reviews/2026-10-23-development-reconciliation.md`。
+- [修复] MCP 在 collector/replica/无效配置下不再广告不可调用的工具；正常角色保留 27 个工具。另修复 collector 测试设置文件的非原子写入与一处夹具清理遗漏。
+- [验证] MCP 先复现 8 个预期断言失败，再通过 215 项可执行测试；collector/AI 审计相关 94 项通过。lint、invariants 与 72 个文档链接检查通过；lint 保留原有 1 warning/40 infos。详见 `CHANGELOG.md`。
+- [未验证] 尚未推送、部署或跑本分支 hosted CI/UI；真实主机 CPU、HQ/M1 cutover 与历史安全/UI 残项仍分别跟进。持续目标通过当前会话推进，没有安装无人值守后台调度器。
+
 ### 2026-10-20
 
 - [部署] Web 视觉精修已上 HQ：Release/arm64 从 main `881d3ffa` 打包为 `remote-server-main-881d3ffa-20261020`（verify-only 通过），receiver 只改 `ProgramArguments[0]`，plist 已备份。第一次 bootstrap 报 `5: Input/output error` 且任务被卸下，03:36:24Z 重试成功，新 PID 79714。`/v1/health` 200，新 CSS 已在线上，调色板未变，本地渲染确认徽标、卡片和转录气泡生效。回滚：用备份 plist bootstrap，旧包保留。详见 `CHANGELOG.md`。

docs/TODO.md

@@ -3,7 +3,51 @@
 Confirmed engineering tasks. Product-level work belongs in `docs/roadmap.md`;
 verification and low-priority follow-ups belong in `docs/followups.md`.
 
-## Open
+Deployment update (2026-10-25): PR #496 runtime source `fb9538a` passed CI and
+is running on HQ index/Web, Daily Collector and the local paired Service (build
+1779). App-only follow-up `a9e59d50` passed all 14 executed CI checks and is now
+installed as Developer ID build 1782, fixing duplicate popover date-group IDs.
+Local IPC and HQ health passed; hosted UI smoke passed 14 tests and screenshot
+comparison. Full App UI remains unverified.
+The dated local-only receipts below are historical; real-host acceptance is
+limited to the checks recorded in CHANGELOG. Collector quiet-source acceptance
+passed on 2026-10-25: 61 samples/30 minutes, CPU 1.921%, RSS 58.5 MiB, 16/16 roots
+converged throughout. Active-write/startup performance and full App UI remain open.
+
+## Current execution status (2026-10-23)
+
+Fresh routing: [development reconciliation](reviews/2026-10-23-development-reconciliation.md).
+The detailed dated entries below preserve history; their presence does not
+make a closed item open again. Local baseline and live remote main were both
+`5216ff3a`. Current development is in `codex/leader-loop-20261023`.
+
+| Item | Current status | Remaining verifier |
+|---|---|---|
+| Collector CPU | Quiet-source 30-minute window passed on deployed `fb9538a`; active-write/startup acceptance remains open | 61 samples over 1800 s: mean CPU 1.921%, max RSS 58.5 MiB, all 16 roots converged and no new publications. Small CPU margin; next measure a representative active-source workload. Details in CHANGELOG and `collector-idle-turn-cost-1`. |
+| W4.1 alias reconciliation | Closed, superseded by D7(c)/D9 | Reopen only as a separately scoped old-DB user-state import. |
+| W4.5 old-receipt bootstrap | Closed, owner won't-do | Preserve retained old DB/archive; no automatic reimplementation. |
+| W4.7 epoch operator command | Implemented | Authorized HQ binary/CLI readback and real recovery; authority-file runbook remains. |
+| W6.2 real-binary gate | Closed, recorded gate exists | Hosted [39098816343](https://github.com/bbingz/engram/actions/runs/39098816343) rechecked success; not a real-host acceptance receipt. |
+| App Open Web Reader | Implemented | Set/verify each intended host's URL under host-change authorization. |
+| P2 duplicate quarantine | Implemented, deployment recorded | Real cross-machine duplicate remains unexercised. |
+| P3 install tooling | Implemented | First real-host installation/upgrade/rollback receipt. |
+| D7 legacy local rows | Closed, purge recorded | Do not rerun destructive action. |
+| `cutover-role-surface-1` | Fixed locally | RED: 8 expected catalog assertions failed. GREEN: 215 MCP executable tests passed; not pushed/deployed. |
+| Collector fixture race/leak | Fixed locally | 85 CollectorRuntime tests plus the affected publication test passed; hosted flake recurrence not measured. |
+| `collector-blocked-root-backoff-1` | Fixed locally | Final 87 runtime tests passed after behavioral RED and scheduler negative control. Per-root 1–60 second monotonic retry preserves healthy-root progress and recovery/reset/restart; no real-host CPU claim. |
+| Round-12 secrets / live navigation / favorites | Existing fixes verified locally | 210 selected App tests passed with an isolated home; source and exact coverage in the reconciliation. Transcript apply concurrency remains a narrower verification item. |
+| Web project audit actor | Fixed locally for four project commands | HTTP/auth/IPC 60, service/registry 29 and native batch/store 34 tests passed after RED. Follow-up real socket/dispatcher regression caught a missing IPC allowlist; fixed and 18 service tests passed. The separate alias audit batch below closes storage; credential throttling remains an accepted design residual. Not deployed. |
+| Web alias audit | Implemented locally | 13 Service, 46 HTTP/client/auth and 20 migration tests passed after RED. Audit and exact-pair mutation share one transaction; bounded last-10,000 history stores published identities. Upgrade Service before RemoteServer; host acceptance remains. |
+| Human keyword ranking | Fixed locally; broader Lifecycle 3.4 remains partial | All terms contribute before LIMIT; App/Service CJK routing aligned. 72 behavioral RED assertions, then 163 App and 63 Service tests passed. Source/project/time/origin/value-band facets now exist; lifecycle facets and weighting remain roadmap decisions. See the ranking contract and reconciliation. |
+| Web AI endpoint/key reuse | Fixed locally | Atomic rejection protects existing/dormant credentials across settings, env, sidecar and Keychain sources, including shared embedding fallback. 26 service, 24 HTTP/IPC/UI-route and 181 JS behavior tests passed after RED. No live Keychain or deployment acceptance. |
+| `session-detail-id-3` | Fixed/verified locally for reproduced scope | Mounted view exposed stale paging cleanup and duplicate scheduling; per-load ownership and synchronous busy state fix both. Five mounted tests passed five iterations; final-harness negative control failed 17 expected assertions; restored final 72 transcript tests passed. Initial success/error and delayed indexing cancellation also verified. No full UI/production acceptance. |
+
+Remaining historical residuals require current repro before promotion;
+remaining Web hardening and measured performance follow the bounded queue
+in the reconciliation. Product choices stay in `roadmap.md`. Production changes,
+push/merge and releases require separate authorization.
+
+## Dated engineering ledger
 
 ### Collector / central index cutover gaps (added 2026-10-02)
 
@@ -62,6 +106,11 @@ authorizes host changes.
   The comparable quiet-host 30-minute window (CPU <= 2%, RSS <= 150 MiB) still
   needs to be taken once that workload ends; record it here and in
   `CHANGELOG.md`.
+  2026-10-25: the missing quiet-source window now passed on deployed `fb9538a`,
+  PID 98680, unchanged 10 s poll: 07:16-07:46Z, 61 samples/1800 s, CPU 1.921%,
+  max RSS 58.5 MiB, all 16 roots converged throughout, no root revision changes
+  or new ACKs. This closes the quiet-source verifier only; active-write/startup
+  acceptance remains open. Evidence in CHANGELOG and the retirement checklist.
 - **Alias reconciliation (plan W4 item 1, design section 4).** Not started. The
   committer refuses any occupied proposed ID, so pre-cutover local rows, their
   insights and parent links stay in the local-ID namespace and are not visible

docs/engram-lifecycle-upgrade-plan.md

@@ -211,6 +211,13 @@ execution queue; `parked-see-roadmap` items are listed in
 
 ### 3.4 人工检索升级（enabler 1）
 
+> Current-source reconciliation (2026-10-23): the proposals below are historical.
+> FTS rank, highlighted snippets, source/project/time/origin filters and Service
+> CJK trigram ranking already exist. The local development loop implements the
+> [all-term ranking and offline CJK consistency contract](superpowers/specs/2026-10-23-keyword-ranking-design.md).
+> Value/lifecycle facets and score weighting remain open in the roadmap; native
+> sqlite-vec remains excluded by current repository instructions.
+
 **目标**：把"已为 AI 接好"的检索能力补给人，CJK 用户拿到真排序。
 
 **方案设计**：
@@ -241,6 +248,17 @@ execution queue; `parked-see-roadmap` items are listed in
 
 ### 3.6 数据模型与规模支撑（enabler 3）
 
+> Measurement tooling implemented locally (2026-10-23):
+> `python3 scripts/db/measure-database.py --database /explicit/path/index.sqlite`
+> emits aggregate JSON to stdout (redirect to the desired report file).
+> It reads one transaction with committed WAL, reports dbstat object bytes/FTS
+> share, tier counts and non-skip score histogram/p10/p33/p50/p67/p90. No default
+> user database, checkpoint, optimize or migration. `--timeout` defaults to 30s;
+> failed reads exit nonzero without partial JSON. Missing dbstat is explicit.
+> SQLite may create/use WAL coordination sidecars; for an unchanged directory,
+> use a pre-existing consistent snapshot. Fixture validation is not a current
+> production distribution or authorization to retune thresholds.
+
 **目标**：用实测数据驱动决策，加必要索引，支撑价值/归档/老化查询。
 
 **方案设计**：