Skip to content

fix: propagate backfilled cwd/source_file across devices (#12) - #37

Merged
juliantanx merged 1 commit into
mainfrom
fix/issue12-backfill-sync-propagation
Jun 26, 2026
Merged

fix: propagate backfilled cwd/source_file across devices (#12)#37
juliantanx merged 1 commit into
mainfrom
fix/issue12-backfill-sync-propagation

Conversation

@juliantanx

@juliantanx juliantanx commented Jun 26, 2026

Copy link
Copy Markdown
Owner

Closes #12

问题

在运行 aiusage serve 的设备上,跨设备的项目统计看不到 Codex(以及其他依赖 cwd 的)项目:同步过来的记录既没有 cwd 也没有 source_file,导致 /api/projects 把它们整条跳过。

根本原因

不是路径推断问题,而是回填的同步传播 bug:

  • cwd 回填与 backfillHermesSourceFiles 更新了记录,但没有 bump updated_at
  • 跨设备传播只会重新上传 updated_at > synced_at 的记录,且远端 mergeRecords 只在 updatedAt 严格更新时才覆盖 → 被回填的字段永远传不到其他设备
  • 读取侧 /api/projectsif (!row.source_file) continue 丢弃空 source_file 行;codex 的源路径里没有项目信息,于是项目从列表中消失。

backfillCodexModels 本就正确 bump 了 updated_at,它是正确参照——另外两个回填是漏写 bump 的兄弟 bug。

bug 版自 v1.5.0 起已发布。已升级用户本地 cwd 早已填上,所以修正后的 WHERE cwd = '' 回填已无法匹配它们 —— 修复存量数据必须配合一次性迁移,而非可选项。

改动

  1. 回填时 bump updated_at:抽取 backfillCwd(对齐 backfillCodexModels),并在 per-file cwd 写入与 backfillHermesSourceFiles 中一并 bump。
  2. 迁移 v12:修复已运行过 bug 版的用户——把已 enriched(有 cwd 或 hermes :session: source_file)的本地记录重新标记为已变更,强制下次同步重传一次完整 wire 格式。记录 id 按设备归属,bump updated_at 只会重传本设备自己的行(last-write-wins 安全)。
  3. 新增针对回填 bump 与 v12 修复的测试。

数据流验证(全 5 链路)

# 链路 状态
1 v12 bump updated_at
2 host B getUnsyncedRecords 重新选中
3 uploadFilemergeRecords 覆盖远端 ndjson(bump 后 updatedAt 严格更大) ✅ 实际断点,bump 修复
4 host A pullinsertSyncedRecordON CONFLICT DO UPDATE ... WHERE excluded.updated_at > synced_records.updated_at 更新旧行 source_file/cwd
5 /api/projects(特定设备)直接查 synced_records → 从 cwd 提取 project

两种故障模式都被覆盖:被 getUnsyncedRecords 漏掉(已 synced),以及被选中但 mergeRecords 因 updatedAt 相等不覆盖 —— 两者都靠 bump 解决。

注意

  • 一次性全量重传:迁移会让已 enriched 的本地记录重传一次(含已正确传播的记录,属冗余但无害——远端只是被同内容 + 更新的 updatedAt 覆盖)。
  • 范围:本 PR 针对 GitHub/S3 ndjson 同步路径。用 HTTP cloud 同步的用户,建议另行确认 site 服务端 upsert 会更新 source_file/cwd
  • 「All devices」汇总视图本就只统计本地项目(既有行为,非本 PR 引入),报告者是选特定设备,不受影响。

Test plan

  • tests/commands/backfill-sync-propagation.test.ts — 回填后记录被重新标记为待同步
  • tests/db/migration-v12.test.ts — v12 只 bump enriched 记录,跳过 synced/ 与无 cwd 记录
  • 更新 schema 版本断言 11 → 12(schema/status 测试)
  • CLI 395 + core 96 全部通过;无新增类型错误
  • 报告者侧验证:host B 升级 → parse/serve(跑 v12)→ 同步 → host A 拉取后可见 Codex 项目

Cross-device project stats lost Codex (and other cwd-dependent) projects
on the device running `aiusage serve`: synced records carried no cwd and
no source_file, so /api/projects skipped them entirely.

Root cause: the cwd/source_file backfills enriched local records but did
not bump `updated_at`. Cross-device propagation only re-uploads records
where `updated_at > synced_at`, and the remote merge only overwrites when
the incoming updatedAt is strictly newer — so the enriched fields never
reached peers. backfillCodexModels already bumps updated_at; the cwd and
hermes source_file backfills were the inconsistent siblings.

- Extract the cwd backfill into backfillCwd and bump updated_at (matching
  backfillCodexModels); also bump in the per-file cwd write and in
  backfillHermesSourceFiles.
- Add migration v12 to repair users who already ran the buggy v1.5.0–v1.5.7
  backfill: their cwd is already populated locally, so the corrected
  `WHERE cwd = ''` backfill no longer matches them — the migration re-marks
  enriched records as changed to force one re-upload. It is required, not
  optional, for existing installs.
- Tests for both the backfill bump and the v12 repair.
@juliantanx
juliantanx force-pushed the fix/issue12-backfill-sync-propagation branch from 13b61e1 to aa50beb Compare June 26, 2026 07:03
@juliantanx
juliantanx merged commit 669cb17 into main Jun 26, 2026
7 checks passed
@juliantanx
juliantanx deleted the fix/issue12-backfill-sync-propagation branch July 16, 2026 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Projects 只能统计的hermes和openclaw根目录,无法统计codex与cc项目

1 participant