feat(hub): Merkle content-hash change detector (Track 1, step 1)

[study8677]

First slice of incremental refresh. A standalone, side-effect-light module
that hashes the workspace at file -> module -> repo granularity and diffs
against a persisted snapshot to report which modules changed — without
trusting git state (sees uncommitted edits, survives branch switches, works
in non-git dirs).

What's here (hub/_merkle.py)

• build_tree(...) — pure: rolls per-file hashes up into module hashes and a
  root hash (deterministic, order-independent).
• build_workspace_tree(workspace) — uses the same module detection and
  file-loading as the refresh pipeline, so a "changed module" equals a unit
  refresh would regenerate. File reads only; no LLM, no network.
• diff_trees(prev, cur) — added / modified / removed modules.
• save_snapshot / load_snapshot — JSON with schema versioning; a
  missing / corrupt / stale snapshot loads as None so callers fall back to
  a full rebuild.

Why module-grained (not Cursor-style per-chunk)

Cursor re-embeds only changed files because an embedding is a context-free
function of one chunk. Antigravity's per-module knowledge is interpreted and
cross-references other modules, so the eventual cache key will be
hash(own files + dependency closure). The raw hashes built here are the
inputs that key is computed from.

Behaviour change

None. This is not wired into the refresh pipeline. The follow-up PR
consumes diff_trees to skip unchanged modules and recompute only a changed
module plus its graph impact closure (with live-source verification as the
correctness backstop).

Testing

• 12 unit tests (test_hub_merkle.py): hashing determinism/order-independence,
  diff add/modify/remove, snapshot round-trip + degradation paths, and a real
  tmp-workspace end-to-end (edit a file -> root changes -> diff flags modified).
• Hub subset green locally (85 passed).

🤖 Generated with Claude Code

[study8677]

总体评价

代码质量很高，设计清晰，架构合理：纯函数 build_tree + 独立 I/O 层 + 版本化快照是教科书级别的模式。12 个测试覆盖了正常路径和所有降级路径，且与主 pipeline 使用相同的模块检测逻辑，确保"已变更模块"语义一致。建议 Approve，有两处🟡建议值得修复，三处🟢优化可选。

---

问题清单

级别	文件 & 行号	描述	建议
🟡 建议	_merkle.py L55–66	@dataclass(frozen=True) 含 dict 字段，隐式不可哈希。frozen=True 通常意味着可哈希性（即可作 dict key / set 元素），但实际调用 hash(node) 会抛 TypeError: unhashable type: 'dict'，会让未来的维护者踩坑	若不需要哈希，改用 @dataclass(eq=True)（去掉 frozen）；若需要真正不可变，将 dict 改为 tuple[tuple[str,str],...] 或明确加 __hash__ = None 并注释
🟡 建议	_merkle.py L87–89	_hash_lines 用 "\n" 作为行分隔符，而文件路径中理论上可含 \n（部分 OS 允许）。注释已说明 _SEP="\0" 防 token 内注入，但行间分隔符漏洞未处理，理论上两条不同的 (path, hash) 组合可以产生相同的拼接串	行分隔符也改用 "\0"（hex hash 和 NUL 分隔的路径都不含 NUL），完全消除歧义
🟢 优化	_merkle.py L84–85	changed_modules 将两个已互不相交的列表（added 来自 cur-prev，modified 来自 cur∩prev）转换为 set 再取并集，有不必要的开销	改为 return sorted(self.added + self.modified) 即可，注释说明两者不相交
🟢 优化	_merkle.py L115–116	延迟导入放在函数体内，原因（避免循环导入）对读者不透明	加一行注释：# deferred to avoid circular imports at module load time
🟢 优化	test_hub_merkle.py	缺少三个边界用例：① 空模块（无文件）build_tree({"m": {}}) — 所有空模块会共享同一 hash；② 仅重命名文件（内容不变、路径改变）应注册为 modified；③ 空工作区（无可检测模块）build_workspace_tree	补充对应 test case，防止后续逻辑假设 module 至少有一个文件

---

亮点

• 纯函数 build_tree：零副作用，与 I/O 完全解耦，极易测试和推理 ✓
• _SEP = "\0" + 注释：主动防御路径注入，有文字说明设计意图 ✓
• SNAPSHOT_VERSION + load_snapshot 返回 None：版本不匹配静默降级为全量重建，健壮且对调用方透明 ✓
• 测试的降级路径覆盖：missing / bad JSON / version mismatch 三个 load_snapshot 失效路径全部有测试 ✓
• PR 描述中的设计文档：为什么是模块粒度而非 Cursor 风格的 per-chunk，解释清晰，非常有助于后续接手的人理解 ✓

---

修改示例

🟡 Issue 1 — frozen dataclass + unhashable dict（若不需要哈希，最简修法）

# 去掉 frozen=True，保留 eq（__eq__ 自动生成）
@dataclass
class ModuleNode:
    hash: str
    files: dict[str, str] = field(default_factory=dict)

@dataclass
class MerkleTree:
    root: str
    modules: dict[str, ModuleNode] = field(default_factory=dict)

若需要真正不可变且可哈希，改用 tuple 存储文件列表：

@dataclass(frozen=True)
class ModuleNode:
    hash: str
    files: tuple[tuple[str, str], ...] = field(default_factory=tuple)
    # usage: dict(node.files) to get {rel_path: hash} view

🟡 Issue 2 — 更安全的行分隔符

def _hash_lines(lines: list[str]) -> str:
    """Hash a set of ``name<SEP>hash`` lines order-independently."""
    # NUL as separator: it cannot appear in file paths or hex digests,
    # so no two distinct line-sets can produce the same joined string.
    return hashlib.sha256("\0".join(sorted(lines)).encode("utf-8")).hexdigest()

🟢 Optimization 1 — changed_modules

@property
def changed_modules(self) -> list[str]:
    """Modules whose knowledge must be (re)generated: added + modified."""
    # added and modified are disjoint by construction (cur-prev vs cur∩prev)
    return sorted(self.added + self.modified)

---

Generated by Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 860e2a9753

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Gracefully reject malformed snapshot payloads

load_snapshot promises to return None for unreadable/corrupt snapshots, but MerkleTree.from_dict assumes modules and each entry are mappings; if merkle.json parses and has the right version but wrong shape (for example "modules": [] or a module value that is not an object), this comprehension raises AttributeError/TypeError and aborts refresh instead of falling back to full rebuild. This can occur with hand-edited or partially corrupted snapshot files that remain valid JSON.

Useful? React with 👍 / 👎.