fix: address dense index review findings

docs/en/wikg-standard.md

@@ -36,7 +36,7 @@ Only the following archive entries are part of the standard layout:
 | `cover/data.bin`               | No       | Binary          | Cover binary payload. Required when `cover/info.json` is present. |
 | `texts/source/<serialId>.txt`  | No       | UTF-8 text      | Source text stream for one chapter serial.                        |
 | `texts/summary/<serialId>.txt` | No       | UTF-8 text      | Summary text stream for one chapter serial.                       |
-| `index.db`                     | No       | SQLite database | Embedded full-text search index.                                  |
+| `index.db`                     | No       | SQLite database | Embedded search index with FTS records, Dense segments, or both.  |
 
 No other entry is currently standard. Examples of non-standard entries include
 SQLite journal files, arbitrary JSON sidecars, and text files outside
@@ -186,7 +186,8 @@ grounding layer.
 
 ### `index.db`
 
-`index.db` is an optional SQLite full-text search index.
+`index.db` is an optional SQLite search index. It may contain FTS records, Dense
+embedding segments, or both.
 
 Writers include `index.db` only when the archive index policy marks the search
 index as embedded. Otherwise, the search index may exist as a local cache and

docs/schema-upgrade.md

@@ -37,14 +37,15 @@ write the target schema marker.
 ## Archive Gate
 
 Archive schema belongs to each `.wikg` file. The archive gate runs at archive
-open/upgrade boundaries and covers archive entries such as `database.db` and an
-embedded `index.db`. It must not be reimplemented across query/list/search/evidence
-business paths.
+open/upgrade boundaries and covers archive entries such as `database.db`, an
+embedded `index.db`, and legacy embedded `fts.db`. It must not be reimplemented
+across query/list/search/evidence business paths.
 
-The v1 -> v2 archive upgrader removes the embedded archive `index.db` as derived
-search index data and preserves important archive content and the mutation token.
-It must refuse active coordinator state for the target archive and non-`index.db`
-overlays, because those can represent uncommitted important data.
+The v1 -> v2 archive upgrader removes embedded archive `index.db` and legacy
+`fts.db` as derived search index data and preserves important archive content
+and the mutation token. It must refuse active coordinator state for the target
+archive and non-search-index overlays, because those can represent uncommitted
+important data.
 
 ## Home Gate
 
@@ -79,7 +80,8 @@ code and tests when adding new home SQLite files:
 
 For v1 -> v2, derived home data is deleted or invalidated: query/search caches,
 external cache, GC state, build queue SQLite/cache when safe, library aggregate
-indexes, external archive search index overlays/workspaces for `index.db`, and
+indexes, external archive search index overlays/workspaces for `index.db` or
+legacy `fts.db`, and
 orphaned SQLite materialization cache overlays whose archive file no longer
 exists. The upgrader must block when active GC, build job, worker lease,
 coordinator owner/lock/sqlite lease/commit lock, or remaining non-derived

docs/zh-CN/wikg-standard.md

@@ -18,17 +18,17 @@
 
 只有以下归档 entry 属于标准布局：
 
-| Entry                          | Required | Type            | Meaning                                         |
-| ------------------------------ | -------- | --------------- | ----------------------------------------------- |
-| `.wikg-mutation-token`         | Yes      | UTF-8 text      | 归档 mutation token。必须是第一个 ZIP entry。   |
-| `manifest.json`                | Yes      | JSON            | 归档格式 manifest。                             |
-| `database.db`                  | Yes      | SQLite database | 主文档、图谱、元数据和 readiness 数据库。       |
-| `toc.json`                     | No       | JSON            | 章节树。                                        |
-| `cover/info.json`              | No       | JSON            | 封面元数据。                                    |
-| `cover/data.bin`               | No       | Binary          | 封面二进制内容。存在 `cover/info.json` 时必需。 |
-| `texts/source/<serialId>.txt`  | No       | UTF-8 text      | 某个 chapter serial 的 source text stream。     |
-| `texts/summary/<serialId>.txt` | No       | UTF-8 text      | 某个 chapter serial 的 summary text stream。    |
-| `index.db`                     | No       | SQLite database | 内嵌全文搜索索引。                              |
+| Entry                          | Required | Type            | Meaning                                                 |
+| ------------------------------ | -------- | --------------- | ------------------------------------------------------- |
+| `.wikg-mutation-token`         | Yes      | UTF-8 text      | 归档 mutation token。必须是第一个 ZIP entry。           |
+| `manifest.json`                | Yes      | JSON            | 归档格式 manifest。                                     |
+| `database.db`                  | Yes      | SQLite database | 主文档、图谱、元数据和 readiness 数据库。               |
+| `toc.json`                     | No       | JSON            | 章节树。                                                |
+| `cover/info.json`              | No       | JSON            | 封面元数据。                                            |
+| `cover/data.bin`               | No       | Binary          | 封面二进制内容。存在 `cover/info.json` 时必需。         |
+| `texts/source/<serialId>.txt`  | No       | UTF-8 text      | 某个 chapter serial 的 source text stream。             |
+| `texts/summary/<serialId>.txt` | No       | UTF-8 text      | 某个 chapter serial 的 summary text stream。            |
+| `index.db`                     | No       | SQLite database | 内嵌搜索索引，包含 FTS records、Dense segments 或两者。 |
 
 当前没有其他标准 entry。非标准 entry 的例子包括 SQLite journal 文件、任意 JSON sidecar，以及 `texts/source/` 或 `texts/summary/` 之外的文本文件。
 
@@ -152,9 +152,9 @@ Summaries 是生成投影。它们不会替代 source text 作为 grounding laye
 
 ### `index.db`
 
-`index.db` 是可选的 SQLite 全文搜索索引。
+`index.db` 是可选的 SQLite 搜索索引。它可以包含 FTS records、Dense embedding segments，或同时包含两者。
 
-只有当归档 index policy 标记搜索索引为 embedded 时，writer 才会包含 `index.db`。否则，搜索索引可以作为本地 cache 存在，并且不得被视为必需归档内容。
+只有当归档 index policy 标记搜索索引为 embedded 时，writer 才会包含 `index.db`。否则，搜索索引可以作为本地 cache 存在，并且不得被视为必需的归档内容。
 
 Reader 必须能打开不包含 `index.db` 的归档。缺失或过期的搜索索引状态应作为 readiness 信息处理，而不是归档损坏。
 

packages/cli/src/commands/archive-command/search-index.ts

@@ -153,6 +153,11 @@ async function isRequestedSearchIndexAlreadyCurrent(
     return (
       capabilities.indexes === "dense" &&
       capabilities.dense.current &&
+      capabilities.dense.model === options.embeddingProvider.model &&
+      isDenseIdentityCurrent(
+        capabilities,
+        options.embeddingProvider.identity,
+      ) &&
       (options.embeddingProvider.dimensions === undefined ||
         capabilities.dense.dimensions === options.embeddingProvider.dimensions)
     );
@@ -162,11 +167,19 @@ async function isRequestedSearchIndexAlreadyCurrent(
     capabilities.indexes === "fts,dense" &&
     capabilities.dense.current &&
     capabilities.dense.model === options.embeddingProvider.model &&
+    isDenseIdentityCurrent(capabilities, options.embeddingProvider.identity) &&
     (options.embeddingProvider.dimensions === undefined ||
       capabilities.dense.dimensions === options.embeddingProvider.dimensions)
   );
 }
 
+function isDenseIdentityCurrent(
+  capabilities: Awaited<ReturnType<typeof readSearchIndexCapabilityStatus>>,
+  identity: string | undefined,
+): boolean {
+  return identity === undefined || capabilities.dense.identity === identity;
+}
+
 async function createSearchIndexBuildOptions(
   indexes: CLIArchiveIndexArguments["indexes"],
 ): Promise<SearchIndexBuildOptions> {

packages/cli/src/runtime/config.test.ts

@@ -39,4 +39,23 @@ describe("runtime/config", () => {
       await rm(tempDir, { force: true, recursive: true });
     }
   });
+
+  it("ignores incomplete embedding config in general CLI config", async () => {
+    const tempDir = await mkdtemp(join(tmpdir(), "wikigraph-config-test-"));
+
+    try {
+      await withWikiGraphStateDirectoryPathForTesting(tempDir, async () => {
+        await putLocalConfigValue(
+          "embeddings",
+          "provider",
+          "openai-compatible",
+        );
+        await putLocalConfigValue("embeddings", "dimensions", 1536);
+
+        await expect(loadCLIConfig()).resolves.not.toHaveProperty("embedding");
+      });
+    } finally {
+      await rm(tempDir, { force: true, recursive: true });
+    }
+  });
 });

packages/cli/src/runtime/config.ts

@@ -127,12 +127,10 @@ function createEmbeddingConfig(
   const name = readString(value.name);
 
   if (
-    apiKey === undefined &&
-    baseURL === undefined &&
-    dimensions === undefined &&
-    model === undefined &&
-    name === undefined &&
-    provider === undefined
+    provider === undefined ||
+    model === undefined ||
+    (provider === "openai-compatible" && baseURL === undefined) ||
+    (provider === "openai" && baseURL !== undefined)
   ) {
     return undefined;
   }

packages/cli/src/runtime/embedding.ts

@@ -62,7 +62,6 @@ export async function embedQueryText(
   const embeddingModel = createEmbeddingModel(provider, model, config);
   const providerOptions = createEmbeddingProviderOptions(provider, config);
   const result = await embed({
-    maxRetries: 0,
     model: embeddingModel,
     ...(providerOptions === undefined ? {} : { providerOptions }),
     value: normalized,
@@ -91,14 +90,14 @@ export function buildSearchIndexEmbeddingProvider(
     ...(config.dimensions === undefined
       ? {}
       : { dimensions: config.dimensions }),
+    identity: createEmbeddingIdentity(provider, model, config),
     model,
     embedTexts: async (texts) => {
       const embeddings: number[][] = [];
       let tokens = 0;
 
       for (const batch of chunkTexts(texts, EMBEDDING_BATCH_SIZE)) {
         const result = await embedMany({
-          maxRetries: 0,
           model: embeddingModel,
           ...(providerOptions === undefined ? {} : { providerOptions }),
           values: [...batch],
@@ -115,6 +114,21 @@ export function buildSearchIndexEmbeddingProvider(
   };
 }
 
+function createEmbeddingIdentity(
+  provider: CLIEmbeddingProvider,
+  model: string,
+  config: CLIEmbeddingConfig,
+): string {
+  return JSON.stringify({
+    ...(config.baseURL === undefined ? {} : { baseURL: config.baseURL }),
+    ...(config.dimensions === undefined
+      ? {}
+      : { dimensions: config.dimensions }),
+    model,
+    provider,
+  });
+}
+
 function* chunkTexts(
   texts: readonly string[],
   size: number,

packages/cli/src/runtime/local-config.ts

@@ -116,17 +116,7 @@ export function normalizeLocalConfigKey(
     throw new Error("Config key cannot be empty.");
   }
 
-  if (section === "llm") {
-    switch (normalized) {
-      case "api-key":
-        return "apiKey";
-      case "base-url":
-        return "baseURL";
-      default:
-        return normalized;
-    }
-  }
-  if (section === "embeddings") {
+  if (section === "llm" || section === "embeddings") {
     switch (normalized) {
       case "api-key":
         return "apiKey";

packages/core/data/help/commands/predicate.jinja

@@ -448,7 +448,8 @@ Notes:
   - This switches the index storage policy to the `.wikg` archive and ensures the embedded index is current.
   - After embedding, CLI archive writes keep the embedded index synchronized automatically.
   - This makes the archive more portable, but increases archive size.
-  - Use it when distributing a `.wikg` with query support ready for another machine.
+  - Use it when distributing a `.wikg` with index materialization ready for another machine.
+  - If the index includes Dense, configure `wikg://local/config/embeddings` on the destination before Dense queries. Hybrid queries can fall back to FTS.
 {% endif %}
 {% elif predicate == "external" %}
 {% if target.name == "index-object" and libraryContext.isLibraryWide %}

packages/core/data/help/topics/readiness.jinja

@@ -138,8 +138,8 @@ WikiSpine readiness:
 How to choose:
   - Need searchable retrieval on this machine: enable the archive index.
   - Need searchable retrieval across a managed folder: enable the library index.
-  - Need local-only indexing with no provider calls: use `--indexes fts`.
-  - Need semantic retrieval: configure embeddings, then use `--indexes auto`, `dense`, or `fts,dense`.
+  - Need local-only indexing with no provider calls: use `{{ commandName }} <archive-uri>/index enable --indexes fts`.
+  - Need semantic retrieval: configure embeddings, then use `{{ commandName }} <archive-uri>/index enable --indexes auto|dense|fts,dense`.
   - Need to send a more self-contained `.wikg`: consider embedding the index.
   - Need to intentionally drop query support or index materialization: disable index.
   - Need generated Reading Graph or summaries: configure LLM and use local jobs.