Merge origin/main into fix branch

Author: Widthdom

AGENT_GUIDE.md

@@ -131,8 +131,9 @@ CI watching must be bounded. Do not loop indefinitely.
 
 ## Status Contract
 
-- `status --json` and related JSON/MCP payloads currently expose the trust fields documented in `README.md` and `DEVELOPER_GUIDE.md`, including `fold_ready`, `fold_ready_reason`, `graph_table_available`, `issues_table_available`, `sql_graph_contract_ready`, `sql_graph_contract_degraded_reason`, `hotspot_family_ready`, `hotspot_family_degraded_reason`, `csharp_symbol_name_ready`, `csharp_metadata_target_ready`, `csharp_metadata_target_degraded_reason`, `indexed_head_commit`, `worktree_head_changed`, `index_writer_version`, `index_newer_than_reader`, `index_newer_than_reader_reason`, `unknown_extension_file_count`, `path_case_sensitive`, `data_dir`, `data_dir_source`, `data_dir_mode`, `mac_profile`, `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`), `hooks`, MCP-only `mcp_session`, and the `status --check`-only `stale_after_seconds` / `index_age_seconds` threshold audit fields.
-- When `fold_ready` or `csharp_metadata_target_ready` is the only degraded readiness bit, the CLI also adds `degraded_reason`, `recommended_action`, and `alternative_action`.
+- `status --json` and related JSON/MCP payloads currently expose the trust fields documented in `README.md` and `DEVELOPER_GUIDE.md`, including `fold_ready`, `fold_ready_reason`, `graph_table_available`, `issues_table_available`, `file_issues_data_current`, `migration_in_progress`, `sql_graph_contract_ready`, `sql_graph_contract_degraded_reason`, `hotspot_family_ready`, `hotspot_family_degraded_reason`, `csharp_symbol_name_ready`, `csharp_metadata_target_ready`, `csharp_metadata_target_degraded_reason`, `indexed_head_commit`, `worktree_head_changed`, `index_writer_version`, `index_newer_than_reader`, `index_newer_than_reader_reason`, `unknown_extension_file_count`, `path_case_sensitive`, `data_dir`, `data_dir_source`, `data_dir_mode`, `mac_profile`, `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`), `hooks`, MCP-only `mcp_session`, and the `status --check`-only `stale_after_seconds` / `index_age_seconds` threshold audit fields.
+- When any readiness field is degraded, the CLI adds `degraded_root_cause`, `degraded_reason`, `recommended_action`, `alternative_action`, and `readiness_degradations[]`. `degraded_root_cause` is the primary stable machine code; `readiness_degradations[]` lists every degraded field with `root_cause`, human reason, and remediation strings.
+- `issues_table_available` reports physical `file_issues` table presence only. `file_issues_data_current` reports whether the table is also stamped current for the active index generation.
 - `index_writer_version` records the `cdidx` version that last wrote to the DB (stamped into `codeindex_meta` as `cdidx_writer_version` on every full scan, update, and MCP index). `index_newer_than_reader` flips to `true` whenever any persisted numeric contract stamp in `codeindex_meta` (or unknown `PRAGMA user_version` readiness bits) exceeds the current binary's compiled maximum, so an older CLI re-opening a DB written by a newer CLI degrades loudly with an audit trail instead of silently dropping back to text-search fallbacks. `index_newer_than_reader_reason` enumerates the specific newer-than-reader stamps.
 - `status` also surfaces indexed-HEAD freshness via `indexed_head_sha`, `indexed_head_branch`, `indexed_head_timestamp`, and `commits_ahead_of_indexed_head`. They are stamped by `cdidx index` on every successful run (full scan AND partial update, distinct from `indexed_head_commit` which is full-scan only) on a best-effort basis (never blocks an otherwise-successful index) and omitted on non-git workspaces, detached HEAD (branch only), or legacy DBs created before this contract.
 - `status` also surfaces unknown-extension scan coverage via `unknown_extension_file_count`, stamped by successful full-repository index runs (`cdidx index <projectPath>` and MCP `index_project`) as the number of non-indexed files with non-empty extensions that do not map to a known language. It is omitted on legacy DBs or before a current full scan has stamped the value.

DEVELOPER_GUIDE.md

@@ -54,6 +54,7 @@ cdidx status --check --json
 cdidx search "handleRequest"
 cdidx definition UserService
 cdidx search "Handle" --project MyApp
+cdidx validate
 cdidx mcp
 ```
 
@@ -81,6 +82,15 @@ Output controls:
 Use `cdidx` when a repository will be searched repeatedly from terminals,
 scripts, CI, or AI tools. Use `rg` when you only need a one-off text scan.
 
+Help discovery:
+
+| Need | Command |
+|---|---|
+| Concise command overview | `cdidx --help` |
+| Full command, flag, and example reference | `cdidx --help-all` or `cdidx --help-extended` |
+| Shared flag reference only | `cdidx --help-flags` |
+| One command's usage line | `cdidx <command> --help` |
+
 Install choice and network notes:
 
 | Need | Use |
@@ -93,13 +103,24 @@ Install choice and network notes:
 See [DISTRIBUTION.md](DISTRIBUTION.md) for the full channel matrix and
 [isolated network install notes](USER_GUIDE.md#isolated-networks-and-proxies).
 
+### Validate
+
+Run `cdidx validate [--db <path>] [--json] [--verbose] [--kind <kind>] [--path <glob>]`
+to report indexed file issues such as replacement characters (`U+FFFD`), BOMs,
+NUL bytes, mixed line endings, UTF-16 BOMs, and likely non-UTF8 content.
+Validation findings are reported in the output and do not by themselves make
+the command fail; the command exits non-zero when the DB cannot be read or the
+command arguments are invalid. Use `--json` for machine-readable issue rows.
+
 ### Shell Completion
 
 Generate completion scripts with `cdidx --completions <bash|zsh|fish|powershell>`.
 The generated scripts complete subcommands, flags, and common flag values:
 `--lang` suggests supported languages, `--kind` suggests symbol/reference
 kinds, and path-like options such as `--db`, `--path`, and `--output` use shell
-file completion.
+file completion. Each generated script includes the `cdidx` version that
+produced it; regenerate installed completion scripts after upgrading or
+downgrading `cdidx`.
 
 ## Highlights
 
@@ -138,6 +159,7 @@ The documented `status --json` trust contract covers these fields:
 <table>
 <tbody>
 <tr><td><code>fold_ready</code></td><td><code>fold_ready_reason</code></td><td><code>graph_table_available</code></td><td><code>issues_table_available</code></td></tr>
+<tr><td><code>file_issues_data_current</code></td><td><code>migration_in_progress</code></td><td><code>degraded_root_cause</code></td><td><code>readiness_degradations</code></td></tr>
 <tr><td><code>sql_graph_contract_ready</code></td><td><code>sql_graph_contract_degraded_reason</code></td><td><code>hotspot_family_ready</code></td><td><code>hotspot_family_degraded_reason</code></td></tr>
 <tr><td><code>csharp_symbol_name_ready</code></td><td><code>csharp_metadata_target_ready</code></td><td><code>csharp_metadata_target_degraded_reason</code></td><td><code>indexed_head_commit</code></td></tr>
 <tr><td><code>worktree_head_changed</code></td><td><code>indexed_head_sha</code></td><td><code>indexed_head_branch</code></td><td><code>indexed_head_timestamp</code></td></tr>
@@ -149,6 +171,8 @@ The documented `status --json` trust contract covers these fields:
 </tbody>
 </table>
 
+When any readiness field is degraded, `degraded_root_cause` identifies the primary stable code and `readiness_degradations[]` lists every degraded field with `root_cause`, human `degraded_reason`, `recommended_action`, and `alternative_action`. `issues_table_available` reports physical table presence; use `file_issues_data_current` to decide whether `file_issues` rows are current for the index generation.
+
 For MCP `status`, `mcp_session` is session-scoped diagnostic data rather than persisted index state. It includes `log_level`, `roots`, optional `client_info`, and optional `client_capabilities`.
 
 `hotspot_family_degraded_reason` uses these values:
@@ -271,6 +295,7 @@ cdidx status --check --json
 cdidx search "handleRequest"
 cdidx definition UserService
 cdidx search "Handle" --project MyApp
+cdidx validate
 cdidx mcp
 ```
 
@@ -298,12 +323,31 @@ cdidx mcp
 ターミナル、スクリプト、CI、AI ツールから同じリポジトリを繰り返し検索する
 場合は `cdidx` が向いています。1回限りのテキスト検索には `rg` が向いています。
 
+help の探し方:
+
+| 目的 | コマンド |
+|---|---|
+| 短いコマンド概要 | `cdidx --help` |
+| 全コマンド、flag、例の完全版 | `cdidx --help-all` または `cdidx --help-extended` |
+| 共有 flag だけの一覧 | `cdidx --help-flags` |
+| 1 コマンドの usage 行 | `cdidx <command> --help` |
+
+### Validate
+
+`cdidx validate [--db <path>] [--json] [--verbose] [--kind <kind>] [--path <glob>]`
+は、index 済みファイルの replacement character (`U+FFFD`)、BOM、NUL byte、
+混在改行、UTF-16 BOM、非 UTF-8 らしい内容などを報告します。validation finding は
+出力で報告され、それ自体では command failure になりません。DB を読めない場合や
+引数が不正な場合は non-zero で終了します。機械処理には `--json` を使えます。
+
 ### シェル補完
 
 `cdidx --completions <bash|zsh|fish|powershell>` で補完スクリプトを生成できます。
 生成されたスクリプトは subcommand、flag、よく使う flag 値を補完します。
 `--lang` は対応言語、`--kind` は symbol / reference kind を提示し、`--db`、
 `--path`、`--output` など path 系 option は shell の file completion を使います。
+生成された script には生成元の `cdidx` version が含まれるため、`cdidx` の
+upgrade / downgrade 後はインストール済み補完 script を再生成してください。
 
 ## 特長
 
@@ -328,6 +372,7 @@ cdidx mcp
 <table>
 <tbody>
 <tr><td><code>fold_ready</code></td><td><code>fold_ready_reason</code></td><td><code>graph_table_available</code></td><td><code>issues_table_available</code></td></tr>
+<tr><td><code>file_issues_data_current</code></td><td><code>migration_in_progress</code></td><td><code>degraded_root_cause</code></td><td><code>readiness_degradations</code></td></tr>
 <tr><td><code>sql_graph_contract_ready</code></td><td><code>sql_graph_contract_degraded_reason</code></td><td><code>hotspot_family_ready</code></td><td><code>hotspot_family_degraded_reason</code></td></tr>
 <tr><td><code>csharp_symbol_name_ready</code></td><td><code>csharp_metadata_target_ready</code></td><td><code>csharp_metadata_target_degraded_reason</code></td><td><code>indexed_head_commit</code></td></tr>
 <tr><td><code>worktree_head_changed</code></td><td><code>indexed_head_sha</code></td><td><code>indexed_head_branch</code></td><td><code>indexed_head_timestamp</code></td></tr>
@@ -339,6 +384,8 @@ cdidx mcp
 </tbody>
 </table>
 
+readiness field のいずれかが degraded の場合、`degraded_root_cause` は primary の安定コードを示し、`readiness_degradations[]` は degraded な各 field と `root_cause`、人間向け `degraded_reason`、`recommended_action`、`alternative_action` を列挙します。`issues_table_available` は物理 table の有無を表し、`file_issues` 行が現在の index generation に対して current かどうかは `file_issues_data_current` を使って判定します。
+
 MCP `status` の `mcp_session` は永続化された index 状態ではなく、セッション単位の診断情報です。`log_level`、`roots`、任意の `client_info`、任意の `client_capabilities` を含みます。
 
 `hotspot_family_degraded_reason` は次の値を使います。

README.md

@@ -0,0 +1,16 @@
+---
+category: fixed
+issues:
+  - 1422
+affected:
+  - src/CodeIndex/Cli/ConsoleUi.cs
+  - tests/CodeIndex.Tests/ConsoleUiTests.cs
+---
+
+## English
+
+- **Per-command usage lines now show `--top` beside `--limit` (#1422)** — query commands that accept the result cap alias now document `[--limit <n>|--top <n>]`, matching the parser and shell-completion schema.
+
+## 日本語
+
+- **各 command の usage 行で `--limit` と並べて `--top` を表示するようになりました (#1422)** — result cap alias を受け付ける query command が `[--limit <n>|--top <n>]` を記載し、parser と shell-completion schema と一致します。

changelog.d/unreleased/1422.fixed.md

@@ -0,0 +1,24 @@
+---
+category: fixed
+issues:
+  - 1484
+  - 1486
+  - 1761
+  - 1931
+affected:
+  - src/CodeIndex/Cli/QueryCommandRunner.cs
+  - src/CodeIndex/Database/DbReader.FilesStatus.cs
+  - src/CodeIndex/Models/QueryResults.cs
+---
+
+## English
+
+- **Status readiness degradation now has structured guidance (#1484, #1931)** — `status --json` now reports `degraded_root_cause` and `readiness_degradations[]` for every degraded readiness field, including multi-degraded states.
+- **Status separates issue-table presence from issue-row freshness (#1486)** — `issues_table_available` now reports physical table presence, while `file_issues_data_current` reports whether validate issue rows are current for the index generation.
+- **Status identifies active migration/write windows (#1761)** — `status --json` now exposes `migration_in_progress` so clients can distinguish temporary writer activity from permanent readiness degradation.
+
+## 日本語
+
+- **status readiness degradation に構造化 guidance を追加しました (#1484, #1931)** — `status --json` は multi-degraded 状態を含むすべての degraded readiness field について `degraded_root_cause` と `readiness_degradations[]` を返すようになりました。
+- **status が issue table の存在と issue row の鮮度を分離しました (#1486)** — `issues_table_available` は物理 table の存在を表し、`file_issues_data_current` が validate issue row が現在の index generation に対して current かどうかを表します。
+- **status が active migration/write window を識別するようになりました (#1761)** — `status --json` は `migration_in_progress` を公開し、クライアントが一時的な writer activity と恒久的な readiness degradation を区別できるようになりました。

changelog.d/unreleased/1484.fixed.md

@@ -0,0 +1,17 @@
+---
+category: docs
+issues:
+  - 1694
+affected:
+  - README.md
+  - DEVELOPER_GUIDE.md
+  - src/CodeIndex/Cli/ConsoleUi.cs
+---
+
+## English
+
+- **Documented `cdidx validate` in user-facing help and guides (#1694)** — `validate` now appears in the top-level command summaries, README, and developer guidance so indexed-content validation is discoverable without reading source.
+
+## 日本語
+
+- **`cdidx validate` をユーザー向け help と guide に記載しました (#1694)** — `validate` が top-level command summary、README、developer guidance に表示されるようになり、source を読まなくても indexed content validation を見つけられます。

changelog.d/unreleased/1694.docs.md

@@ -0,0 +1,20 @@
+---
+category: changed
+issues:
+  - 1702
+affected:
+  - src/CodeIndex/Cli/ConsoleUi.cs
+  - src/CodeIndex/Cli/ProgramRunner.cs
+  - tests/CodeIndex.Tests/ConsoleUiTests.cs
+  - tests/CodeIndex.Tests/ProgramCliTests.cs
+  - README.md
+  - DEVELOPER_GUIDE.md
+---
+
+## English
+
+- **Top-level help is now brief by default (#1702)** — `cdidx --help` prints a concise overview, while `--help-all` / `--help-extended` retain the full reference and `--help-flags` prints only shared flag tables.
+
+## 日本語
+
+- **top-level help の既定表示を短くしました (#1702)** — `cdidx --help` は簡潔な概要を表示し、完全版は `--help-all` / `--help-extended`、共有 flag 一覧は `--help-flags` で確認できます。

changelog.d/unreleased/1702.changed.md

@@ -0,0 +1,17 @@
+---
+category: added
+issues:
+  - 1730
+affected:
+  - src/CodeIndex/Cli/QueryCommandRunner.cs
+  - src/CodeIndex/Database/DbContext.cs
+  - DEVELOPER_GUIDE.md
+---
+
+## English
+
+- **Added read-only database opens for query commands (#1730)** — query commands now accept `--read-only` / `--immutable` and translate normal database paths into SQLite immutable read-only URIs.
+
+## 日本語
+
+- **クエリコマンドで読み取り専用 DB オープンを追加しました (#1730)** — クエリコマンドは `--read-only` / `--immutable` を受け付け、通常の DB パスを SQLite の immutable read-only URI に変換します。

changelog.d/unreleased/1730.added.md

@@ -0,0 +1,17 @@
+---
+category: security
+issues:
+  - 1794
+affected:
+  - src/CodeIndex/Database/DbContext.cs
+  - src/CodeIndex/Models/QueryResults.cs
+  - DEVELOPER_GUIDE.md
+---
+
+## English
+
+- **Restricted SQLite database file permissions on POSIX (#1794)** — cdidx now applies `0600` to `codeindex.db` and WAL/SHM sidecars and reports `db_file_mode` in `status --json`.
+
+## 日本語
+
+- **POSIX で SQLite DB ファイル権限を制限しました (#1794)** — cdidx は `codeindex.db` と WAL/SHM sidecar に `0600` を適用し、`status --json` に `db_file_mode` を出します。

changelog.d/unreleased/1794.security.md

@@ -0,0 +1,18 @@
+---
+category: fixed
+issues:
+  - 1798
+affected:
+  - src/CodeIndex/Cli/IndexCommandRunner.cs
+  - src/CodeIndex/Database/DbContext.cs
+  - src/CodeIndex/Database/DbReader.FilesStatus.cs
+  - src/CodeIndex/Models/QueryResults.cs
+---
+
+## English
+
+- **Checkpoint WAL before read-only fallback (#1798)** — writable-open fallback now attempts `wal_checkpoint(TRUNCATE)` first and exposes fallback/checkpoint diagnostics in `status --json`.
+
+## 日本語
+
+- **読み取り専用フォールバック前に WAL checkpoint を試みるようにしました (#1798)** — writable open のフォールバック前に `wal_checkpoint(TRUNCATE)` を試行し、fallback / checkpoint 診断を `status --json` に出します。