Skip to content

Commit b1e8bf4

Browse files
authored
Fix indexing diagnostics and freshness edge cases (#3634)
* Fix worktree head freshness flag (#3367) * Surface best-effort index metadata diagnostics (#3546) * Report skipped byte-size estimates (#3474) * Diagnose language-map read failures (#3472) * Fail explicitly on case probe errors (#3439) * Surface marker and submodule traversal warnings (#3473) * Handle symlink indexing permission cases (#3515) * Avoid duplicate marker traversal warnings (#3473) * Align worktree head freshness tests (#3367)
1 parent 614e2e3 commit b1e8bf4

37 files changed

Lines changed: 1427 additions & 173 deletions

AGENT_GUIDE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -136,14 +136,14 @@ CI watching must be bounded. Do not loop indefinitely.
136136
- `hotspot_family_degraded_reason` currently uses `hotspot_family_support_not_indexed`, `hotspot_family_metadata_stale`, `hotspot_family_disabled_at_index_time`, `partial_family_key_population`, and `hotspot_family_marker_fingerprint_incomplete`; the incomplete marker fingerprint code means marker traversal hit safety caps and should stay synchronized with README / developer-guide recovery notes.
137137
- `issues_table_available` reports physical `file_issues` table presence only. `file_issues_data_current` reports whether the table is also stamped current for the active index generation.
138138
- `index_writer_version` records the `cdidx` version that last wrote to the DB (stamped into `codeindex_meta` as `cdidx_writer_version` on every full scan, update, and MCP index). `index_newer_than_reader` flips to `true` whenever any persisted numeric contract stamp in `codeindex_meta` (or unknown `PRAGMA user_version` readiness bits) exceeds the current binary's compiled maximum, so an older CLI re-opening a DB written by a newer CLI degrades loudly with an audit trail instead of silently dropping back to text-search fallbacks. `index_newer_than_reader_reason` enumerates the specific newer-than-reader stamps.
139-
- `status` also surfaces indexed-HEAD freshness via `indexed_head_sha`, `indexed_head_branch`, `indexed_head_timestamp`, and `commits_ahead_of_indexed_head`. They are stamped by `cdidx index` on every successful run (full scan AND partial update, distinct from `indexed_head_commit` which is full-scan only) on a best-effort basis (never blocks an otherwise-successful index) and omitted on non-git workspaces, detached HEAD (branch only), or legacy DBs created before this contract.
139+
- `status` also surfaces indexed-HEAD freshness via `indexed_head_sha`, `indexed_head_branch`, `indexed_head_timestamp`, and `commits_ahead_of_indexed_head`. They are stamped by `cdidx index` on every successful run (full scan AND partial update, distinct from `indexed_head_commit` which is full-scan only) on a best-effort basis (never blocks an otherwise-successful index) and omitted on non-git workspaces, detached HEAD (branch only), or legacy DBs created before this contract. `worktree_head_changed` compares runtime HEAD with this latest stamp when available and falls back to `indexed_head_commit` only for legacy DBs.
140140
- `status` also surfaces unknown-extension scan coverage via `unknown_extension_file_count`, stamped by successful full-repository index runs (`cdidx index <projectPath>` and MCP `index_project`) as the number of non-indexed files with non-empty extensions that do not map to a known language. Current scans also stamp `unknown_extension_files` as a path sample bounded by `unknown_extension_file_path_limit` items and the string-list decoded-character budget, `unknown_extension_files_truncated` when more paths existed than were emitted for either bound, and `unknown_extension_file_path_limit` as the item cap rather than a guarantee that that many paths are returned. Newer scans also expose `unknown_extension_extension_counts`, `unknown_extension_category_counts`, and `unknown_extension_groups`; groups classify common non-code buckets such as repository metadata, licenses, binary assets, configuration, structural metadata, and language-support candidates, and include `recommended_action` values of `ignore_configuration`, `first_class_structural_extraction`, or `language_support`. These fields are omitted on legacy DBs or before a current full scan has stamped them.
141141
- `status` also surfaces extractor plugin and pattern-config runtime diagnostics via `extractors`, including loaded counts, skipped file counts, and a bounded diagnostics list for incompatible or malformed plugin/pattern files. Diagnostic paths, categories, and messages are sanitized before output; `diagnostics[].category` is the stable machine-readable failure code.
142142
- `status` also surfaces metadata-only post-extraction hook candidates and callback budgets through `hooks[]` / `hooks[].callback_budget_ms` without loading hook assemblies. `hook_diagnostics[]` reports sanitized hook discovery and callback diagnostics such as candidate-limit truncation, assembly load failure, constructor failure, callback failure, and timeout. `hook_diagnostics[].category` is the stable machine-readable failure code. Index runs still enforce `CDIDX_HOOK_CALLBACK_BUDGET_MS` (default: 5000 ms) on scratch copies, discard timed-out mutations, and disable timed-out hooks for the remainder of the current run.
143143
- `status` also surfaces `.cdidx` data-directory permissions via `data_dir_mode` on POSIX filesystems. New `.cdidx` data directories are forced to `0700`; the field is omitted on Windows, URI DBs, or when the directory mode cannot be inspected.
144144
- `status` also surfaces filesystem case-sensitivity via `path_case_sensitive`, stamped on every successful `cdidx index` run (full scan AND partial update, plus MCP-driven indexes) from `core.ignorecase` + a live filesystem probe. `true` means the volume is case-sensitive (`Foo.cs` and `foo.cs` are distinct); `false` means case-insensitive. Omitted on legacy DBs that predate the stamp. Use it to audit path-equality decisions on case-sensitive APFS, WSL NTFS / dev-drive, and ReFS mounts where the prior OS-keyed heuristic could mis-classify the workspace (#1546).
145145
- `status` also surfaces Linux mandatory-access-control context via `mac_profile` when `/proc/self/attr/current` or `/proc/self/attr/exec` indicates an AppArmor or SELinux profile. If proc attribute reads fail on Linux, `mac_profile_diagnostics[]` reports bounded `path`, `category`, and `message` entries so users can distinguish "no profile" from "profile detection failed" (#1768, #3480).
146-
- `status` also surfaces DB/WAL size, per-language symbol-kind histograms, current process heap/GC/working-set metrics, and the last successful index run metadata. `process` is captured at status-call time; `last_index_run` is persisted at the end of successful CLI and MCP index runs and can include a peak-memory summary when CLI `--memory-trace` was used. `last_workspace_freshened_at` is the latest successful index/update timestamp and can be newer than `indexed_at` when a partial or no-op update confirms freshness without rewriting indexed file rows.
146+
- `status` also surfaces DB/WAL size, per-language symbol-kind histograms, current process heap/GC/working-set metrics, and the last successful index run metadata. `process` is captured at status-call time; `last_index_run` is persisted at the end of successful CLI and MCP index runs and can include a peak-memory summary when CLI `--memory-trace` was used. `last_index_run.bytes_read_skipped_file_count` and `bytes_read_incomplete` report whether unreadable files were omitted from the `bytes_read` total. `last_index_run.diagnostics`, `diagnostic_count`, and `diagnostics_truncated` carry bounded warnings for best-effort index metadata writes that failed after the index data itself was successfully written. `last_workspace_freshened_at` is the latest successful index/update timestamp and can be newer than `indexed_at` when a partial or no-op update confirms freshness without rewriting indexed file rows.
147147
- MCP `status` also surfaces session diagnostics via `mcp_session`. It is not persisted DB state; it includes the current `log_level`, bounded captured `roots`, optional `client_info`, and bounded optional `client_capabilities`. When advertised roots are capped, `roots_truncated`, `root_count`, `root_limit`, and `root_uri_length_limit` describe the truncation. When client capabilities are capped, `client_capabilities_truncated`, `client_capabilities_truncation_reason`, `client_capabilities_serialized_bytes`, `client_capabilities_byte_limit`, and `client_capabilities_depth_limit` describe the retained diagnostic subset.
148148
- Keep `README.md`, `DEVELOPER_GUIDE.md`, and this file synchronized if this contract changes.
149149

0 commit comments

Comments
 (0)