Merge remote-tracking branch 'origin/main' into fix-issue5161

Author: Widthdom

DEVELOPER_GUIDE.md

@@ -257,7 +257,7 @@ Set `CDIDX_SLOW_QUERY_MS=<milliseconds>` to write slow SQLite command diagnostic
 | Worker protocol JSON | Isolated worker stdin frames are read through `BoundedLineReader`. The symbol-worker client serializes requests directly to UTF-8 and writes newline-framed bytes to the process stream; the worker serializes responses directly to its stdout stream, and the client reads each bounded response frame as UTF-8 bytes for direct deserialization. This avoids an additional UTF-16 JSON string and encoding pass in each direction for every source file. The default frame cap is 32 MiB for both characters and UTF-8 bytes. When a larger `--max-file-bytes` setting needs JSON-escaping headroom, the protocol frame cap may expand up to `WorkerProtocolLineLimits.MaxExtendedLineUtf8Bytes` (384 MiB), never to `int.MaxValue`. `WorkerProtocolJsonValidator` rejects payloads over the negotiated character/UTF-8 byte cap before `JsonDocument.Parse`, parses with `DefaultMaxJsonDepth` (32), rejects more than 1,000,000 object properties, and rejects strings longer than the frame cap. |
 | User regex find | `find --regex` keeps the classic .NET regex engine for lookaround/backreference compatibility, adds `RegexOptions.CultureInvariant`, adds `IgnoreCase` unless `--exact` is set, and uses `BoundedRegex.DefaultMatchTimeout` per match. Timeouts surface as `E014_REGEX_MATCH_TIMEOUT` / `regex_timeout` in CLI JSON, and human output includes the same recovery hint. `find --all` additionally applies candidate-file and line-scan caps before walking the whole index. |
 | Shared regex construction | Production regex construction is centralized through `BoundedRegex`, `RegexRegistry`, or `RegexTimeoutPolicy`. Use `BoundedRegex` for extractor patterns and bounded static regex APIs, `RegexRegistry` for raw BCL regex factories that must preserve timeout exceptions (`find --regex`, ignore glob regexes, generated-code path patterns), and `RegexTimeoutPolicy` for diagnostic/redaction surfaces. `RegexRegistry` owns the named ignore-glob timeout (100 ms), generated-code pattern timeout (50 ms), and find-regex factory using `BoundedRegex.DefaultMatchTimeout`. Search-audit recipes treat only `BoundedRegex` aliases and `RegexRegistry.cs` as centralized positive evidence, so new production raw constructors require a deliberate factory or generated-regex entry plus tests. |
-| Filesystem traversal helpers | `FileSystemTraversalPolicy` keeps top-directory-only enumeration explicit (`IgnoreInaccessible=false`, no implicit recursion) and exposes opt-in `CancellationToken` / entry-budget options. Expected traversal failures are classified centrally so command diagnostics share the same permission, I/O, invalid-path, unsupported-path, path-too-long, and budget-exceeded taxonomy. |
+| Filesystem traversal helpers | `FileSystemTraversalPolicy` keeps top-directory-only enumeration explicit (`IgnoreInaccessible=false`, no implicit recursion) and exposes opt-in `CancellationToken` / entry-budget options. Expected traversal failures are classified centrally so command diagnostics share the same permission, I/O, invalid-path, unsupported-path, path-too-long, and budget-exceeded taxonomy. Existing-child case probes retain one exact-name set capped by `CaseSensitivityProbeDirectory.MaxExistingChildProbeEntries` (4,096), return unknown on truncation so callers use the isolated-write or cached root-policy fallback, and propagate available cancellation tokens. |
 | `MaxValue` sentinels | `int.MaxValue` may be used only as an internal sentinel when the next operation clamps before SQL limits, allocation, traversal, payload sizing, or timeout conversion. User-influenced values must be reduced to named practical constants before multiplication, buffer sizing, protocol framing, or query expansion. |
 
 ### Indexing pipeline
@@ -4323,7 +4323,7 @@ query コマンドも JSON profile block 用の `--profile` と command-scoped p
 | worker protocol JSON | isolated worker の stdin frame は `BoundedLineReader` で読みます。symbol-worker client は request を直接 UTF-8 に serialize して改行区切りの byte を process stream へ書き、worker は response を stdout stream へ直接 serialize し、client は bounded response frame を UTF-8 byte のまま読み取って直接 deserialize します。これにより source file ごとに両方向で発生していた追加 UTF-16 JSON string と encoding pass を避けます。既定の frame 上限は文字数・UTF-8 byte 数ともに 32 MiB です。大きな `--max-file-bytes` によって JSON escape 分の余裕が必要な場合、protocol frame 上限は `WorkerProtocolLineLimits.MaxExtendedLineUtf8Bytes`（384 MiB）まで拡張できますが、`int.MaxValue` までは拡張しません。`WorkerProtocolJsonValidator` は `JsonDocument.Parse` の前に合意済みの文字数 / UTF-8 byte 上限を超える payload を拒否し、`DefaultMaxJsonDepth`（32）で parse し、object property 1,000,000 件超と frame 上限を超える string を拒否します。 |
 | user regex find | `find --regex` は lookaround / backreference 互換性のため classic .NET regex engine を維持し、`RegexOptions.CultureInvariant` を付け、`--exact` でない場合は `IgnoreCase` も付け、各 match に `BoundedRegex.DefaultMatchTimeout` を使います。timeout は CLI JSON で `E014_REGEX_MATCH_TIMEOUT` / `regex_timeout` として返り、人間向け出力にも同じ recovery hint が出ます。`find --all` は index 全体を走査する前に candidate file と line scan の上限も適用します。 |
 | shared regex construction | production の regex 構築は `BoundedRegex`、`RegexRegistry`、または `RegexTimeoutPolicy` に集約します。extractor pattern と bounded static regex API には `BoundedRegex`、timeout 例外を維持する必要がある raw BCL regex factory（`find --regex`、ignore glob regex、generated-code path pattern）には `RegexRegistry`、diagnostic / redaction surface には `RegexTimeoutPolicy` を使います。`RegexRegistry` は ignore glob timeout（100 ms）、generated-code pattern timeout（50 ms）、および `BoundedRegex.DefaultMatchTimeout` を使う find-regex factory の名前付き policy を所有します。search-audit recipe は `BoundedRegex` alias と `RegexRegistry.cs` だけを集約済みの positive evidence と見なすため、新しい production raw constructor は明示的な factory または generated-regex entry とテストを伴う必要があります。 |
-| filesystem traversal helper | `FileSystemTraversalPolicy` は top-directory-only enumeration を明示し（`IgnoreInaccessible=false`、暗黙の再帰なし）、任意指定の `CancellationToken` / entry budget option を公開します。想定内の traversal failure は中央で分類し、command diagnostic が permission、I/O、invalid-path、unsupported-path、path-too-long、budget-exceeded の taxonomy を共有します。 |
+| filesystem traversal helper | `FileSystemTraversalPolicy` は top-directory-only enumeration を明示し（`IgnoreInaccessible=false`、暗黙の再帰なし）、任意指定の `CancellationToken` / entry budget option を公開します。想定内の traversal failure は中央で分類し、command diagnostic が permission、I/O、invalid-path、unsupported-path、path-too-long、budget-exceeded の taxonomy を共有します。既存 child の case probe は `CaseSensitivityProbeDirectory.MaxExistingChildProbeEntries`（4,096）を上限とする1つの exact-name set だけを保持し、truncation 時は unknown を返して caller の isolated-write または cached root-policy fallback に委ね、利用可能な cancellation token を伝播します。 |
 | `MaxValue` sentinel | `int.MaxValue` は、次の操作が SQL limit、allocation、traversal、payload sizing、timeout conversion の前に clamp する場合だけ内部 sentinel として使えます。ユーザー影響値は multiplication、buffer sizing、protocol framing、query expansion の前に、名前付きの実用上限へ落としてください。 |
 
 ### インデックスパイプライン

TESTING_GUIDE.md

@@ -193,6 +193,7 @@ Candidate-ordered parallel-index recovery tests must prove that the fatal result
   Project-marker fingerprint coverage requires the C#/VB/F#/MSBuild batch to share one directory-tree traversal while retaining one enumeration per distinct marker glob and one child-directory enumeration per visited directory. Pin known hashes and single-language delegation, independent budgets/truncation/warning order, ignore/nested-repository/submodule and platform-casing boundaries, traversal failure and cancellation propagation, and MCP fingerprint/authorization parity. Family-scope snapshot coverage must keep exact marker counts distinct until publication, then resolve only marker directories and their ancestors through the scan-cached per-directory case policy; use injected entry snapshots to cover case-sensitive children below insensitive roots and insensitive-child aliases without any post-scan filesystem probe or live marker enumeration.
   Normalized-content facts coverage compares normalization, all derived facts, chunk payloads, and validation issue order against an independent fixed-seed oracle. Keep exact UTF-16 line, unicode61 rune, normalized UTF-8 conflict-budget, replacement-line, trailing-newline, and 80/10 chunk boundaries explicit. The blocking `net8.0` allocation checks use 100,000-line inputs to prevent per-line boundary arrays from returning and require high-ratio invalid UTF-8 loads to discard replacement-line details while preserving aggregate/fallback issue parity.
 - `PathCompatibilityMatrixTests.cs`
+  Existing-child case-probe coverage keeps below-limit results compatible, proves the injectable entry cap stops after the single look-ahead needed to detect truncation, requires truncated snapshots to return unknown, and checks pre-cancellation plus cancellation during enumeration.
   Cross-platform path compatibility matrix coverage for path casing, boundary-prefix comparisons, private-child case probes (including numeric root basenames), filesystem-aware exact/prefix filename language detection, Windows long-path prefixing, POSIX sensitive-file permissions, symlink/dangling-entry scan behavior, submodule passthrough under default skip directories, and git skip-worktree path normalization. Keep new platform/path fixture scenarios here when the same assumption needs to be visible across indexing, Git helper, DB/query, installer, or status surfaces.
 - `SymbolKindCatalogTests.cs`
   Cross-language taxonomy coverage requires every declared symbol and reference kind to be unique and accepted by the exact Ordinal lookup, while null, empty, whitespace-only, case variants, trailing-space variants, and unknown values remain rejected. Keep the writer's unknown symbol/reference and container-kind diagnostics, schema/catalog parity, and pattern-sidecar invalid-kind rejection in the same focused validation set.
@@ -1315,6 +1316,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
   project marker fingerprint の coverage では、C#/VB/F#/MSBuild batch が1回の directory-tree traversal を共有しつつ、各directoryで固有marker globごとに1回、child directoryは1回だけ列挙することを必須とします。既知hashとsingle-language delegation、言語別budget/truncation/warning順、ignore/nested repository/submoduleとplatform casingの境界、traversal failure/cancellationの伝播、MCP fingerprint/authorization parityを固定してください。family-scope snapshot の coverage は、publish まで marker count を完全一致で分離し、publish 後は marker directory とその祖先だけを scan 済み directory ごとの case policy で解決する契約を維持します。case-insensitive root 配下の case-sensitive child と case-insensitive child alias は injected entry snapshot で検証し、scan 後の filesystem probe や live marker 列挙が0であることも固定してください。
   normalized-content facts の coverage は、正規化、全 derived facts、chunk payload、validation issue 順を独立した固定 seed oracle と比較します。UTF-16 line、unicode61 rune、正規化後 UTF-8 の conflict budget、replacement line、末尾改行、80/10 chunk の正確な境界を明示的に維持してください。blocking な `net8.0` allocation check は100,000行 input で行単位の境界 array が戻ることを防ぎ、高比率 invalid UTF-8 load が aggregate / fallback issue parity を保ったまま replacement-line detail を破棄することを必須とします。
 - `PathCompatibilityMatrixTests.cs`
+  既存 child の case-probe coverage では、上限未満の結果互換性、truncation 検出に必要な1件だけの look-ahead 後に停止する注入可能な entry cap、truncated snapshot が unknown を返すこと、事前 cancellation と列挙中 cancellation を固定します。
   path casing、boundary-prefix 比較、数字だけの root basename を含む private-child case probe、filesystem-aware な完全一致/prefix ファイル名言語判定、Windows long-path prefix、POSIX の sensitive file 権限、symlink / dangling entry の scan 挙動、既定 skip directory 配下の submodule passthrough、git skip-worktree path 正規化を横断する compatibility matrix カバレッジです。同じ platform/path 前提を indexing、Git helper、DB/query、installer、status の各 surface で見える形にしたい場合は、新しい fixture シナリオをここに追加してください。
 - `SymbolKindCatalogTests.cs`
   全言語共通の taxonomy coverage では、宣言済みの全 symbol / reference kind が重複せず、完全一致の Ordinal lookup で受理されることを必須とします。null、空文字、空白のみ、case 違い、末尾空白、未知の値は引き続き拒否してください。writer の未知 symbol/reference kind と container kind の診断、schema/catalog parity、pattern sidecar の invalid-kind 拒否も同じ focused validation set で維持します。

changelog.d/unreleased/5160.fixed.md

@@ -0,0 +1,21 @@
+---
+category: fixed
+issues:
+  - 5160
+affected:
+  - src/CodeIndex/Indexer/Scanning/CaseSensitivityProbeDirectory.cs
+  - src/CodeIndex/Indexer/Scanning/FileIndexer.CaseSensitivity.cs
+  - src/CodeIndex/Indexer/Scanning/FileIndexer.DirectoryEnumeration.cs
+  - src/CodeIndex/Cli/GitHelper.cs
+  - tests/CodeIndex.Tests/PathCompatibilityMatrixTests.cs
+  - DEVELOPER_GUIDE.md
+  - TESTING_GUIDE.md
+---
+
+## English
+
+- **Case-sensitivity probes now use bounded, cancelable directory traversal (#5160)** — existing-child probes retain one capped name snapshot, treat truncation as unknown so deterministic fallbacks remain authoritative, and observe available cancellation tokens.
+
+## 日本語
+
+- **大小文字区別 probe が上限付き・キャンセル可能な directory traversal を使うようになりました (#5160)** — 既存 child の probe は上限付きの name snapshot を1つだけ保持し、truncation を unknown として扱うことで deterministic fallback の authority を維持しつつ、利用可能な cancellation token を監視します。

src/CodeIndex/Cli/GitHelper.cs

@@ -1217,13 +1217,13 @@ internal static bool ResolveIgnoreCase(
     {
         var repoRoot = TryGetRepositoryRoot(projectRoot, gitEnvironmentOverrides, cancellationToken);
         if (repoRoot == null)
-            return ProbeFileSystemIgnoreCase(projectRoot);
+            return ProbeFileSystemIgnoreCase(projectRoot, cancellationToken);
 
         var configured = TryRunGit(repoRoot, gitEnvironmentOverrides, cancellationToken, "config", "--bool", "--get", "core.ignorecase")?.Trim();
         if (bool.TryParse(configured, out var ignoreCase))
             return ignoreCase;
 
-        return ProbeFileSystemIgnoreCase(projectRoot);
+        return ProbeFileSystemIgnoreCase(projectRoot, cancellationToken);
     }
 
     /// <summary>
@@ -1518,7 +1518,9 @@ private static (int ExitCode, string Output, string Error)? RunProcessCapturingO
     private static string FormatGitDiagnostic(string diagnostic) =>
         GitProcessRunner.FormatDiagnostic(diagnostic);
 
-    private static bool ProbeFileSystemIgnoreCase(string projectRoot)
+    private static bool ProbeFileSystemIgnoreCase(
+        string projectRoot,
+        CancellationToken cancellationToken)
     {
         var normalizedRoot = projectRoot;
         try
@@ -1531,7 +1533,7 @@ private static bool ProbeFileSystemIgnoreCase(string projectRoot)
             {
                 try
                 {
-                    if (CaseSensitivityProbeDirectory.ProbeExistingChildIgnoreCase(normalizedRoot) is { } existingChildIgnoreCase)
+                    if (CaseSensitivityProbeDirectory.ProbeExistingChildIgnoreCase(normalizedRoot, cancellationToken) is { } existingChildIgnoreCase)
                         return existingChildIgnoreCase;
                 }
                 catch (Exception ex) when (IsCaseSensitivityProbeFailure(ex))
@@ -1546,14 +1548,14 @@ private static bool ProbeFileSystemIgnoreCase(string projectRoot)
                     if (!string.IsNullOrEmpty(parent)
                         && !string.Equals(parent, normalizedRoot, StringComparison.Ordinal))
                     {
-                        return ProbeFileSystemIgnoreCase(parent);
+                        return ProbeFileSystemIgnoreCase(parent, cancellationToken);
                     }
 
                     throw;
                 }
             }
 
-            return CaseSensitivityProbeDirectory.ProbeIgnoreCase(normalizedRoot, "case-probe-");
+            return CaseSensitivityProbeDirectory.ProbeIgnoreCase(normalizedRoot, "case-probe-", cancellationToken);
         }
         catch (CaseSensitivityProbeException ex)
         {