Fix search snippet focus for identifiers (#4618)

Author: Widthdom

DEVELOPER_GUIDE.md

@@ -1324,6 +1324,8 @@ Exact-match flag compatibility is documented in [USER_GUIDE.md](USER_GUIDE.md#fl
 
 `search --json` and MCP `search` project full chunks into compact match-centered snippets with `chunk_start_line`, `chunk_end_line`, `snippet_start_line`, `snippet_end_line`, `snippet`, `match_lines`, `highlights`, `context_before`, `context_after`, `truncated_line_count`, `dropped_match_line_count`, and `truncation_context`. Compact CLI rows and MCP search results also echo effective output options with `snippet_lines` / `snippetLines`, `max_line_width` / `maxLineWidth`, `exact`, `raw_fts` / `rawFts`, `literal_highlights_available` / `literalHighlightsAvailable`, and optional `literal_highlight_warning` / `literalHighlightWarning`. `--snippet-lines` caps the snippet length up front (default: 8, max: 20), and `--max-line-width` (CLI) / `maxLineWidth` (MCP) clamps each individual snippet line around the first match token via the shared `LineWidthFormatter.ClampLine` contract used by `find` / `references` / `excerpt` / `inspect` (default: 512, max: 4096) so a single match inside a minified / transpiled / generated single-line file no longer returns hundreds of KB per hit. Clamped lines surface `...(+N)...` markers inside the snippet and expose `truncation_context.char_counts`, `truncation_context.total_chars`, `highlights[].truncated`, `highlights[].original_line_length`, and `highlights[].truncated_char_counts` so AI clients can detect clamping and quantify omitted characters. `highlights[].terms` remains a distinct term list for compatibility; `highlights[].term_occurrences` records every matched occurrence with `term`, 1-based `line`, 1-based `column`, `length`, plus `visible`, `visible_column`, and `visible_length` for the portion still present in the returned snippet text after line clamping. Exact substring search also adds `highlights[].literal_terms` and `highlights[].literal_term_occurrences` (camelCase in MCP) so clients can render only the requested literal phrase while preserving the broader diagnostic token list; raw FTS rows set `literal_highlight_warning` / `literalHighlightWarning` to `literal_highlights_unavailable_raw_fts` because FTS syntax can no longer be mapped to one literal phrase. Non-exact punctuation-heavy code-phrase searches add `exact_substring_hint` to CLI JSON compact results and `recovery_hint` to MCP `search` responses so clients can retry with exact substring semantics when FTS tokenization is likely to hide punctuation. `focus_mode`, `focus_line`, `focus_column`, and `focus_reason` describe the match window selected for the snippet, while `dropped_match_line_count` and optional `next_match` report match lines omitted because they fell outside that selected snippet window.
 
+Default `quality` snippet focus treats a single query made only of letters, digits, and underscores as identifier-shaped. When a result mixes matching code with earlier comments or strings, the first `code`-origin match becomes the preferred snippet line; space-delimited phrase queries and explicit `leftmost` / `proximity` focus modes retain their existing selection. The selected line remains auditable through the existing focus and origin metadata, and `next_match` continues from the selected window.
+
 When the match line falls inside an indexed symbol range, `search --json` and MCP `search` also include optional `enclosing_symbol_name`, `enclosing_symbol_kind`, `enclosing_symbol_start_line`, `enclosing_symbol_end_line`, and `enclosing_container_name`.
 
 `find --json` remains line-delimited for repeated matches and adds bounded match-span/truncation metadata to each row: `length` reports the 1-based `column` span length, `original_line_length` reports the source line length before any line-width clamp, and `snippet_truncation_context.line_count` / `char_counts` / `total_chars` / optional `reason` describe snippet clamping. `reason` is `line_width` when `--max-line-width` elides one or more snippet lines.
@@ -4191,6 +4193,8 @@ literal-safe な `search` query は reader 層で FTS5 sanitization 前に 1000
 
 `search --json` と MCP の `search` は、フルチャンクを `chunk_start_line`、`chunk_end_line`、`snippet_start_line`、`snippet_end_line`、`snippet`、`match_lines`、`highlights`、`context_before`、`context_after`、`truncated_line_count`、`dropped_match_line_count`、`truncation_context` を持つ軽量スニペットへ投影します。compact CLI row と MCP search result は有効な出力オプションも `snippet_lines` / `snippetLines`、`max_line_width` / `maxLineWidth`、`exact`、`raw_fts` / `rawFts`、`literal_highlights_available` / `literalHighlightsAvailable`、任意の `literal_highlight_warning` / `literalHighlightWarning` として返します。`--snippet-lines` で抜粋長を先に制限でき（デフォルト: 8、最大: 20）、`--max-line-width`（CLI）/ `maxLineWidth`（MCP）は `find` / `references` / `excerpt` / `inspect` と同じ共有 `LineWidthFormatter.ClampLine` 契約（デフォルト: 512、最大: 4096、`0` で切り詰め解除）で各スニペット行を最初のマッチトークン周辺にクランプするため、minified / transpiled / 生成された 1 行ファイル内の 1 ヒットで数百 KB を返さなくなります。クランプされた行はスニペットに `...(+N)...` マーカーが入り、`truncation_context.char_counts`、`truncation_context.total_chars`、`highlights[].truncated`、`highlights[].original_line_length`、`highlights[].truncated_char_counts` で AI クライアントがクランプの有無と省略文字数を検出できます。`highlights[].terms` は互換性のため distinct な term list のまま残し、`highlights[].term_occurrences` は一致ごとの `term`、1-based の `line` / `column`、`length` に加えて、行クランプ後に返却 snippet text 内へ残っている部分を示す `visible`、`visible_column`、`visible_length` を記録します。exact substring search では `highlights[].literal_terms` と `highlights[].literal_term_occurrences`（MCP では camelCase）も追加され、広めの診断 token list を残したまま、要求された literal phrase だけを render できます。raw FTS row は FTS 構文を単一の literal phrase へ対応付けられないため、`literal_highlight_warning` / `literalHighlightWarning` に `literal_highlights_unavailable_raw_fts` を設定します。exact ではない記号の多い code phrase 検索では、FTS tokenization が記号を失いやすい場合に exact substring semantics で再検索できるよう、CLI JSON compact result に `exact_substring_hint`、MCP `search` に `recovery_hint` を追加します。`focus_mode`、`focus_line`、`focus_column`、`focus_reason` は snippet に選ばれた match window を説明し、`dropped_match_line_count` と任意の `next_match` は選択された snippet window 外に落ちた一致行を示します。
 
+既定の `quality` snippet focus は、文字、数字、underscore だけで構成される単一 query を identifier 形状として扱います。一つの result に一致する code と、それより前の comment / string が混在する場合は、最初の `code` origin match を snippet の優先行にします。空白区切りの phrase query と、明示的な `leftmost` / `proximity` focus mode は従来の選択を維持します。選択行は既存の focus / origin metadata で監査でき、`next_match` は選択された window の先を引き続き示します。
+
 マッチ行がインデックス済みシンボル範囲内にある場合、`search --json` と MCP の `search` は任意フィールドの `enclosing_symbol_name`、`enclosing_symbol_kind`、`enclosing_symbol_start_line`、`enclosing_symbol_end_line`、`enclosing_container_name` も返します。
 
 `find --json` は繰り返し一致でも line-delimited のまま維持し、各 row に bounded な match span / truncation metadata を追加します。`length` は 1-based の `column` から始まる一致長、`original_line_length` は行幅クランプ前のソース行長、`snippet_truncation_context.line_count` / `char_counts` / `total_chars` / 任意の `reason` は snippet クランプを表します。`--max-line-width` によって snippet 行が省略された場合、`reason` は `line_width` になります。

TESTING_GUIDE.md

@@ -53,6 +53,7 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding
 
 - `ChunkSplitterTests.cs`, `SymbolExtractorTests.cs`, `ReferenceExtractorTests.cs`, `SearchSnippetFormatterTests.cs`, `DbPathResolverTests.cs`, `ExcerptRecoveryCommandFormatterTests.cs`, `ConsoleUiTests.cs`
   Pure or mostly pure behavior tests with in-memory inputs.
+  Search snippet origin-priority coverage keeps PascalCase, snake_case, and phrase queries in one mixed comment/string/code fixture so identifier focus and the phrase control share the same `next_match` boundary.
   Recovery-command coverage asserts structured argv, current `dotnet`/apphost prefix preservation, replay of option-like paths, and display-only escaping for both POSIX sh and PowerShell, including spaces, quotes, dollar signs, and shell metacharacters.
   Console writer synchronization coverage yields between character writes instead of sleeping per character; use enough whole-line iterations to expose interleaving without adding wall-clock delay.
 - `SymbolExtractor*Tests.cs` and `ReferenceExtractor*Tests.cs`
@@ -756,6 +757,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
 
 - `ChunkSplitterTests.cs`、`SymbolExtractorTests.cs`、`ReferenceExtractorTests.cs`、`SearchSnippetFormatterTests.cs`、`DbPathResolverTests.cs`、`ExcerptRecoveryCommandFormatterTests.cs`、`ConsoleUiTests.cs`
   インメモリ入力中心の、純粋またはほぼ純粋な振る舞いのテスト。
+  search snippet の origin 優先順位 coverage は PascalCase、snake_case、phrase query を単一の comment / string / code 混在 fixture にまとめ、identifier focus と phrase の control が同じ `next_match` 境界を共有するようにします。
   recovery command の coverage では構造化 argv、現在の `dotnet` / apphost prefix の維持、option と紛らわしい path の再実行、POSIX sh と PowerShell 双方の表示専用 escaping を、空白、quote、dollar sign、shell metacharacter を含めて検証します。
   console writer synchronization coverageは文字writeごとのsleepではなくyieldを使い、wall-clock delayを追加せずinterleavingを露出できる十分なwhole-line iterationを維持してください。
 - `SymbolExtractor*Tests.cs` と `ReferenceExtractor*Tests.cs`

changelog.d/unreleased/4618.fixed.md

@@ -0,0 +1,18 @@
+---
+category: fixed
+issues:
+  - 4618
+affected:
+  - src/CodeIndex/Cli/SearchSnippetFormatter.cs
+  - tests/CodeIndex.Tests/SearchSnippetFormatterTests.cs
+  - DEVELOPER_GUIDE.md
+  - TESTING_GUIDE.md
+---
+
+## English
+
+- **Identifier searches now focus executable code ahead of comments and strings (#4618)** — default `quality` snippets prefer the first `code`-origin match for PascalCase, snake_case, and other identifier-shaped queries while preserving phrase-query focus metadata and `next_match` navigation.
+
+## 日本語
+
+- **識別子検索は comment や string より実行 code を先に focus するようになりました (#4618)** — 既定の `quality` snippet は PascalCase、snake_case、その他の identifier 形状 query で最初の `code` origin match を優先しつつ、phrase query の focus metadata と `next_match` navigation を維持します。

src/CodeIndex/Cli/SearchSnippetFormatter.cs

@@ -78,8 +78,11 @@ public static CompactSearchResult ToCompactResult(SearchResult result, SearchSni
     {
         ArgumentNullException.ThrowIfNull(queryContext);
 
-        var excerpt = BuildExcerpt(result.Content, queryContext, result.StartLine, maxLines, caseSensitive, maxLineWidth, lang ?? result.Lang, focusMode, exposeLiteralHighlights, preferredMatchLine);
-        var matchFacets = BuildMatchFacets(result, queryContext, caseSensitive, lang ?? result.Lang, exposeLiteralHighlights);
+        var effectiveLang = lang ?? result.Lang;
+        var matchFacets = BuildMatchFacets(result, queryContext, caseSensitive, effectiveLang, exposeLiteralHighlights);
+        var effectivePreferredMatchLine = preferredMatchLine
+            ?? SelectPreferredCodeMatchLine(queryContext.ForLanguage(effectiveLang).NormalizedQuery, focusMode, matchFacets);
+        var excerpt = BuildExcerpt(result.Content, queryContext, result.StartLine, maxLines, caseSensitive, maxLineWidth, effectiveLang, focusMode, exposeLiteralHighlights, effectivePreferredMatchLine);
         AttachHighlightOrigins(excerpt.Highlights, matchFacets);
         return new CompactSearchResult
         {
@@ -131,6 +134,41 @@ public static CompactSearchResult ToCompactResult(SearchResult result, SearchSni
         };
     }
 
+    private static int? SelectPreferredCodeMatchLine(
+        string query,
+        SearchSnippetFocusMode focusMode,
+        IReadOnlyList<SearchMatchFacet> matchFacets)
+    {
+        if (focusMode != SearchSnippetFocusMode.Quality || !IsIdentifierLikeQuery(query))
+            return null;
+
+        return matchFacets
+            .Where(facet => string.Equals(facet.Origin, SearchMatchClassifier.Code, StringComparison.Ordinal))
+            .Select(facet => (int?)facet.Line)
+            .FirstOrDefault();
+    }
+
+    private static bool IsIdentifierLikeQuery(string query)
+    {
+        var value = query.Trim();
+        if (value.Length == 0)
+            return false;
+
+        if (!char.IsLetter(value[0]) && value[0] != '_')
+            return false;
+
+        for (var index = 1; index < value.Length; index++)
+        {
+            var character = value[index];
+            if (char.IsLetterOrDigit(character) || character == '_')
+                continue;
+
+            return false;
+        }
+
+        return true;
+    }
+
     private static List<SearchMatchFacet> BuildMatchFacets(SearchResult result, SearchSnippetQueryContext queryContext, bool caseSensitive, string? lang, bool exposeLiteralHighlights)
     {
         var facets = new List<SearchMatchFacet>();
@@ -375,6 +413,12 @@ public static SearchSnippetExcerpt BuildExcerpt(string content, SearchSnippetQue
             });
         }
 
+        var nextMatchIndex = matchIndexes
+            .Where(index => index > focusEnd)
+            .Select(index => (int?)index)
+            .FirstOrDefault()
+            ?? matchScan.FirstDroppedMatchIndex;
+
         return new SearchSnippetExcerpt
         {
             StartLine = absoluteStartLine + start,
@@ -392,7 +436,7 @@ public static SearchSnippetExcerpt BuildExcerpt(string content, SearchSnippetQue
             FocusLine = matchIndexes.Count > 0 ? absoluteStartLine + focusStart : null,
             FocusColumn = focusColumn,
             FocusReason = focusReason,
-            NextMatchLine = matchScan.FirstDroppedMatchIndex.HasValue ? absoluteStartLine + matchScan.FirstDroppedMatchIndex.Value : null,
+            NextMatchLine = nextMatchIndex.HasValue ? absoluteStartLine + nextMatchIndex.Value : null,
             TruncationContext = new SearchTruncationContext
             {
                 LineCount = truncatedLineCount,

tests/CodeIndex.Tests/SearchSnippetFormatterTests.cs

@@ -510,6 +510,40 @@ public void ToCompactResult_ReportsFocusAndNextDroppedMatchMetadata_Issue3556()
         Assert.Equal(2, compact.NextMatch.RemainingMatchLineCount);
     }
 
+    [Theory]
+    [InlineData("PascalCase", "PascalCase();", 13, "code", 15, 3)]
+    [InlineData("snake_case", "snake_case();", 13, "code", 15, 3)]
+    [InlineData("plain phrase", "plain phrase;", 10, "comment", 13, 2)]
+    public void ToCompactResult_IdentifierQueriesPreferCodeOriginsOverCommentsAndStrings_Issue4618(
+        string query,
+        string codeExpression,
+        int expectedFocusLine,
+        string expectedFocusOrigin,
+        int expectedNextMatchLine,
+        int expectedDroppedMatchLineCount)
+    {
+        var result = new SearchResult
+        {
+            Path = "src/app.cs",
+            Lang = "csharp",
+            StartLine = 10,
+            EndLine = 15,
+            Content = $"// {query}\nvar text = \"{query}\";\nvar padding = 0;\n{codeExpression}\nvar gap = 0;\n{codeExpression}",
+            Score = -1.0,
+        };
+
+        var compact = SearchSnippetFormatter.ToCompactResult(result, query, maxLines: 2);
+
+        Assert.Equal(expectedFocusLine, compact.FocusLine);
+        Assert.Equal("full_query", compact.FocusReason);
+        var focusHighlight = Assert.Single(compact.Highlights, highlight => highlight.Line == expectedFocusLine);
+        Assert.Equal([expectedFocusOrigin], focusHighlight.MatchOrigins);
+        Assert.Equal(expectedDroppedMatchLineCount, compact.DroppedMatchLineCount);
+        Assert.NotNull(compact.NextMatch);
+        Assert.Equal(expectedNextMatchLine, compact.NextMatch.Line);
+        Assert.Equal(expectedDroppedMatchLineCount, compact.NextMatch.RemainingMatchLineCount);
+    }
+
     [Fact]
     public void ToCompactResult_ReportsTruncationContext_WhenMultipleSnippetLinesAreClamped()
     {