Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions TESTING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ Candidate-ordered parallel-index recovery tests must prove that the fatal result
- `SymbolExtractor*Tests.cs` and `ReferenceExtractor*Tests.cs`
Extractor coverage is split by language or feature area with partial test classes, while shared helpers remain on the root `SymbolExtractorTests` / `ReferenceExtractorTests` parts.
YAML sequence-hierarchy fixtures must keep sequence items path-only, count empty/scalar/alias items in stable indexes, preserve anchor and block-scalar boundaries, assert logical qualified paths separately from nearest emitted symbol parents, and prove a normal full scan refreshes unchanged YAML rows stamped with the prior extractor contract.
C# declaration-boundary regressions should pair a direct extractor fixture with a real-index `symbols --exact-name` query. Keep invocation and parameter continuations beside valid multi-line methods, constructors, delegates, and local functions so both false-positive rejection and declaration ranges remain observable.
C# declaration-boundary regressions should pair a direct extractor fixture with a real-index `symbols --exact-name` query. Keep invocation and parameter continuations beside valid multi-line methods, constructors, delegates, and local functions so both false-positive rejection and declaration ranges remain observable. Scope-continuation fixtures must also place unmatched delimiters in character/string literals and comments before later methods, then assert that the later methods remain definitions rather than unresolved declaration-shaped calls.
C# explicit-interface identity coverage pairs extractor assertions for methods, properties, events, and indexers with persisted exact qualified/unqualified queries, fold rewrite validation, inspect/outline checks, and LSP definition/reference scoping. Include multiple and inherited interfaces, generic arity, a same-named public member, and a qualified return type that must not be mistaken for an explicit-interface qualifier.
C# callable-containment fixtures should cover block-bodied test methods, local and nested local functions, named lambdas, expression-bodied members, and nested types together, asserting both symbol parents and call-reference containers.
C# switch-expression returned-lambda coverage pairs a direct `ReferenceExtractorCSharpTests` regression with an `IndexCommandRunnerFullScanTests` `--rebuild` graph fixture. Keep expression/block, parenthesized, multiline, nested-switch, async, qualified/unqualified, and non-lambda arms together; assert raw call rows, enclosing containers, exact locations, callers/callees, complete graph readiness, and negative positional-pattern/enum controls including guarded, nested, and contextual-keyword-designation patterns with a following-line arrow (#5085).
Expand Down Expand Up @@ -1242,7 +1242,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
- `SymbolExtractor*Tests.cs` と `ReferenceExtractor*Tests.cs`
extractor のカバレッジは言語または機能領域ごとの partial test class に分割し、共有 helper は root 側の `SymbolExtractorTests` / `ReferenceExtractorTests` に残します。
YAML sequence hierarchy の fixture では、sequence item を path 専用のまま保ち、empty / scalar / alias item を安定した index に数え、anchor と block scalar の境界を維持し、論理 qualified path と最寄りの出力済み symbol parent を分けて検証し、旧 extractor contract が stamp された未変更 YAML row を通常の full scan が更新することも確認してください。
C# の declaration boundary に関する regression では、extractor を直接呼ぶ fixture と、実 index に対する `symbols --exact-name` query を組み合わせてください。呼び出し・parameter の continuation と、正当な複数行 method、constructor、delegate、local function を同居させ、false positive の拒否と宣言 range の両方を観測可能にします。
C# の declaration boundary に関する regression では、extractor を直接呼ぶ fixture と、実 index に対する `symbols --exact-name` query を組み合わせてください。呼び出し・parameter の continuation と、正当な複数行 method、constructor、delegate、local function を同居させ、false positive の拒否と宣言 range の両方を観測可能にします。スコープ継続用のテストでは、後続メソッドより前の文字・文字列リテラルとコメントに対応しない区切り記号も置き、後続メソッドが未解決の宣言形式呼び出しではなく定義のままになることも検証してください。
C# の明示的 interface identity coverage では、method、property、event、indexer の extractor assertion と、永続化後の修飾 / 非修飾の完全一致 query、fold rewrite 検証、inspect / outline、LSP の definition / reference scope を組み合わせます。複数および継承 interface、generic arity、同名 public member に加え、明示的 interface qualifier と誤認してはならない修飾 return type を含めてください。
C# generic outline 表示の coverage は、generic / 非 generic overload、extension / local / explicit-interface method、nested generic の tuple / nullable / array parameter type、constraint、overload を区別する modifier、修飾型 / 具体型名との衝突、escaped / Unicode identifier、attribute / 既定値 literal 内の区切り文字を、抽出と reader setup を共有する1つの `DbReaderTests` fixture に維持します。line 移動を検証する対の fixture は先頭改行以外を同一にし、canonical name / path が不変であることを assertion に含め、旧 index の `Name@line` fallback 用に不完全 signature を手動永続化した row も1件残してください(#4916)。
C# の位置 record に対する検索帰属 coverage は、extractor の座標 assertion、1つの indexed reader fixture、1つの CLI fixture を組み合わせます。record keyword と型名、attribute と component 型、base 引数、body member、同一行の通常 property、複数行 `record struct` component、正規化された verbatim identifier の列、言語未指定の exact grouping、multi-hit の focus 選択、focus の行・列出力、symbol grouping を共通契約に維持します。component 列を持たない旧 symbol では行ベースの fallback を維持してください。C# の座標維持のために comment padding を変更する場合、Java/Kotlin の record component comment が既存 signature を維持することも assertion してください(#5095)。
Expand Down
17 changes: 17 additions & 0 deletions changelog.d/unreleased/5182.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
category: fixed
issues:
- 5182
affected:
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs
---

## English

- **C# function extraction now continues after delimiters inside literals and comments (#5182)** — C# declaration scope scans now reuse the full C# lexer's raw-column-aligned masking, preventing character literals, strings, and comments from suppressing later methods. Existing C# files are automatically re-extracted through the updated extractor contract.

## 日本語

- **C# の関数抽出がリテラルやコメント内の区切り記号より後も継続するよう修正しました (#5182)** — C# の宣言スコープ走査で、完全な C# 字句解析器が生成する元の列位置に対応したマスクを再利用するようにし、文字リテラル・文字列・コメントによって後続メソッドが抽出されなくなる問題を防ぎました。更新した抽出契約により、既存の C# ファイルも自動的に再抽出されます。
20 changes: 14 additions & 6 deletions src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3909,26 +3909,34 @@ private static bool ShouldSkipCSharpSwitchExpressionPropertyCandidate(
&& csharpSwitchExpressionLines[lineIndex];

private static string[] BuildCSharpMatchLines(
string[] structuralLines,
string[] rawLines,
IReadOnlyList<SymbolPattern> applicablePatterns,
bool applyRequiredLiteralMatchInputGate,
RequiredLiteralGateCounts? requiredLiteralGateCounts,
bool applyCSharpRegexProbeOptimizations,
CSharpRegexProbeCounts? csharpRegexProbeCounts,
out int[]?[] collapsedToRaw)
out int[]?[] collapsedToRaw,
out string[] scopeLines)
{
var matchLines = new string[structuralLines.Length];
collapsedToRaw = new int[]?[structuralLines.Length];
var matchLines = new string[rawLines.Length];
collapsedToRaw = new int[]?[rawLines.Length];
scopeLines = new string[rawLines.Length];
var csharpLexState = new CSharpLexState();
var inLeadingAttributeBlock = false;
var attributeBracketDepth = 0;
var attributeParenDepth = 0;
var pendingEnumDeclaration = false;
var activeEnumBodyDepth = 0;
for (int lineIndex = 0; lineIndex < structuralLines.Length; lineIndex++)
for (int lineIndex = 0; lineIndex < rawLines.Length; lineIndex++)
{
var lexedLine = LexCSharpLine(structuralLines[lineIndex], csharpLexState);
var lexedLine = LexCSharpLine(rawLines[lineIndex], csharpLexState);
csharpLexState = lexedLine.EndState;
// Scope scans need the full C# lexer's literal/comment masking while preserving
// raw columns; generic structural masking does not understand every C# literal.
// scope scan では raw column を保った完全な C# lexer の literal/comment
// masking が必要であり、汎用 structural masking だけでは全 C# literal を扱えない。
scopeLines[lineIndex] = BlankCSharpStringDelimitersForCrossLineScan(
lexedLine.SanitizedLine);
matchLines[lineIndex] = CollapseCSharpGenericTypeWhitespace(
StripLeadingCSharpAttributeLists(
lexedLine.SanitizedLine,
Expand Down
2 changes: 1 addition & 1 deletion src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ public static partial class SymbolExtractor
public const int ExpandedLanguageContractVersion = 2;
public const int YamlContractVersion = 3;
public const int PythonContractVersion = 2;
public const int CSharpContractVersion = 13;
public const int CSharpContractVersion = 14;
public const int DockerfileContractVersion = 2;
public const int MakefileContractVersion = 2;
public const int StyleAndXamlContractVersion = 2;
Expand Down
16 changes: 10 additions & 6 deletions src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractionPhases.cs
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ public PatternScanInputs(
: null;

int[]?[] csharpMatchColumnToRaw = null!;
string[]? csharpScopeLines = null;
CSharpMatchLines = lang == "csharp"
? BuildCSharpMatchLines(
lines,
Expand All @@ -87,9 +88,11 @@ public PatternScanInputs(
requiredLiteralGateCounts,
applyCSharpRegexProbeOptimizations,
csharpRegexProbeCounts,
out csharpMatchColumnToRaw)
out csharpMatchColumnToRaw,
out csharpScopeLines)
: null;
CSharpMatchColumnToRaw = csharpMatchColumnToRaw;
CSharpScopeLines = csharpScopeLines;
GetCSharpLineStartStates = lang == "csharp"
? BuildCSharpLineStartStates
: null;
Expand All @@ -116,6 +119,7 @@ public PatternScanInputs(
public bool[]? PowershellEnumBodyLines { get; }
public int[]?[] CSharpMatchColumnToRaw { get; }
public string[]? CSharpMatchLines { get; }
public string[]? CSharpScopeLines { get; }
public Func<CSharpLexState[]>? GetCSharpLineStartStates { get; }
public Func<JavaScriptScopePrivacyFlags[][]>? GetPrivateScopeColumns { get; }
public Func<bool[]?>? GetCSharpSwitchExpressionLines { get; }
Expand All @@ -128,16 +132,16 @@ public DartClassBodyScope GetDartInsideClassBody() =>
_dartInsideClassBody ??= BuildDartClassBodyScope(StructuralLines);

public CSharpTypeBodyScope GetCSharpInsideTypeBody() =>
_csharpInsideTypeBody ??= BuildCSharpTypeBodyScope(StructuralLines);
_csharpInsideTypeBody ??= BuildCSharpTypeBodyScope(CSharpScopeLines!);

public CSharpCallableParameterScope GetCSharpCallableParameterScope() =>
_csharpCallableParameterScope ??= BuildCSharpCallableParameterScope(
StructuralLines,
CSharpScopeLines!,
GetCSharpInsideTypeBody());

public CSharpDeclarationStartScope GetCSharpDeclarationStartScope() =>
_csharpDeclarationStartScope ??= BuildCSharpDeclarationStartScope(
StructuralLines,
CSharpScopeLines!,
GetCSharpInsideTypeBody());

private CSharpLexState[] BuildCSharpLineStartStates() =>
Expand Down Expand Up @@ -165,8 +169,8 @@ private JavaScriptScopePrivacyFlags[][] BuildPrivateScopeColumns()
if (!_csharpSwitchExpressionLinesInitialized)
{
_csharpSwitchExpressionLinesInitialized = true;
_csharpSwitchExpressionLines = LinesContain(StructuralLines, "switch", StringComparison.Ordinal)
? FindCSharpSwitchExpressionLines(StructuralLines)
_csharpSwitchExpressionLines = LinesContain(CSharpScopeLines!, "switch", StringComparison.Ordinal)
? FindCSharpSwitchExpressionLines(CSharpScopeLines!)
: null;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,8 +171,8 @@ private static bool TrySkipCSharpBracePropertyCandidate(
// interface / record / enum). Used to gate the plain-field pattern so that
// local variable declarations inside a method, property accessor, lambda, or
// other non-type body are not misclassified as the public `field` kind. The scan uses
// `structuralLines` (strings / chars / comments already masked), so it is not
// fooled by braces or type-declaration-looking text inside literals. Only
// the raw-column-aligned C# scope lines produced by the full C# lexer, so it is
// not fooled by braces or type-declaration-looking text inside literals. Only
// brace-delimited types push a type-body frame — `new { ... }`, collection
// initializers, and lambda bodies all carry the `class|struct|interface|record|enum`
// keyword absent from the preceding buffer, so they correctly stay non-type.
Expand All @@ -181,8 +181,8 @@ private static bool TrySkipCSharpBracePropertyCandidate(
// 直下にあるか」を行単位で事前計算する。新しい通常フィールド抽出パターンが
// メソッド本体・プロパティアクセサ・ラムダなど「非型本体」に含まれる
// ローカル変数宣言を公開 kind `field` として誤抽出しないよう、このフラグで
// ゲートする。走査は既に文字列・文字・コメントを空白化した
// `structuralLines` を使うため、リテラル内の `{` や `class` 相当の文字列に
// ゲートする。走査は完全な C# lexer が生成した、raw column と対応する
// C# scope line を使うため、リテラル内の `{` や `class` 相当の文字列に
// 騙されない。`new { ... }` や collection initializer、ラムダ本体の `{` は
// 直前バッファに `class|struct|interface|record|enum` を含まないため
// 非型本体として扱われる。Closes #298 の codex レビュー blocker 対応。
Expand Down
4 changes: 2 additions & 2 deletions tests/CodeIndex.Tests/IndexCommandRunnerFullScanTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6283,7 +6283,7 @@ FROM symbol_references
SymbolExtractor.CSharpContractVersion.ToString(
System.Globalization.CultureInfo.InvariantCulture),
versionCmd.ExecuteScalar() as string);
Assert.Equal(13, SymbolExtractor.CSharpContractVersion);
Assert.Equal(14, SymbolExtractor.CSharpContractVersion);
}
finally
{
Expand Down Expand Up @@ -6549,7 +6549,7 @@ ORDER BY symbol_name
SymbolExtractor.CSharpContractVersion.ToString(
System.Globalization.CultureInfo.InvariantCulture),
versionCmd.ExecuteScalar() as string);
Assert.Equal(13, SymbolExtractor.CSharpContractVersion);
Assert.Equal(14, SymbolExtractor.CSharpContractVersion);
}
finally
{
Expand Down
62 changes: 62 additions & 0 deletions tests/CodeIndex.Tests/QueryCommandRunnerIssue5182Tests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
using CodeIndex.Cli;
using static CodeIndex.Tests.QueryCommandTestSupport;

namespace CodeIndex.Tests;

public partial class QueryCommandRunnerTests
{
[Fact]
public void RunSymbols_CSharpExactNameFindsMethodsAfterLiteralDelimiter_Issue5182()
{
var projectRoot = TestProjectHelper.CreateTempProject("cdidx_symbols_csharp_literal_delimiter_5182");
try
{
Directory.CreateDirectory(Path.Combine(projectRoot, "src"));
File.WriteAllText(
Path.Combine(projectRoot, "src", "Tail.cs"),
"""
internal static class DbDebugExtensions
{
public static void Trigger(string text)
{
var end = 0;
while (end < text.Length && text[end] != '(')
end++;
}

public static void ExecuteTrackedReader(this object command) { }
public static bool TrackedRead(this object reader) => true;
}
""");

var dbPath = Path.Combine(projectRoot, ".cdidx", "codeindex.db");
var (indexExitCode, _, indexStderr) = CaptureConsole(() => IndexCommandRunner.Run(
[projectRoot, "--json", "--quiet"],
_jsonOptions));

Assert.Equal(CommandExitCodes.Success, indexExitCode);
Assert.Equal(string.Empty, indexStderr);

AssertExactFunction("ExecuteTrackedReader");
AssertExactFunction("TrackedRead");

void AssertExactFunction(string name)
{
var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols(
[name, "--db", dbPath, "--json", "--exact-name", "--lang", "csharp"],
_jsonOptions));
var row = Assert.Single(ParseJsonLines(stdout));

Assert.Equal(CommandExitCodes.Success, exitCode);
Assert.Equal(string.Empty, stderr);
Assert.Equal(name, row.RootElement.GetProperty("name").GetString());
Assert.Equal("function", row.RootElement.GetProperty("kind").GetString());
Assert.Equal("DbDebugExtensions", row.RootElement.GetProperty("container_name").GetString());
}
}
finally
{
TestProjectHelper.DeleteDirectory(projectRoot);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ public void WithDefault(int value =
}

[Fact]
public void CSharpPhysicalInputNegativePrefix_RequiresContiguousNonTimeoutFailures()
public void CSharpPhysicalInputNegativePrefix_RequiresContiguousNonTimeoutFailures_Issue5182()
{
var cache = new SymbolExtractor.CSharpPhysicalInputNegativePrefixCache();
var timeoutRegex = new BoundedRegex("(a+)+$", default, TimeSpan.FromMilliseconds(1));
Expand Down
Loading
Loading