Skip to content

Commit 536e7a2

Browse files
author
SqlRush
committed
Support Grep multiline matching
1 parent 7321bd9 commit 536e7a2

5 files changed

Lines changed: 133 additions & 23 deletions

File tree

‎docs/cc-100-roadmap.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -512,7 +512,7 @@ M7 补充：prompt history `LogEntry` 读取现在接受 `sessionID`/`session`/`
512512
- 文本版 `Read`、PDF text/page-selection 初版（含常见 Page/Contents 间接对象、Pages/Kids 页序、FlateDecode 文本流和 UTF-16 BOM 字符串）、PNG/JPEG/GIF/WebP image Read、Jupyter notebook cell 渲染初版、Read 大文本 tool-result budget 截断/落盘、`Write`、`Edit` 初版已完成。
513513
- 已覆盖读前写、mtime stale guard、唯一匹配、`replace_all`、Write/Edit structured diff hunks、`.claude/settings.json`/`settings.local.json` 写前 JSON/语义校验、team-memory secret guard、Read 去重、跨 tool round read-state。
514514
- `Bash` 初版已完成，支持 command/timeout/description 输入校验、`/bin/sh -c` 执行、stdout/stderr/exit code/timeout 结构化结果、动态 read-only/concurrency-safe/destructive 分类、Git branch/tag/remote/push 参数级安全分类、`find -delete/-exec rm` 与 `xargs rm` 破坏性分类、权限规则接入、后台启动、同会话 `BashOutput` 输出读取和 `KillBash` 取消。
515-
- `Glob`/`Grep` 纯 Go 初版已完成，支持 `**` 递归 glob、基础 ignored dirs、层级 `.gitignore`/`.ignore`、mtime/path 排序、Grep regex/fixed string (`fixed_strings`/`-F`)、glob/type 过滤、`output_mode`/`outputMode` 的 `files_with_matches`/`content`/`count` 输出模式、`context`/`before_context`/`after_context` 和 `-C`/`-B`/`-A` 上下文行、`-n` line-number 控制、`offset`/`head_limit` 分页和大小写不敏感搜索。
515+
- `Glob`/`Grep` 纯 Go 初版已完成，支持 `**` 递归 glob、基础 ignored dirs、层级 `.gitignore`/`.ignore`、mtime/path 排序、Grep regex/fixed string (`fixed_strings`/`-F`)、multiline 跨行 dotall 搜索、glob/type 过滤、`output_mode`/`outputMode` 的 `files_with_matches`/`content`/`count` 输出模式、`context`/`before_context`/`after_context` 和 `-C`/`-B`/`-A` 上下文行、`-n` line-number 控制、`offset`/`head_limit` 分页和大小写不敏感搜索。
516516
- `TodoWrite` 初版已完成，支持完整 todo list 写入、状态/优先级校验、重复 id 拒绝、单个 `in_progress` 约束、结构化结果、tool metadata 状态保存和 session-scoped 本地持久化/恢复。
517517
- `WebFetch` 初版已完成，支持 URL/timeout/max_bytes 输入校验、HTTP GET、HEAD preflight、metadata/raw `skipWebFetchPreflight` skip-preflight、二进制 preflight 跳过 GET、文本/二进制判定、截断、非 2xx error 标记、结构化结果、HTML-to-text rendering、prompt-focused excerpt 和 `WebFetch(domain:...)` 权限规则适配。
518518
- `WebSearch` HTML 搜索适配初版已完成，支持 query/max_results/timeout/domain filters 输入校验、可注入搜索 endpoint、DuckDuckGo HTML 链接解析、DuckDuckGo result snippet 抽取、domain allow/block 过滤、结构化结果和 query 权限规则匹配。

‎docs/claude-code-go-rewrite-plan.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -168,7 +168,7 @@ test/parity/ # golden tests against TS/official behavior
168168

169169
- Read/Edit/Write 初版已落地，覆盖文本 Read、PDF text/page-selection 初版（含常见 Page/Contents 间接对象、Pages/Kids 页序、FlateDecode 文本流和 UTF-16 BOM 字符串）、PNG/JPEG/GIF/WebP image Read、Jupyter notebook cell 渲染初版、Read 大文本 tool-result budget 截断/落盘、read-before-write、mtime stale guard、唯一匹配、`replace_all`、Write/Edit structured diff hunks、`.claude/settings.json`/`settings.local.json` 写前 JSON/语义校验、team-memory secret guard、Read 去重和跨 tool round read-state。
170170
- Bash 初版已落地，覆盖 command/timeout/description 输入校验、`/bin/sh -c` 执行、stdout/stderr/exit code/timeout 结构化结果、动态 read-only/concurrency-safe/destructive 分类、Git branch/tag/remote/push 参数级安全分类、`find -delete/-exec rm` 与 `xargs rm` 破坏性分类、权限规则接入、后台启动、同会话 `BashOutput` 输出读取和 `KillBash` 取消；完整 shell parser、真实 sandbox、interrupt、后台任务完整生命周期和官方 golden 仍需继续补。
171-
- Glob/Grep 纯 Go 初版已落地，覆盖 `**` 递归 glob、基础 ignored dirs、层级 `.gitignore`/`.ignore`、mtime/path 排序、Grep regex/fixed string (`fixed_strings`/`-F`)、glob/type 过滤、`output_mode`/`outputMode` 的 `files_with_matches`/`content`/`count` 输出模式、`context`/`before_context`/`after_context` 及 `-C`/`-B`/`-A` 上下文行、`-n` line-number 控制、`offset`/`head_limit` 分页和大小写不敏感搜索；完整 ripgrep parity 和剩余输出参数仍需继续补。
171+
- Glob/Grep 纯 Go 初版已落地，覆盖 `**` 递归 glob、基础 ignored dirs、层级 `.gitignore`/`.ignore`、mtime/path 排序、Grep regex/fixed string (`fixed_strings`/`-F`)、multiline 跨行 dotall 搜索、glob/type 过滤、`output_mode`/`outputMode` 的 `files_with_matches`/`content`/`count` 输出模式、`context`/`before_context`/`after_context` 及 `-C`/`-B`/`-A` 上下文行、`-n` line-number 控制、`offset`/`head_limit` 分页和大小写不敏感搜索；完整 ripgrep parity 和剩余输出参数仍需继续补。
172172
- TodoWrite 初版已落地，覆盖完整 todo list 写入、状态/优先级校验、重复 id 拒绝、单个 `in_progress` 约束、结构化结果、tool metadata 状态保存和 session-scoped 本地持久化/恢复；TUI 同步和官方 golden 仍需继续补。
173173
- WebFetch 初版已落地，覆盖 URL/timeout/max_bytes 输入校验、HTTP GET、HEAD preflight、metadata/raw `skipWebFetchPreflight` skip-preflight、二进制 preflight 跳过 GET、文本/二进制判定、截断、非 2xx error 标记、结构化结果、HTML-to-text rendering、prompt-focused excerpt 和 `WebFetch(domain:...)` 权限规则适配；browser 渲染、完整 prompt-aware summarization 和官方 golden 仍需继续补。
174174
- WebSearch HTML 搜索适配初版已落地，覆盖 query/max_results/timeout/domain filters 输入校验、可注入搜索 endpoint、DuckDuckGo HTML 链接解析、DuckDuckGo result snippet 抽取、domain allow/block 过滤、结构化结果和 query 权限规则匹配；官方搜索后端、ranking parity 和 golden 仍需继续补。

‎docs/first-second-parity-audit.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ After the first/second batch hardening, the Go rewrite now includes initial `int
105105

106106
Covered behavior:
107107

108-
- `Glob`/`Grep` initial pure-Go file search tools, including recursive `**` glob matching, basic ignored directories, hierarchical `.gitignore`/`.ignore`, modified-time/path sorting, regex and fixed-string grep (`fixed_strings`/`-F`), glob/type filtering, `output_mode`/`outputMode` `files_with_matches`/`content`/`count` output modes, `context`/`before_context`/`after_context` and `-C`/`-B`/`-A` content context lines, `-n` line-number control, `offset`/`head_limit` pagination, and case-insensitive search aliases.
108+
- `Glob`/`Grep` initial pure-Go file search tools, including recursive `**` glob matching, basic ignored directories, hierarchical `.gitignore`/`.ignore`, modified-time/path sorting, regex and fixed-string grep (`fixed_strings`/`-F`), multiline dotall grep, glob/type filtering, `output_mode`/`outputMode` `files_with_matches`/`content`/`count` output modes, `context`/`before_context`/`after_context` and `-C`/`-B`/`-A` content context lines, `-n` line-number control, `offset`/`head_limit` pagination, and case-insensitive search aliases.
109109
- `Bash` initial shell execution, including command/timeout/description validation, `/bin/sh -c` execution, stdout/stderr/exit-code/timeout structured results, dynamic read-only/concurrency-safe/destructive classification, Git branch/tag/remote/push argument-aware safety classification, destructive `find -delete/-exec rm` and `xargs rm` classification, permission-rule matching through the existing tool permission adapter, background command start, same-session `BashOutput` reads, and `KillBash` cancellation.
110110
- `TodoWrite` initial todo state, including full-list writes, status/priority validation, duplicate-id rejection, one `in_progress` guard, structured result payloads, tool metadata state storage, and session-scoped local persistence/restore.
111111
- `WebFetch` initial URL fetch behavior, including URL/timeout/max-byte validation, HTTP GET, HEAD preflight, metadata/raw `skipWebFetchPreflight` skip-preflight, binary preflight GET skipping, text/binary detection, truncation, non-2xx error marking, structured result payloads, HTML-to-text rendering, prompt-focused excerpts, and `WebFetch(domain:...)` permission-rule adaptation.

‎internal/tools/file/search_tools.go‎

Lines changed: 86 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ type grepInput struct {
4747
FixedStrings bool `json:"fixed_strings,omitempty"`
4848
FixedStringsAlt bool `json:"fixedStrings,omitempty"`
4949
ShortFixedStrings bool `json:"-F,omitempty"`
50+
Multiline bool `json:"multiline,omitempty"`
5051
}
5152

5253
type fileSearchMatch struct {
@@ -70,6 +71,7 @@ type grepOptions struct {
7071
BeforeContext int
7172
AfterContext int
7273
LineNumbers bool
74+
Multiline bool
7375
}
7476

7577
func NewGlobTool() tool.Tool {
@@ -143,11 +145,12 @@ func NewGrepTool() tool.Tool {
143145
"fixed_strings": map[string]any{"type": "boolean"},
144146
"fixedStrings": map[string]any{"type": "boolean"},
145147
"-F": map[string]any{"type": "boolean"},
148+
"multiline": map[string]any{"type": "boolean"},
146149
},
147150
},
148151
},
149152
PromptFunc: func(tool.PromptContext) (string, error) {
150-
return "Searches text files under path using a regular expression or fixed string. output_mode may be files_with_matches, content, or count; glob and type optionally filter file paths. content mode supports context, before_context, after_context, -C, -B, -A, -n line-number control, offset, and head_limit pagination. Use fixed_strings or -F for literal matching.", nil
153+
return "Searches text files under path using a regular expression or fixed string. output_mode may be files_with_matches, content, or count; glob and type optionally filter file paths. content mode supports context, before_context, after_context, -C, -B, -A, -n line-number control, offset, and head_limit pagination. Use fixed_strings or -F for literal matching. Set multiline to allow patterns to span lines with dot matching newlines.", nil
151154
},
152155
ValidateFunc: validateGrep,
153156
CallFunc: callGrep,
@@ -270,6 +273,7 @@ func callGrep(ctx tool.Context, raw json.RawMessage, _ tool.ProgressSink) (contr
270273
BeforeContext: before,
271274
AfterContext: after,
272275
LineNumbers: grepLineNumbers(input, mode),
276+
Multiline: input.Multiline,
273277
}
274278
matches, totalMatches, truncated, err := collectGrepMatches(root, input.Glob, input.Type, expr, options)
275279
if err != nil {
@@ -297,6 +301,7 @@ func callGrep(ctx tool.Context, raw json.RawMessage, _ tool.ProgressSink) (contr
297301
"line_numbers": options.LineNumbers,
298302
"case_insensitive": grepCaseInsensitive(input),
299303
"fixed_strings": grepFixedStrings(input),
304+
"multiline": input.Multiline,
300305
"truncated": truncated,
301306
},
302307
}, nil
@@ -317,7 +322,7 @@ func decodeGrep(raw json.RawMessage) (grepInput, error) {
317322
"head_limit": {}, "headLimit": {}, "offset": {},
318323
"context": {}, "-C": {}, "before_context": {}, "beforeContext": {}, "-B": {}, "after_context": {}, "afterContext": {}, "-A": {}, "-n": {},
319324
"case_insensitive": {}, "caseInsensitive": {}, "-i": {},
320-
"fixed_strings": {}, "fixedStrings": {}, "-F": {},
325+
"fixed_strings": {}, "fixedStrings": {}, "-F": {}, "multiline": {},
321326
}, &input); err != nil {
322327
return grepInput{}, err
323328
}
@@ -390,7 +395,7 @@ func collectGrepMatches(root string, glob string, typeFilter string, expr *regex
390395
if err != nil {
391396
return nil
392397
}
393-
lineMatches := grepFileMatches(rel, content, expr, options.BeforeContext, options.AfterContext)
398+
lineMatches := grepFileMatches(rel, content, expr, options)
394399
if len(lineMatches) == 0 {
395400
return nil
396401
}
@@ -427,25 +432,14 @@ func collectGrepMatches(root string, glob string, typeFilter string, expr *regex
427432
return matches, totalMatches, truncated, nil
428433
}
429434

430-
func grepFileMatches(path string, content string, expr *regexp.Regexp, beforeContext int, afterContext int) []grepMatch {
435+
func grepFileMatches(path string, content string, expr *regexp.Regexp, options grepOptions) []grepMatch {
431436
lines := strings.Split(strings.ReplaceAll(content, "\r\n", "\n"), "\n")
432437
matched := map[int]bool{}
433438
included := map[int]bool{}
434-
for i, line := range lines {
435-
if expr.MatchString(line) {
436-
matched[i] = true
437-
start := i - beforeContext
438-
if start < 0 {
439-
start = 0
440-
}
441-
end := i + afterContext
442-
if end >= len(lines) {
443-
end = len(lines) - 1
444-
}
445-
for n := start; n <= end; n++ {
446-
included[n] = true
447-
}
448-
}
439+
if options.Multiline {
440+
markMultilineMatches(lines, strings.ReplaceAll(content, "\r\n", "\n"), expr, options.BeforeContext, options.AfterContext, matched, included)
441+
} else {
442+
markLineMatches(lines, expr, options.BeforeContext, options.AfterContext, matched, included)
449443
}
450444
matches := make([]grepMatch, 0, len(included))
451445
for i := range lines {
@@ -457,6 +451,73 @@ func grepFileMatches(path string, content string, expr *regexp.Regexp, beforeCon
457451
return matches
458452
}
459453

454+
func markLineMatches(lines []string, expr *regexp.Regexp, beforeContext int, afterContext int, matched map[int]bool, included map[int]bool) {
455+
for i, line := range lines {
456+
if expr.MatchString(line) {
457+
markGrepLineRange(i, i, len(lines), beforeContext, afterContext, matched, included)
458+
}
459+
}
460+
}
461+
462+
func markMultilineMatches(lines []string, content string, expr *regexp.Regexp, beforeContext int, afterContext int, matched map[int]bool, included map[int]bool) {
463+
if content == "" {
464+
return
465+
}
466+
starts := make([]int, len(lines))
467+
offset := 0
468+
for i, line := range lines {
469+
starts[i] = offset
470+
offset += len(line)
471+
if i < len(lines)-1 {
472+
offset++
473+
}
474+
}
475+
for _, span := range expr.FindAllStringIndex(content, -1) {
476+
first := -1
477+
last := -1
478+
for i, lineStart := range starts {
479+
lineEnd := lineStart + len(lines[i])
480+
lineSpanEnd := lineEnd
481+
if i < len(lines)-1 {
482+
lineSpanEnd++
483+
}
484+
if grepSpanTouchesLine(span[0], span[1], lineStart, lineEnd, lineSpanEnd) {
485+
if first == -1 {
486+
first = i
487+
}
488+
last = i
489+
}
490+
}
491+
if first >= 0 {
492+
markGrepLineRange(first, last, len(lines), beforeContext, afterContext, matched, included)
493+
}
494+
}
495+
}
496+
497+
func grepSpanTouchesLine(matchStart int, matchEnd int, lineStart int, lineEnd int, lineSpanEnd int) bool {
498+
if matchStart == matchEnd {
499+
return matchStart >= lineStart && matchStart <= lineEnd
500+
}
501+
return matchStart < lineSpanEnd && matchEnd > lineStart
502+
}
503+
504+
func markGrepLineRange(first int, last int, lineCount int, beforeContext int, afterContext int, matched map[int]bool, included map[int]bool) {
505+
for i := first; i <= last; i++ {
506+
matched[i] = true
507+
}
508+
start := first - beforeContext
509+
if start < 0 {
510+
start = 0
511+
}
512+
end := last + afterContext
513+
if end >= lineCount {
514+
end = lineCount - 1
515+
}
516+
for n := start; n <= end; n++ {
517+
included[n] = true
518+
}
519+
}
520+
460521
func countMatchedLines(matches []grepMatch) int {
461522
count := 0
462523
for _, match := range matches {
@@ -523,8 +584,13 @@ func compileGrepPattern(input grepInput) (*regexp.Regexp, error) {
523584
if grepFixedStrings(input) {
524585
pattern = regexp.QuoteMeta(pattern)
525586
}
526-
if grepCaseInsensitive(input) {
587+
switch {
588+
case grepCaseInsensitive(input) && input.Multiline:
589+
pattern = "(?is:" + pattern + ")"
590+
case grepCaseInsensitive(input):
527591
pattern = "(?i:" + pattern + ")"
592+
case input.Multiline:
593+
pattern = "(?s:" + pattern + ")"
528594
}
529595
return regexp.Compile(pattern)
530596
}

‎internal/tools/file/tools_test.go‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1002,6 +1002,50 @@ func TestGrepToolFixedStrings(t *testing.T) {
10021002
}
10031003
}
10041004

1005+
func TestGrepToolMultiline(t *testing.T) {
1006+
dir := t.TempDir()
1007+
content := strings.Join([]string{
1008+
"alpha start",
1009+
"middle",
1010+
"gamma end",
1011+
"tail",
1012+
}, "\n")
1013+
if err := os.WriteFile(filepath.Join(dir, "multi.txt"), []byte(content), 0o644); err != nil {
1014+
t.Fatal(err)
1015+
}
1016+
executor := fileExecutor(t)
1017+
ctx := fileToolContext(dir)
1018+
1019+
defaultResult, err := executor.Execute(ctx, contracts.ToolUse{
1020+
ID: "toolu_grep_default_single_line",
1021+
Name: "Grep",
1022+
Input: json.RawMessage(`{"pattern":"alpha.*gamma","output_mode":"content"}`),
1023+
}, nil)
1024+
if err != nil {
1025+
t.Fatal(err)
1026+
}
1027+
if defaultResult.Content != "No matches found" || defaultResult.StructuredContent["multiline"] != false {
1028+
t.Fatalf("default single-line result = %#v", defaultResult)
1029+
}
1030+
1031+
multilineResult, err := executor.Execute(ctx, contracts.ToolUse{
1032+
ID: "toolu_grep_multiline",
1033+
Name: "Grep",
1034+
Input: json.RawMessage(`{"pattern":"alpha.*gamma","output_mode":"content","multiline":true}`),
1035+
}, nil)
1036+
if err != nil {
1037+
t.Fatal(err)
1038+
}
1039+
want := "multi.txt:1:alpha start\nmulti.txt:2:middle\nmulti.txt:3:gamma end"
1040+
if multilineResult.Content != want || multilineResult.StructuredContent["multiline"] != true {
1041+
t.Fatalf("multiline result = %#v", multilineResult)
1042+
}
1043+
matches := multilineResult.StructuredContent["matches"].([]map[string]any)
1044+
if len(matches) != 3 || matches[0]["matched"] != true || matches[2]["line"] != 3 {
1045+
t.Fatalf("multiline structured matches = %#v", matches)
1046+
}
1047+
}
1048+
10051049
func TestGrepToolTypeFilter(t *testing.T) {
10061050
dir := t.TempDir()
10071051
if err := os.MkdirAll(filepath.Join(dir, "src"), 0o755); err != nil {

0 commit comments

Comments
 (0)