regex_code_search

A fast code search engine written in Rust. Indexes a directory using an n-gram inverted index, then supports interactive literal and regex queries.

How it works

1. Indexing — walks the target directory (respecting .gitignore), extracts 3-grams and 4-grams from every line of every file, and builds an in-memory inverted index.
2. Querying — for a literal query, candidate lines are narrowed via n-gram intersection before an exact match. For a regex query, a literal prefix is extracted from the pattern where possible to use the same fast path; otherwise it falls back to a full scan.

Usage

cargo run <directory>

Example:

cargo run ~/kaamkaaj/regex_code_search

Output after indexing:

Indexing directory: "~/kaamkaaj/regex_code_search"

=== Index Built Successfully ===
Time taken: 12.34ms
Files: 11, Lines: 842, N-grams: 9321

=== Interactive Search ===
Enter search queries (or 'quit' to exit):
  literal: <text>  - Literal string search
  regex: <pattern> - Regular expression search

>

Search commands

Input	Behavior
literal: fn search	Exact substring match for fn search
regex: fn\s+\w+	Regex match
any text	Defaults to literal search
quit / exit	Exit the program

Example session:

>> literal: fn search_literal

Found 2 results (0.31ms)

1. src/search.rs:23
   21:
   22:     pub fn search_literal(&self, query: &str) -> Vec<SearchResult> {
>  23:         let ngrams = extract_query_ngrams(query);
   24:
   25:         if ngrams.is_empty() {


>> regex: pub fn \w+\(

Found 18 results (0.87ms)

1. src/search.rs:19
   17:
   18:     pub fn new(index: &'a InvertedIndex) -> Self {
>  19:         Self { index }
   20:     }
   21:

2. src/search.rs:23
   21:
   22:     pub fn search_literal(&self, query: &str) -> Vec<SearchResult> {
>  23:         let ngrams = extract_query_ngrams(query);
   24:
   25:         if ngrams.is_empty() {

3. src/search.rs:47
   45:
   46:     pub fn search_regex(&self, pattern: &str) -> Result<Vec<SearchResult>, String> {
>  47:         let re = Regex::new(pattern).map_err(|e| format!("invalid regex pattern: {}", e))?;
   48:
   49:         let literal = extract_literal_from_regex(pattern);

... and 15 more results

> quit
Goodbye!

Building

cargo build --release

Run the release binary directly:

./target/release/regex_code_search <directory>

Dependencies

• walkdir / ignore — directory traversal with .gitignore support
• regex — regex matching and literal extraction
• rayon — parallel n-gram extraction
• fxhash — fast hashing for the index
• memmap2 — memory-mapped file I/O