Skip to content

01-architecture.md

Latest commit

 

History

History
95 lines (80 loc) · 4 KB

01-architecture.md

File metadata and controls

95 lines (80 loc) · 4 KB
Raw

Architecture

Crate Structure

crates/
├── code-index-core/        Data models, error types, SymbolStore trait
├── code-index-parser/      Tree-sitter parsing and extraction (per-language modules)
├── code-index-store/       SQLite schema, CRUD, search, bulk operations
├── code-index-watcher/     File system watching (notify 7) with debouncing
├── code-index-embeddings/  Code chunker + embedding client (feature-gated: semantic)
└── code-index-server/      CLI, indexing pipeline, JSON-RPC + MCP handlers

Data Flow

Source files
    │
    ▼
File Discovery ──── source_roots.rs (Gradle/Maven/Cargo aware)
    │                config.rs (brand/source-rule filtering)
    ▼
Tree-sitter Parse ─ per-language modules (java.rs, kotlin.rs, rust_lang.rs, go_lang.rs, cpp_lang.rs)
    │
    ▼
Extraction Passes ─ symbols, imports, calls, hierarchy, references
    │
    ▼
SQLite Storage ──── symbols, call_edges, type_relations, references, imports, file_metadata
    │
    ▼
Resolution ──────── cross-file import/call/type resolution (6-step first-match-wins)
    │
    ▼
Query Layer ─────── JSON-RPC (jsonrpsee) or MCP stdio (rmcp)

Key Design Decisions

  • Tree-sitter cursor walking (not .scm queries) for robustness across grammar versions
  • SQLite with WAL mode and r2d2 connection pool (4 connections)
  • Single-writer thread for incremental indexing (crossbeam channel)
  • Symbol diffing preserves DB IDs across re-parses so cross-file edges survive
  • Parallel initial indexing with rayon (parse) + serial DB writes
  • Bulk mode (--bulk) for large codebases: SQL batch resolution instead of per-file
  • No macro expansion — tree-sitter only, so macro-generated code is invisible
  • No stdlib indexing — only project source code

Parser Module Layout

Each supported language has up to 6 modules:

Module Purpose
{lang}_lang.rs or {lang}.rs Grammar init, top-level AST dispatch
{lang}_symbols.rs or symbols.rs Symbol extraction (classes, functions, fields, etc.)
{lang}_imports.rs or imports.rs Import statement extraction
{lang}_calls.rs or calls.rs Call graph extraction
{lang}_hierarchy.rs or hierarchy.rs Type hierarchy (extends/implements)
{lang}_references.rs or references.rs Cross-reference extraction (read/write/call/type usage)

Java and Kotlin share the base module names. Rust, Go, and C++ use prefixed names.

Server Module Layout

Module Purpose
main.rs CLI entry point, startup orchestration
config.rs CLI args (clap), source rules, brand config
indexer.rs Initial parallel indexing pipeline
incremental.rs Per-file re-index on change
resolution.rs Cross-file name resolution (imports, calls, types)
source_roots.rs Build-system-aware source root discovery
writer.rs Single-writer thread consuming file events
rpc.rs JSON-RPC endpoint handlers
rpc_lsp.rs LSP-compatible endpoints
rpc_semantic.rs Semantic search endpoints (feature-gated)
mcp_handler.rs MCP stdio tool handlers (rmcp)
handlers.rs Shared query logic used by both RPC and MCP
embedding_pipeline.rs Async embedding generation after indexing

Storage

SQLite with schema versioning (migrations in store/migrations.rs). Key tables:

  • symbols — all extracted symbols with qualified names, kinds, signatures, spans
  • call_edges — caller→callee with confidence level
  • type_relations — extends/implements edges
  • references — definition, read, write, call, type usage, import
  • imports — import statements with resolution status
  • file_metadata — indexed files with content hash and parse error count
  • pending_resolution — unresolved names awaiting future re-resolution
  • embeddings — BLOB vectors for semantic search (when enabled)
  • symbols_fts — FTS5 trigram index for text search