Skip to content

roadmap.md

Latest commit

 

History

History
120 lines (87 loc) · 14.1 KB

roadmap.md

File metadata and controls

120 lines (87 loc) · 14.1 KB
Raw

llammer Roadmap — Planned / Not Yet Implemented

This roadmap is the honest, load-bearing companion to the rest of the documentation. llammer v0.1.0 ships a working correction and RAG core, but it also carries a fair amount of scaffolding: flags that are parsed and ignored, config toggles that build a pipeline but add no layer, and whole modules (a wire protocol, a second RAG API, alternate color schemes) that no live code path reaches. This document classifies every such item so nothing is mistaken for a finished feature. When a doc elsewhere marks something Planned, this is where it is accounted for.

llammer feature status map: Implemented / Partial / Planned

Status vocabulary

The same three-word vocabulary is used here and in the project README status matrix; the colours match the feature-status diagram above.

  • ● Implemented (green) — runs today on the primary code path.
  • ◐ Partial (amber) — present and reachable, but a flag/toggle is accepted-and-ignored or wired to a stub, so it has no real effect yet.
  • ○ Planned (grey) — code or configuration exists but is not connected to any live path.

Each Partial and Planned entry below records four things: what it is, its current state, the code that already exists (so a contributor knows where to start), and the wiring it needs.

● Implemented — what runs today

Capability Where it lives
correct — single string and batch (--input/--output), text/json/tsv output src/cli/commands/correct.rs + llammer-pipeline
Damerau–Levenshtein candidate generation (adjacent transpositions cost 1) liblevenshtein Transducer::with_transposition, via llammer-pipeline
WFST lattice construction + Viterbi decode over the tropical (min-plus) semiring lling-llang, via llammer-pipeline/src/lattice.rs
Phonetic rescoring layer (on by default) PhoneticRescoreLayer, gated by the phonetic-rescore feature
Case-preserving, punctuation-aware, Unicode-safe tokenizer llammer-pipeline/src/tokenizer.rs
repl — standalone / agent / customer roles (local, single-process) src/repl/{base,standalone,agent,customer}.rs
REPL slash-commands + conversation history (/save / /load JSON) src/repl/mod.rs, Conversation
rag init / query / extend — build, search, and grow a RagIndex src/cli/commands/rag.rs + libgrammstein
rag query --correct-query — spell-correct the query before embedding LatticeCorrectionPipelineExt::from_app_config
Conversation context, language tags (library APIs) llammer-pipeline (context, language)

◐ Partial — present but not fully wired

Item What it is Current state Code that exists Wiring it needs
--neural-rescore / enable_lm_rerank CLI flag + config toggle meant to enable neural/LM candidate rescoring Accepted; correct sets enable_lm_rerank = true, which builds a LayerPipeline but adds no layer (// LM rerank layer would be added here) — a complete no-op CorrectArgs.neural_rescore (src/cli/args.rs); mapping in src/cli/commands/correct.rs; the enable_lm_rerank branch in llammer-pipeline/src/lattice.rs Implement an LM-rerank Layer and add it when the toggle is set (see Planned → Language-model rerank layer)
enable_cfg_filter Config toggle to prune ungrammatical paths with a context-free grammar Off by default; when set, builds the pipeline but adds no layer (// CFG filter layer would be added here) The enable_cfg_filter branch in llammer-pipeline/src/lattice.rs Implement a CFG-filter Layer (see Planned → CFG-filter layer)
Context-aware correction (LatticeContextualCorrector) Correction that blends conversation history into scoring The library type works, but it only reweights confidence (confidence = confidence·(1−w) + avg_context_match·w); it does not change which candidate is chosen, and the CLI/REPL never call it (REPL correction is stateless) llammer-pipeline/src/contextual.rs; re-export in src/pipeline/mod.rs Call correct_with_context from the REPL (see Planned → Context-aware correction in the REPL)
TOML configuration A llammer.toml for tuning correction, models, and RAG Loaded and partially consumed by rag query --correct-query and repl --mode rag; correct never reads it; many keys are reserved src/config/schema.rs; from_app_config in src/pipeline/mod.rs Have correct load and honor the file; consume the reserved keys (see the inert-keys table below)
PipelineBuilder (library) A fluent builder for assembling a correction pipeline Only with_phonetic_matching (→ enable_phonetic) and with_ngram_scoring (→ enable_lm_rerank, a stub) affect the build; with_fuzzy_matching, with_wfst, with_neural_rescoring, with_context_aware are recorded but inert; the binary does not use the builder llammer-pipeline/src/builder.rs Give the recorded components real effects, or scope the builder to documented presets

○ Planned — not yet implemented

Item What it is Current state Code that exists Wiring it needs
Agent/customer networking + wire protocol Networked agent (server) and customer (client) REPLs exchanging newline-delimited JSON messages No sockets anywhere (std::net / tokio::net absent); the roles are local single-process REPLs whose "Customer connected / disconnected" lines are static banner text, not events; --address is stored and never read src/repl/protocol.rs — a complete, unit-tested wire format: Message { id, message_type, sender, original_text, text, timestamp, was_corrected, confidence, metadata }, MessageType { Chat, System, Correction, Typing, Connected, Disconnected, Ping, Pong, Error }, Protocol { version "1.0", encode, decode }; used by nothing A tokio::net transport in the agent/customer roles that reads --address, frames messages with Protocol::encode/decode, and emits real connection events
Language-model rerank layer Probabilistic LM rescoring of lattice paths (the capability behind enable_lm_rerank) No layer is added; lm_weight, ngram_path, and models.ngram are plumbed through config but unused for scoring, so candidate ties are broken without context Config fields; the empty branch in llammer-pipeline/src/lattice.rs; PipelineBuilder::with_ngram_scoring A Layer that scores paths with an n-gram LM (e.g. modified Kneser–Ney) or masked-LM pseudo-log-likelihood, added to the LayerPipeline
CFG-filter layer Grammar-constrained pruning of candidate paths (the capability behind enable_cfg_filter) No layer is added The empty branch in llammer-pipeline/src/lattice.rs A grammar-constrained Layer in the pipeline
Neural rescoring on the correction path ModernBERT (or similar) rescoring of correction candidates No neural model runs during correct. ModernBERT executes only in rag query embedding; the [neural] config section and correction.neural_weight are inert NeuralConfig in src/config/schema.rs; PipelineBuilder::with_neural_rescoring (inert) A neural rescoring Layer fed by libgrammstein::neural, gated behind the LM/neural toggle
Context-aware correction in the REPL Wiring the contextual corrector into the interactive loop so history influences correction REPL correction is stateless; the contextual corrector is never called, and even when called it only reweights confidence llammer-pipeline/src/contextual.rs (see Partial) Call correct_with_context(text, &conversation) from BaseRepl::process_input, and extend it to influence path selection, not just confidence
REPL tab-completion + syntax highlighting Slash-command completion and input highlighting in the REPL editor The editor is Editor<(), DefaultHistory> — the () helper installs no completer and no highlighter, so neither runs src/repl/completer.rs (ReplCompleter); src/repl/highlighter.rs (ReplHighlighter) A rustyline Helper that wires both, installed via editor.set_helper(...)
repl --mode rag live search An interactive RAG query REPL A placeholder RagRepl (prompt query> ) whose execute_query corrects the query then prints "No results (index not implemented in demo)"; /load only checks path.exists() and stores it src/repl/rag.rs (RagRepl, RagCommand) Load a real libgrammstein::rag::RagIndex, embed with ModernBertEmbedder, run index.query(...), and render results (mirror the rag query command path)
Parallel internal RAG query API A second, richer query/result-formatting API distinct from the live path QueryConfig, QueryBuilder, QueryFilters, ResultFormatter, ResultSet are defined but used by nothing outside src/rag/; the live path is src/cli/commands/rag.rs calling libgrammstein directly src/rag/{mod,query,results}.rs Route rag query (and the REPL rag mode) through this API, or remove it
Selectable color schemes Alternative terminal palettes for REPL output Colors::default() is always used; Colors::minimal() and Colors::high_contrast() are never selected (no flag or config key) src/repl/colors.rs (minimal, high_contrast) A --color <scheme> flag (or config key) selecting the palette passed to BaseRepl

Inert CLI flags

These flags are declared, parsed, and shown in --help, but are accepted and currently ignored. They are kept documented (rather than hidden) so the CLI surface is transparent; the CLI component overview cross-references each one.

Flag Subcommand(s) Why it is inert
-c, --config all (global) correct builds its config from arguments and never reads a file; the other paths load config from a fixed order, not from this flag.
-v, --verbose all (global) Logging is driven solely by RUST_LOG; the verbosity count is never consumed.
-m, --models correct, repl, rag * The model directory is never used to load models on these paths.
--insertion-penalty correct The lattice uses a single edit_weight; there is no separate insertion cost.
--deletion-penalty correct Same — there is no separate deletion cost.
--beam-width correct Stored in config, but Viterbi is exact, so beam width has no effect.
--neural-rescore correct Maps to enable_lm_rerank, a no-op stub layer (see Partial above).
--address repl Stored but never read — there is no networking.
-l, --language rag init Ignored by index construction. (On correct and rag query the language is consumed.)
--summarize rag init Accepted but ignored — no summarization runs.

Inert configuration keys

The full schema (src/config/schema.rs) defines more keys than the code consumes. The following are reserved / not yet wired; the configuration overview documents the live keys in detail.

Key(s) Status
all of [neural] (enabled, device, batch_size, top_k) Reserved — no neural model runs on the correction path.
correction.insertion_penalty, correction.deletion_penalty, correction.neural_weight Reserved — mirror the corresponding inert flags.
rag.include_explicit_synopsis, rag.include_generated_synopsis, rag.summarization_sentences Reserved — the query path renders synopses and topics from the index directly.
models.embedding, models.hybrid, models.modernbert Reserved — embedding uses EmbeddingConfig::default().

For contrast, the keys consumed today are correction.{max_edit_distance, edit_weight, lm_weight}, models.{dictionary, ngram} (via from_app_config), rag.top_k (the REPL rag mode), and language. There is no [repl], [output], [logging], or [language] section, and no config subcommand.

Known issues (code, not documentation)

These are honest notes about the current source. They are code defects, out of scope for the documentation to fix, but contributors should be aware of them.

  • benches/correction.rs does not compile. It calls a stale two-argument constructor LatticeCorrectionPipeline::new(config, &app_config); the shipped signature takes one argument — LatticeCorrectionPipeline::new(config: LatticePipelineConfig). cargo bench therefore fails until the benchmark is updated to the one-argument API. Building or installing the binary is unaffected, because benchmarks are compiled only under cargo bench.

    $ cargo build --release   # OK — binary builds
$ cargo bench             # FAILS — benches/correction.rs uses the old 2-arg new()

  • src/data/basic_dictionary.txt is an orphan. The dictionary embedded at build time (via include_str!) is llammer-pipeline/data/basic_dictionary.txt; the copy under src/data/ is not referenced by any code and can drift from the real one.

  • Correction quality is bounded by the embedded dictionary and the missing LM. correct always uses the 363-word embedded dictionary with no override, and ties are broken without a language model, so out-of-vocabulary tokens pass through and some picks are surprising (for example helo → help). The larger/custom dictionary and Language-model rerank layer items above are the intended fixes.

See Also