rag-workflow.md

RAG Workflow Examples

This guide walks through llammer's retrieval workflow end to end with real commands: build an index from a corpus, query it (optionally spell-correcting the query first), grow it with new documents, and explore the interactive RAG REPL. Every command and flag shown here exists in the shipped CLI; the surrounding concepts are explained in the RAG overview.

What retrieval does here. rag indexes documents with a ModernBERT embedder and ranks them against a query by cosine similarity (Salton et al., 1975; Warner et al., 2024). The only grammar-specific twist is --correct-query, which spell-corrects the query before embedding so a typo still retrieves the right documents (the retrieval half of the RAG idea of Lewis et al., 2020).

Retrieval scores, titles, and topics depend on your corpus and the ModernBERT model, so the result blocks below are representative — treat the exact numbers as illustrative.

---

0. Prerequisites

Build the binary once:

$ cargo build --release
$ ls target/release/llammer
target/release/llammer

The examples below invoke llammer directly; if it is not on your PATH, either use the built path (./target/release/llammer …) or run through Cargo (cargo run --release -- rag …). All three forms are equivalent.

Logging goes to stdout. llammer configures its tracing logger to write to stdout (default filter llammer=info,libgrammstein=warn), so any log line is not removed by 2>/dev/null. The rag commands are quiet at the default level, but when you pipe output into another program, prefix RUST_LOG=off for guaranteed-clean output — for example RUST_LOG=off llammer rag query "customer service" --index ./index.

---

1. Prepare a corpus

rag init indexes a directory of text documents. Create a small customer-support knowledge base:

$ mkdir -p corpus
$ printf 'Customer service resolves a customer problem and answers questions.\n' > corpus/customer-service.md
$ printf 'A business manager oversees each project and its budget.\n'           > corpus/business.md
$ printf 'Store every document with its address and phone information.\n'        > corpus/documents.md
$ printf 'Accurate information helps a customer make a good decision.\n'         > corpus/information.md
$ printf 'The manager assigns work on the project to the support team.\n'       > corpus/manager.md
$ printf 'A problem report records the customer issue and the fix.\n'           > corpus/problem.md

That is six documents. This support/office vocabulary is chosen deliberately: it lets the query-correction examples in §3.1 use words the corrector actually knows (see the note there about llammer's built-in dictionary).

---

2. Build the index

$ llammer rag init --documents ./corpus --output ./index
[1/3] Initializing RAG index from ./corpus
  Indexed 6 documents
[2/3] Saving index to ./index
[3/3] Done!

--documents and -o/--output are the only meaningful flags. --summarize and --language/-l are accepted but ignored by init (the index is built with libgrammstein's default builder configuration), so this is equivalent:

$ llammer rag init --documents ./corpus --output ./index --summarize --language en-US
[1/3] Initializing RAG index from ./corpus
  Indexed 6 documents
[2/3] Saving index to ./index
[3/3] Done!

While it runs, an indicatif spinner shows transient Processing i/6 documents... progress before settling on the Indexed 6 documents line above.

---

3. Query the index

A basic query loads the index, embeds the query string, and prints ranked results:

$ llammer rag query "customer service" --index ./index --top-k 3
Loaded index with 6 documents

Results:

1. [0.93] customer-service
   URI: ./corpus/customer-service.md
   Synopsis (generated): Customer service resolves a customer problem and answers questions.
   Topics: customer support, service quality

2. [0.84] problem
   URI: ./corpus/problem.md
   Synopsis (generated): A problem report records the customer issue and the fix.

3. [0.78] information
   URI: ./corpus/information.md
   Synopsis (generated): Accurate information helps a customer make a good decision.

Reading the output (each field is described in rag/overview §4.1):

• Loaded index with N documents — confirms the index opened and how many documents it holds.
• 1. [0.93] customer-service — rank, cosine score to two decimals, then the title (a document's title falls back to its URI when none is set).
• URI: — the document's source location.
• Synopsis (explicit|generated): — the description; explicit when the document provided its own, otherwise generated by libgrammstein's summarizer.
• Topics: — printed only when the index has a topic model and the document has topic assignments.

3.1 Correcting a misspelled query

Add --correct-query to spell-correct the query before it is embedded. When the correction changes the string, the command reports it on a Corrected query: line:

$ llammer rag query "custmer servce" --index ./index --top-k 3 --correct-query
Loaded index with 6 documents

Corrected query: custmer servce -> customer service

Results:

1. [0.93] customer-service
   URI: ./corpus/customer-service.md
   Synopsis (generated): Customer service resolves a customer problem and answers questions.
   Topics: customer support, service quality

2. [0.84] problem
   URI: ./corpus/problem.md
   Synopsis (generated): A problem report records the customer issue and the fix.

3. [0.78] information
   URI: ./corpus/information.md
   Synopsis (generated): Accurate information helps a customer make a good decision.

Multi-word queries are corrected token by token, so documnet and infrmation becomes document and information and still finds the documents you meant. Without --correct-query, the misspelled string is embedded verbatim and retrieval quality degrades.

Which typos get fixed. Query correction reuses the same lattice pipeline as llammer correct, which by default carries a small (363-word) common-English dictionary. It reliably repairs everyday misspellings whose intended word is in that dictionary — custmer → customer, servce → service, documnet → document, infrmation → information, thier → their, problm → problem. It will not fix terms it has never seen (for example machine or learning are not in the built-in dictionary), and with no language model to break edit-distance ties it can map an unknown word to a near dictionary word instead of the one you intended. Unlike the correct command, rag query --correct-query honors a custom [models] dictionary from your llammer.toml, so point it at a domain lexicon to correct domain-specific queries. A larger default dictionary and a language-model tie-breaker are tracked in the roadmap.

3.2 Controlling result count

-k/--top-k sets how many results are returned (default 10):

$ llammer rag query "store a document" --index ./index -k 1
Loaded index with 6 documents

Results:

1. [0.88] documents
   URI: ./corpus/documents.md
   Synopsis (generated): Store every document with its address and phone information.

The rag query command reads --top-k directly; it does not consult the rag.top_k config key (that key is used only by the RAG REPL — see §5).

---

4. Extend the index

rag extend loads an existing index and appends documents from another directory. Unlike init, extend requires -o/--output and scans only the top-level files of the new directory (non-recursive), taking each document's title from its filename stem:

$ mkdir -p more
$ printf 'The refund policy explains how a customer returns an order.\n' > more/refunds.md
$ printf 'Contact the support team by phone for any problem.\n'         > more/contact.md

$ llammer rag extend --index ./index --documents ./more --output ./index2
[1/4] Loading existing index from ./index
[2/4] Scanning new documents from ./more
[3/4] Adding 2 new documents
  Added 2 documents
[4/4] Saving extended index to ./index2
Index extended: 6 -> 8 documents

The grown index is written to --output (./index2 here), leaving the original ./index untouched. To update in place instead, pass the same path for both:

$ llammer rag extend --index ./index --documents ./more --output ./index

Query the extended index exactly as before:

$ llammer rag query "customer problem" --index ./index2 --top-k 2
Loaded index with 8 documents

Results:

1. [0.90] problem
   URI: ./corpus/problem.md
   Synopsis (generated): A problem report records the customer issue and the fix.

2. [0.82] contact
   URI: ./more/contact.md
   Synopsis (generated): Contact the support team by phone for any problem.

---

5. Interactive RAG REPL

llammer repl --mode rag opens a query prompt that spell-corrects each line. It is currently a placeholder/demo: it does not load or search an index, and it prints a stub instead of results. Use it to see the query-correction behavior; use the rag query command (§3) for real retrieval.

$ llammer repl --mode rag --index ./index

llammer - RAG Query REPL
Language: en-US, Top-k: 10
Type /help for commands, /exit to quit

query> customer service
Searching for: "customer service" (top 10)

No results (index not implemented in demo)

In production, this would query the RAG index and display:
  - Document titles
  - Relevance scores
  - Synopsis/summaries
  - Source URIs

query> /exit
Goodbye!

Queries are spell-checked by default; when the corrector changes the input it is shown before the (stub) search:

query> custmer servce
  [original query] custmer servce
[corrected query] customer service

Searching for: "customer service" (top 10)

No results (index not implemented in demo)

The REPL's own commands (source src/repl/rag.rs) are:

Command	Effect
/help, /h, /?	Show the command list
/exit, /quit, /q	Leave the REPL (Goodbye!)
/clear, /cls	Clear the terminal screen
/load <path>	Record an index path (checks it exists; does not actually open it in the demo)
/info	Print the stored index path (demo placeholder)
/topk <n>, /k <n>	Set the result count
/autocorrect, /ac	Toggle query auto-correction
/history	Show the queries entered this session

The Top-k: 10 shown in the banner is where the rag.top_k config key is consumed; /topk <n> overrides it for the session.

---

6. What is not here

The following appeared in earlier drafts but do not exist in the shipped CLI — omit them:

• No rag info, rag export, or rag import subcommands. The only rag subcommands are init, query, and extend. To inspect an index, query it or read the files under the index directory directly.
• No --min-score, --topic, or --format json on rag query. Results are always the human-readable text block shown above. A richer query/formatting API (score thresholds, filters, JSON/TSV) is implemented in the unused src/rag/ module but is not wired to the CLI — see rag/overview §6.
• No /topics or /open in the RAG REPL. The demo REPL exposes only the commands in the table above, and its search is a placeholder.

---

References

1. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS. https://arxiv.org/abs/2005.11401
2. Warner, B., Chaffin, A., Clavié, B., Weller, O., Hallström, O., Taghadouini, S., et al. (2024). Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder (ModernBERT). arXiv:2412.13663. https://arxiv.org/abs/2412.13663
3. Salton, G., Wong, A., & Yang, C. S. (1975). A Vector Space Model for Automatic Indexing. Communications of the ACM, 18(11), 613–620. https://doi.org/10.1145/361219.361220

---

See Also

• RAG Overview — architecture, math, and the full component map.
• CLI Overview — the complete rag command surface.
• REPL Modes — all four REPL modes, including the RAG demo.
• Basic Correction — the correction engine reused by --correct-query.
• Data Flow — index-build and query flows in context.
• libgrammstein Integration — the neural/RAG dependency.
• Configuration — [rag] keys and correction settings.
• Roadmap — planned generation-side RAG and the unused query API.
• libgrammstein on GitHub — upstream retrieval library.