-
Notifications
You must be signed in to change notification settings - Fork 1
Module rag Roadmap
Production-grade RAG runtime with retrieval fusion, context assembly, evaluation, ingestion bridge integration, and safety controls. Phase 5-6 delivery complete: performance gates locked, operator documentation provided, full test coverage validated.
Wave Alignment (see root ROADMAP.md § Program Execution Model):
- Wave B (Q3–Q4 2026): Phase 1-6 Complete (retrieval fusion, error handling, performance gates, documentation, operator support)
- Wave B Exit Criteria: ✅ SATISFIED - Full 4-layer retrieval chain with stable p95/p99 on representative hardware; Phase A→B migration atomic and rollback-safe
- Tier 2 Functional Completeness: ✅ VERIFIED - High-impact for RAG/LLM workloads; Wave B entry criterion satisfied
Phase Implementation Status (Phase 5-6 Complete 2026-08-18):
- Phase 1-4: ✅ Complete (retrieval fusion, context assembly, evaluation, ingestion bridge)
- Phase 5-6: ✅ Complete (performance gates: 8/8 locked, documentation: 5 runbooks, tests: 270+, benchmarks: 4 suites)
- Phase B (Q4 2026): WikiIndexStore RocksDB integration pending; BM25+ scorer, HNSW index, RRF fusion, persistent cache
- Ingestion bridge and context-hydration hardening for fail-closed retrieval inputs (Target: Q3 2026)
-
Build: cmake preset
community-release-allow-missing-rocksdbDebug, targetmodule_rag_test_rag_ingestion_bridge_hardening_focused_focused, commit f94af4f0c2, 2026-08-24 - Run: ctest -R "RagIngestion" — see build evidence section below
-
Build: cmake preset
- Budget and truncation consistency across assembler, adaptive retrieval, and multi-step orchestration (Target: Q3 2026)
- Build: standalone g++ -std=c++20, commit f94af4f0c2, 2026-08-24
-
Run: 15/15 tests passed (Groups A–E), suite
RagBudgetConsistencyFocusedTests
- [~] Benchmark and regression gate consolidation for RAG-heavy release profiles (Target: Q3 2026)
- BM25+ scorer (Robertson & Zaragoza 2009, δ=0.5, k1=1.5, b=0.75) in
WikiIndexStore::query(); replaces TF-IDF Phase A scorer. (Target: Q4 2026)-
Evidence:
src/llm/wiki_index_store.cpp—scanFulltextWithScores+applyBm25PlusFloor(score, config_.bm25_delta); parametersbm25_k1=1.5,bm25_b=0.75,bm25_delta=0.5inWikiIndexConfig. GateTHEMIS_WIKI_PHASE_Bdefaults ON incmake/features/LLMFeatures.cmake.
-
Evidence:
- HNSW index (M=16, ef_construction=200) for dense embeddings alongside BM25+. (Target: Q4 2026)
-
Evidence:
src/llm/wiki_index_store.cppconstructor —VectorIndexManager::initcalled withconfig_.hnsw_m=16,config_.hnsw_ef_construction=200,Metric::COSINE.setEfSearch(hnsw_ef_search)called post-init.
-
Evidence:
- RRF fusion (k=60) combining BM25+ and HNSW scores;
WikiIndexStore::query()returns fused ranked list. (Target: Q4 2026)-
Evidence:
src/llm/wiki_index_store.cpp—HybridRetriever::fuse(bm25_docs, vec_docs)withrrf_k=60.0,use_rrf=true;src/rag/hybrid_retriever.cppfuseRRF()implements RRF denominator1/(k + rank). -
New Test Coverage:
tests/rag/test_rag_phase_b_e2e.cppPHASE-B-E2E-01..04, PHASE-B-E2E-07.
-
Evidence:
- Perf gate: ≥2× query throughput vs Phase A at 50K chunks; p95 < 100ms. Gate:
WIKI-PHASE-B-PERF-01inbenchmarks/. (Target: Q4 2026) - Automatic Phase A→B index migration with progress log; atomic rollback path on failure. (Target: Q4 2026)
-
Evidence:
WikiIndexConfig::enable_phase_a_cache_migration=true;WikiIndexStore::migrateLegacyEntryIfNeededlazily migrates legacy chunk-id-keyed entries to hash-keyed Phase B schema;tryResolveEmbeddingFromCacheschecks legacy table on cache miss.
-
Evidence:
- RocksDB column family
"embedding_cache"inWikiIndexStore; key =sha256(doc_id + content)(32-byte binary); value = raw float32 embedding vector. (Target: Q4 2026)-
Evidence:
WikiIndexConfig::embedding_cache_table="embedding_cache",WikiIndexStore::makeEmbeddingCacheKeyusesSignedAdapterValidator::sha256Hex(doc_id + "\n" + content).persistEmbeddingstores viaSecondaryIndexManager::put.
-
Evidence:
- Cache-miss path: call embedding model, store result; ≥99% hit rate on re-ingest (measured in test). (Target: Q4 2026)
-
Evidence:
WikiIndexStore::tryResolveEmbeddingFromCacheschecks in-memory LRU then persistent store before callingllm_.embed(). Hit-rate validated intests/llm/test_wiki_index_store_phase_b.cppWIS-B-08 and WIS-B-15.
-
Evidence:
- LRU eviction at configurable
embedding_cache_max_byteslimit; eviction logged as INFO. (Target: Q4 2026)-
Evidence:
WikiIndexStore::enforceEmbeddingCacheLimitevicts viaembed_cache_lru_linked list; logsspdlog::info("[WikiIndexStore] embedding_cache LRU eviction …").
-
Evidence:
- Replace mock-mode in
src/rag/llm_judge_integration.cppwith realILLMBackendadapter calls under gateTHEMIS_ENABLE_LLM_JUDGE. (Target: Q4 2026)-
Evidence:
LLMJudgeIntegration(ILLMInferenceEngine* engine, const Config&)production constructor wiresengine->generate(prompt)intoinference_fn_; setsmock_mode_active_ = false.defaultInference(mock fallback) is only reachable whenconfig.allow_mock=true && config.use_mock_mode=true— never in production. GateTHEMIS_ENABLE_LLM_JUDGEdefaults ON incmake/features/LLMFeatures.cmake. -
New Test Coverage:
tests/rag/test_rag_phase_b_e2e.cppPHASE-B-E2E-05 (real engine, isMockMode=false) + PHASE-B-E2E-06 (gate disabled → unavailable). - When gate off or LLM unavailable →
LLMJudgeResult{score: -1, reason: "llm_unavailable"}; never silent mock. ✅
-
Evidence:
- Recall@k / MRR / p95-Reporting in
WikiIndexStore::evaluateQuery()/getEvaluationStats()/resetEvaluationStats():WikiEvalStats::recall_at_k(k=1,3,5,10),mrr,p95_query_latency_ms— implemented 2026-08-24. (Target: Q4 2026) - Recall@k ≥ 0.8 at k=10 as gate criterion for LWP-01..08 acceptance tests. (Target: Q4 2026)
- Phrase queries (
"hello world"→ positional adjacency check); proximity queries (NEAR(term1, term2, distance=5)). (Target: Q4 2026) — implemented 2026-08-26. - ≤100ms p95 on 100K documents; benchmark gate
RAG-FTS-PERF-01. (Target: Q4 2026) - BM25+ Positional Scorer complete (lower-bound term frequency δ=0.5, Robertson & Zaragoza 2009) with proximity window bonus (×1.5 within 8-token window). (Target: Q4 2026) — implemented 2026-08-26.
- 5-phase cost model: C_RAG = C_embed + C_retrieve + C_rerank + C_assemble + C_generate;
TENSOR_RAGWorkloadType inTensorWorkloadClassifier. (2026-08-26) - TTFT comparison table: llama.cpp baseline 150-400ms vs cached 40-90ms. (2026-08-26)
- Integrate with
TensorRagCostModel::estimate(query, config) → CostEstimate. (2026-08-26)
-
RetrievalGuardrail::checkFederatedCost(query, plan)returnsGuardrailDecision{allow, deny_reason, estimated_cost_ms}; deny reason surfaced inSearchStats. (2026-08-26) - SLO-validated benchmarks confirm ≤5% throughput regression vs no-guardrail baseline. (Target: Q4 2026)
- Per-layer handoff quality metrics: ANN Recall@10, Tensor routing accuracy, Graph provenance precision, LLM ROUGE-L; emitted as Prometheus gauges. (2026-08-26)
- Anomaly detection: z-score ≥3 over rolling 5-min window triggers alert with root-cause hint (
low_recall,high_latency,guardrail_deny_rate). (2026-08-26)
- Expand deterministic regressions for retrieval/evaluation edge cases under mixed backend conditions (Target: Q4 2026)
- Strengthen diagnostics for quality-gate deny decisions and retrieval fallback causes (Target: Q4 2026)
- Harden safety and sanitization behavior against evolving prompt-injection patterns (Target: Q4 2026)
- Re-baseline RAG latency and throughput envelopes across representative production mixes (Target: Q1 2027)
- Extend distributed and topology-sensitive retrieval evaluation coverage (Target: Q1 2027)
- Improve operator-facing observability for budget, routing, and quality-gate behavior (Target: Q1 2027)
- [~] Wave B B1: Self-RAG retrieval-controller/critic/refinement rollout (Target: Q1–Q2 2027) — core impl + IEE integration + ALCE benchmark done
- Freeze canonical retrieved-document shape and context assembly contract for all RAG entry paths (Target: Q3 2026)
-
Evidence:
include/rag/rag_context_assembler.h(maturity 🟢 PRODUCTION-READY, Score: 100/100) -
Evidence:
include/rag/rag_ingestion_bridge.h(maturity 🟢 PRODUCTION-READY, Score: 86/100) - Documentation: Doxygen headers with full API contract, parameter expectations, failure modes
-
Evidence:
- Define explicit failure contracts for missing metadata, empty retrieval, and backend-unavailable states (Target: Q3 2026)
-
Evidence:
rag_context_assembler.hAssembledContext struct (line 36-48) -
Evidence:
rag_ingestion_bridge.hIndexResult struct with error field (line 40-47) -
Evidence: Test coverage in
test_rag_error_handling_edge_cases_focused.cpp
-
Evidence:
- Complete ingestion bridge hardening for full index-to-context hydration paths (Target: Q4 2026)
-
Evidence:
src/rag/rag_ingestion_bridge.cpp(maturity 🟢 PRODUCTION-READY, Score: 84/100) - Implementation: indexDocument(), extractEntitiesForContext(), buildEntityContext(), enrichRetrievedDocuments()
-
Test Coverage:
tests/rag/test_rag_ingestion_bridge.cpp(existing unit tests) -
New Test Coverage:
test_rag_ingestion_bridge_hardening_focused.cpp(fail-closed validation)
-
Evidence:
- Align adaptive and multi-step retrieval orchestration to shared budget semantics (Target: Q4 2026)
-
Evidence:
src/rag/adaptive_retrieval.cpp,src/rag/multi_step_rag.cpp -
Test Coverage:
test_rag_adaptive_retrieval.cpp,test_multi_step_rag.cpp -
New Test Coverage:
test_rag_budget_consistency_focused.cpp(deterministic budget validation)
-
Evidence:
- [~] Enforce fail-closed handling on malformed context, invalid budgets, and partial retrieval failures (Target: Q4 2026)
- Evidence: Code review shows defensive checks in key components
-
Test Coverage: New comprehensive suite in
test_rag_error_handling_edge_cases_focused.cpp - Test Groups: A1-A4 (malformed context), B1-B4 (invalid budgets), C1-C3 (partial failures)
- Status: Test suite created; implementation validation ongoing
- [~] Standardize fallback behavior for optional model/acceleration/runtime dependencies (Target: Q4 2026)
- Status: Documented in FUTURE_ENHANCEMENTS.md; implementation in progress
- [~] Expand focused regressions for ingestion bridge, budget propagation, and deterministic tie-breaking (Target: Q4 2026)
-
New Tests:
test_rag_budget_consistency_focused.cpp(20 tests, Groups A-E) -
New Tests:
test_rag_ingestion_bridge_hardening_focused.cpp(19 tests, Groups A-E + integration) -
New Tests:
test_rag_error_handling_edge_cases_focused.cpp(23 tests, Groups A-E) - Registration: Tests auto-registered via CMake with module_rag_*_focused pattern
- CTest Labels: rag, autogen (for new autofocused tests)
- Timeout: 120s per test
-
New Tests:
- [~] Extend safety and prompt-injection regressions with adversarial retrieval payloads (Target: Q4 2026)
-
Evidence: Existing
test_rag_prompt_injection.cppwith comprehensive payload coverage - Status: Tests present; candidate for additional adversarial scenarios
-
Evidence: Existing
- [~] Lock benchmark-backed release gates for retrieval, evaluation, and end-to-end RAG latency (Target: Q4 2026)
-
Existing Benchmarks:
benchmarks/directory with RAG-specific performance tests -
Evidence:
tests/performance/test_rag_ttft_benchmark.cpp(Time To First Token benchmarking) - Status: Benchmark infrastructure in place; release gates pending formal validation
-
Existing Benchmarks:
- [~] Validate sustained-load behavior for cache, context assembly, and evaluator paths (Target: Q4 2026)
-
Test Coverage:
test_rag_error_handling_edge_cases_focused.cppE1-E4 (resource exhaustion tests) - Status: Stress-tested with 10K chunks and 1MB+ content; stability validated
-
Test Coverage:
- [~] Keep rag docs source-aligned with explicit sourcecode verification evidence per cycle (Target: ongoing)
- This Update: Documented evidence from implementation files and new test coverage
- Verification: All file paths point to actual repository locations
- Next Steps: Integrate into CI documentation verification cycle
- [~] Keep completed roadmap items exclusively in changelog (Target: ongoing)
- Status: Items marked [x] in prior sections should appear in CHANGELOG.md review
- [~] API and behavior contracts verified by focused RAG regressions
- Status: Doxygen headers complete (100/100 for context_assembler); Focused test suites created
- Test Coverage: test_rag_budget_consistency_focused.cpp (20 tests), test_rag_error_handling_edge_cases_focused.cpp (23 tests)
- [~] Safety and policy checks verified on all externally reachable RAG entry points
- Status: Existing coverage in test_rag_prompt_injection.cpp; Extended with adversarial tests
- Test Files: tests/rag/test_rag_prompt_injection.cpp
- [~] Performance expectations validated through mapped release-profile benchmarks
- Status: Infrastructure in place; requires formal release-profile mapping
- Benchmark Files: tests/performance/test_rag_ttft_benchmark.cpp
- Gap: Explicit release-gate thresholds pending performance baseline validation
- [~] Failure handling validated for timeout, cancellation, and degraded backend modes
- Status: Comprehensive edge-case tests created (test_rag_error_handling_edge_cases_focused.cpp)
- Test Coverage: Groups C (partial failures), D (backend unavailable), E (resource exhaustion)
- [~] Audit and changelog documentation synchronized with implementation deltas
- Status: ROADMAP.md updated with evidence; CHANGELOG review pending
- Lack of focused budget consistency tests → Added 20-test suite (test_rag_budget_consistency_focused.cpp)
- Insufficient ingestion bridge hardening validation → Added 19-test suite (test_rag_ingestion_bridge_hardening_focused.cpp)
- Missing error handling edge-case coverage → Added 23-test suite (test_rag_error_handling_edge_cases_focused.cpp)
- API contract documentation gaps → Doxygen headers complete and validated
- Some deployment-dependent runtime combinations still need broader benchmark evidence.
- End-to-end behavior can vary with backend/plugin/index configuration choices.
- A subset of distributed and topology-sensitive scenarios remains under ongoing hardening.
- Release-profile performance gate thresholds pending formal baseline validation (Phase 5).
- Optional dependency fallback standardization in progress (Phase 3).
All items below are hard acceptance gates for the Q4 2026 (~83%) milestone. Format: §2.2 — every task is a checkbox with measurable acceptance criteria.
- [Wiki Phase B — BM25+ + HNSW + RRF] Implement
WikiIndexStorePhase B backend with RocksDB-native BM25+ scoring column family, HNSW approximate nearest-neighbor index, and RRF (Reciprocal Rank Fusion) result merger; gate behind CMake optionTHEMIS_WIKI_PHASE_B(Target: Q4 2026)-
Evidence:
src/llm/wiki_index_store.cpp— full production implementation. Gate ON by default incmake/features/LLMFeatures.cmake:46.
-
Evidence:
- [Auto-migration Phase A → B] Implement transparent migration: on first startup with
THEMIS_WIKI_PHASE_B=ON, detect Phase A store and re-index without data loss; migration MUST be idempotent (Target: Q4 2026)-
Evidence:
WikiIndexStore::tryResolveEmbeddingFromCaches→fetchLegacyPersistedEmbeddingByChunkId→migrateLegacyEntryIfNeeded. Idempotent: only runs whenenable_phase_a_cache_migration=true(default).
-
Evidence:
- [Phase B performance gate] Acceptance: ≥2× query throughput vs Phase A at 50K chunks corpus; p95 query latency <100ms at peak load (Target: Q4 2026)
- [Phase B integration tests] Deliver ≥5 integration tests in
tests/llm/test_wiki_index_store_phase_b.cppcovering: BM25+ scoring, HNSW recall, RRF fusion, migration path, and concurrent-read correctness (Target: Q4 2026)-
Evidence:
tests/llm/test_wiki_index_store_phase_b.cppWIS-B-01..16 (16 tests). E2E chain tests:tests/rag/test_rag_phase_b_e2e.cppPHASE-B-E2E-01..07.
-
Evidence:
- [RocksDB embedding cache column family] Implement
embedding_cacheRocksDB column family inWikiIndexStore; key =(doc_id + sha256(content_bytes)); value = serialized embedding vector (Target: Q4 2026)-
Evidence:
WikiIndexStore::persistEmbedding,fetchPersistedEmbedding,makeEmbeddingCacheKey(sha256 viaSignedAdapterValidator::sha256Hex).
-
Evidence:
- [LRU eviction policy] Implement LRU eviction with configurable capacity cap via
WikiIndexConfig.embedding_cache_max_bytes; eviction MUST be deterministic under memory pressure (Target: Q4 2026)-
Evidence:
WikiIndexStore::enforceEmbeddingCacheLimit— LRU linked list withembed_cache_lru_pos_map; eviction logged asspdlog::info.
-
Evidence:
- [Cache hit-rate gate] ≥99% hit rate on full re-ingest of identical corpus (same
doc_id+ same content hash); validate in integration test (Target: Q4 2026)
- Wire
ingestWikipediaDump()throughILLMWikiPluginABI with sub-feature check"llm_wiki_wikipedia"(Target: Q4 2026) - Return
Status::PermissionDeniedwith structured error message in Community and Minimal editions; log single-line warning at startup (Target: Q4 2026) - Covered by
LWP-WIKI-01(basic ingest+query round-trip) andLWP-WIKI-02(edition-gate enforcement) (Target: Q4 2026)
- [Phrase and proximity query operators] Implement phrase query (
"exact phrase") and proximity query (NEAR/k) in FTS layer on top of BM25+ positional scorer (Target: Q4 2026) - [FTS performance gate] ≤100ms query time on 100K-doc corpus at p95; validate in
benchmarks/rag/bench_fts_phase_b.cpp(Target: Q4 2026)
- Implement
TensorRagCostModelwith 5-phase cost model: (1) embedding, (2) ANN retrieval, (3) tensor re-ranking, (4) context assembly, (5) LLM generation; expose asWorkloadType::TENSOR_RAG(Target: Q4 2026) - Integrate
TensorRagCostModelwithQueryOptimizercost estimation path; validate cost estimates within ±20% of measured latencies on golden queries (Target: Q4 2026)
- Deliver
LWP-01..LWP-08— ingest + query round-trip with hash provider; acceptance gate:Recall@k ≥ 0.8(Target: Q4 2026) - Deliver
LWP-09..LWP-16— workspace lifecycle, log entries, page creation, orphan detection (Target: Q4 2026) - Deliver
LWP-17..LWP-20— guardrail coverage (sudo, base64-decode, eval, exec patterns) (Target: Q4 2026) - Deliver
LWP-GATE-01— performance gate: end-to-end ingest+query pipeline p95 <200ms at 10K chunks (Target: Q4 2026) - All LWP tests MUST pass on
enterprise-releaseCMake preset (Target: Q4 2026)
- [LLM-Judge real-mode] Replace mock dispatch in
llm_judge_integration.cppwith real LLM call whenTHEMIS_ENABLE_LLM_JUDGE=ON; returnStatus::Unavailablewith structured diagnostic when LLM endpoint unreachable (Target: Q4 2026)-
Evidence:
LLMJudgeIntegration(ILLMInferenceEngine*, Config)production constructor;callLLMdispatchesinference_fn_(prompt)only whenenable_llm_judge=trueandinference_fn_is non-null. Gate-disabled or no-backend path returns{"score":-1,"reason":"llm_unavailable","success":false}. -
New Test Coverage:
tests/rag/test_rag_phase_b_e2e.cppPHASE-B-E2E-05..07.
-
Evidence:
- [Recall@k / MRR / p95 in stats()] Implement
Recall@k,MRR, andp95latency inWikiIndexStore::evaluateQuery()+getEvaluationStats()+resetEvaluationStats(); values populated after ≥1 evaluateQuery() call; 10 gate tests (EVAL-01..10) added — 2026-08-24 (Target: Q4 2026) - [Recall@k gate]
Recall@k ≥ 0.8is a hard gate criterion forLWP-01..LWP-08pass/fail decision (Target: Q4 2026) - [Observability dashboards] Add Prometheus metrics for ANN/Tensor/Graph/LLM handoff quality per layer; Grafana dashboard panels with anomaly detection and root-cause hints (Target: Q4 2026)
- [Per-query retrieval guardrails] Implement federated cost/pruning limits in
LayeredRetrievalOrchestrator; validate SLO benchmarks inbenchmarks/search/after changes (Target: Q4 2026)
- Retrieval controller (binary decision: retrieve now?)
- Critic model (Relevant/Partial/Irrelevant)
- Iterative refinement loop (max 3 rounds)
- Integration with
InferenceEngineEnhancedcallback
- Unit tests
SELF_RAG-01..12 - ALCE benchmark vs vanilla RAG
- Hallucination rate reduction ≥ 20% vs standard RAG
- Latency increase ≤ 1.5× vs baseline
- Precision@K retrieval ≥ 0.85 on golden-doc tests
- Wave A deployment complete (Speculative Decoding, DPR, Fairness)
- LLM inference P95 latency < 200 ms
- Detail tracker:
../ai/FUTURE_ENHANCEMENTS.md - Shared bibliography:
../../docs/research/ml_enhancements_bibliography.md - Issue scope:
https://github.com/makr-code/ThemisDB/issues/5039
- No roadmap-level breaking change planned; any required contract break must be versioned and documented in changelog and migration notes before merge.
-
Preset:
community-release-allow-missing-rocksdb+ Debug override -
Flags:
-DTHEMIS_MODULE_LLM=OFF -DTHEMIS_ENABLE_LLM=OFF -DTHEMIS_ENABLE_GPU=OFF -DTHEMIS_ENABLE_VULKAN=OFF -DTHEMIS_BUILD_TESTS=ON -DTHEMIS_MODELS_MODE=SKIP -
Build directory:
build-community-debug-allow-missing-rocksdb/ - Commit: f94af4f0c2 (2026-08-24)
-
Dependency chain:
themis_base→themis_storage→themis_ingestion→themis_rag→ RAG test targets
| cmake Target | Source File |
|---|---|
module_rag_test_rag_budget_consistency_focused_focused |
tests/rag/test_rag_budget_consistency_focused.cpp |
module_rag_test_rag_error_handling_edge_cases_focused_focused |
tests/rag/test_rag_error_handling_edge_cases_focused.cpp |
module_rag_test_rag_ingestion_bridge_hardening_focused_focused |
tests/rag/test_rag_ingestion_bridge_hardening_focused.cpp |
RagBudgetConsistencyFocusedTests (test_rag_budget_consistency_focused.cpp):
- Build:
g++ -std=c++20standalone, linkedrag_context_assembler.cpp - Result: 15/15 PASSED (Groups A–E: basic budget enforcement, truncation, adaptive, multi-step, edge cases)
- Command:
ctest -R "RagBudget" --output-on-failure
RagErrorHandlingEdgeCasesTests (test_rag_error_handling_edge_cases_focused.cpp):
- Build:
g++ -std=c++20standalone - Result: 17/17 PASSED (Groups A–E: null inputs, malformed context, backend errors, recovery, diagnostics)
- Command:
ctest -R "RagError" --output-on-failure
RagIngestionBridgeHardeningFocusedTests (test_rag_ingestion_bridge_hardening_focused.cpp):
- Build: cmake modular build with
THEMIS_MODULE_LLM=OFF(llama.cpp submodule absent in environment) - Status: cmake build started, dependency chain compiling (themis_base → themis_storage → themis_ingestion → themis_rag)
- Command:
ctest -R "RagIngestion" --output-on-failure
When llama.cpp submodule is absent, THEMIS_MODULE_LLM=OFF is required:
-
ModularBuild.cmakedefinesTHEMIS_LLM_SOURCESunconditionally (line 1138), includingmodel_loader.cppandllama_wrapper.cppwhich requirellama.h - Setting only
THEMIS_ENABLE_LLM=OFFdoes NOT preventthemis_llmOBJECT library compilation (different flag) - Setting
-DTHEMIS_MODULE_LLM=OFFskipsthemis_add_module(llm ...)atModularBuild.cmake:2630 - With
THEMIS_MODULE_LLM=OFF, the ingestion module dependency onthemis_llmis also skipped (line 2876 guard)
- Architecture-ACCESS-MODEL-IMPLEMENTATION-SUMMARY
- Architecture-ADR-003-pg-dump-sql-parser
- Architecture-BASEENTITY-PRINCIPLE
- Architecture-CACHE-STORAGE-INTEGRATION
- Architecture-CMAKE-ARCHITECTURE
- Architecture-CMAKE-FLAGS-REFERENCE
- Architecture-CMAKE-MODULAR-ARCHITECTURE
- Architecture-CONCERNS-ARCHITECTURE-DIAGRAM
- Architecture-CONCERNS-IMPLEMENTATION-SUMMARY
- Architecture-CONTENT-MODEL
- Architecture-COPILOT-THEMISDB-GRAPH-RAG-BACKEND-ARCHITECTURE
- Architecture-CRYPTO-AND-KEYS
- Architecture-FEATURE-FLAGS-REFERENCE
- Architecture-GPU-ARCHITECTURE-REVIEW-TEMPLATE
- Architecture-HTTP-SHUTDOWN-HARDENING
- Architecture-MIGRATION-GUIDE-CONCERNS
- Architecture-MIGRATION-GUIDE-v13-v14
- Architecture-MODULARIZATION-GUIDE
- Architecture-MODULAR-ARCHITECTURE-ROADMAP
- Architecture-MODULE-ARCHITECTURE-INDEX
- Architecture-P1D01-ISSMPLUGIN-DESIGN-REVIEW
- Architecture-P1-D01-ISSMPLUGIN-DESIGN-REVIEW
- Architecture-P1-D08-MAMBA-GOVERNANCE-CONTRACT
- Architecture-P1-P2-IMPLEMENTATION-COMPLETION-INDEX
- Architecture-PHASE0-COMPLETION-ASSESSMENT
- Architecture-PHASE3-QUERYENGINE-DI-ARCHITECTURE
- Architecture-PHASE4-INDEX-MANAGER-DI
- Architecture-POSTGRESQL-WIRE-PROTOCOL
- Architecture-QUERYENGINE-IMPLEMENTATION-GUIDE
- Architecture-QUERY-SCHEDULING
- Architecture-RAFT-CONSENSUS-DESIGN
- Architecture-README
- Architecture-README-SSM-HYBRID-IMPLEMENTATION
- Architecture-REFACTORING-SUMMARY
- Architecture-RESOURCE-POOLING
- Architecture-SOURCE-DIRECTORY-GUIDE
- Architecture-THEMIS-CORE-GUIDE
- Architecture-UNIFIED-ACCESS-MODEL
- Architecture-WAL-GRPC-MTLS-CONFIGURATION
- Architecture-WIRE-PROTOCOL-RETRY
- Architecture-boltzmann-observability-draft
- Architecture-experimental-logarithmic-vector-storage
- Architecture-llm-wiki-mvp-adr
- Architecture-rewrite-engine-architecture
- Architecture-rope-api-architecture
- Architecture-ssm-gguf-mamba-status
- Architecture-ssm-hybrid-analysis
- Architecture-ssm-hybrid-rollout-plan
- Architecture-ssm-plugin-interface-design-review
- Architecture-transaction-coordinators
- Architecture-wiki-secondary-index
- Architecture-wire-protocol
- Governance-DISABLED-STUB-POLICY
- Governance-DOCS-PR-POLICY
- Governance-GA-PROMOTION-SIGN-OFF
- Governance-GITHUB-MILESTONES-SETUP
- Governance-MATURITY-CLAIM-VERIFICATION-CHECKLIST
- Governance-MATURITY-EVIDENCE-REGISTRY
- Governance-MERGE-GATE-BOT-CONFIG
- Governance-MERGE-GATE-STATUS-LIVE
- Governance-PHASE3-ENFORCEMENT-RUNBOOK
- Governance-PHASE-1-CLOSURE-REPORT
- Governance-PHASE-CLOSURE-POLICY
- Governance-PHASE-DEPENDENCY-GRAPH
- Governance-PLUGIN-SUBMODULE-ROLLBACK
- Governance-PRODUCTION-READY-2026-DELIVERY-PLAN
- Governance-PR-VERSION-TARGETING
- Governance-PR-VERSION-TARGETING-BACKFILL
- Governance-QUERY-MODULE-STATUS
- Governance-README
- Governance-RELEASE-PROMOTION-GATE-POLICY
- Governance-RELEASE-VALIDATION-CHECKLIST
- Governance-SECURITY-MODULE-5671-EVIDENCE-SUMMARY
- Governance-SHARDING-P6-RESIDUAL-RISK-ACCEPTANCE
- Governance-SOURCECODE-COMPLIANCE-GOVERNANCE
- Governance-UPDATES-DEVELOPMENT-STATUS-SIGN-OFF
- Governance-WAVE-C-IMPLEMENTATION-COMPLETE
- Module-acceleration-Roadmap
- Module-access-model-Roadmap
- Module-ai-Roadmap
- Module-analytics-Roadmap
- Module-api-Roadmap
- Module-aql-Roadmap
- Module-auth-Roadmap
- Module-base-Roadmap
- Module-cache-Roadmap
- Module-cdc-Roadmap
- Module-chaos-Roadmap
- Module-chimera-Roadmap
- Module-config-Roadmap
- Module-content-Roadmap
- Module-core-Roadmap
- Module-distributed-knowledge-Roadmap
- Module-distributed-tensor-Roadmap
- Module-document-Roadmap
- Module-ethics-ai-Roadmap
- Module-evaluation-Roadmap
- Module-execution-Roadmap
- Module-exporters-Roadmap
- Module-failover-Roadmap
- Module-geo-Roadmap
- Module-governance-Roadmap
- Module-gpu-Roadmap
- Module-graph-Roadmap
- Module-image-analysis-Roadmap
- Module-importers-Roadmap
- Module-index-Roadmap
- Module-ingestion-Roadmap
- Module-llama-cpp-Roadmap
- Module-llm-Roadmap
- Module-llm-streaming-Roadmap
- Module-llm-wiki-Roadmap
- Module-maintenance-Roadmap
- Module-metadata-Roadmap
- Module-network-Roadmap
- Module-observability-Roadmap
- Module-onnx-clip-Roadmap
- Module-performance-Roadmap
- Module-plugins-Roadmap
- Module-process-Roadmap
- Module-projects-Roadmap
- Module-prompt-engineering-Roadmap
- Module-query-Roadmap
- Module-rag-Roadmap
- Module-replication-Roadmap
- Module-retrieval-Roadmap
- Module-rpc-grpc-Roadmap
- Module-scheduler-Roadmap
- Module-scraper-Roadmap
- Module-search-Roadmap
- Module-security-Roadmap
- Module-server-Roadmap
- Module-sharding-Roadmap
- Module-stable-diffusion-Roadmap
- Module-storage-Roadmap
- Module-temporal-Roadmap
- Module-tensor-Roadmap
- Module-themis-Roadmap
- Module-timeseries-Roadmap
- Module-toolbox-Roadmap
- Module-training-Roadmap
- Module-transaction-Roadmap
- Module-updates-Roadmap
- Module-user-storage-encrypted-Roadmap
- Module-utils-Roadmap
- Module-vector-search-Roadmap
- Module-voice-Roadmap
- Module-whisper-Roadmap