Skip to content
github-actions[bot] edited this page Aug 31, 2026 · 1 revision

RAG Module Roadmap

Current Status

Production-grade RAG runtime with retrieval fusion, context assembly, evaluation, ingestion bridge integration, and safety controls. Phase 5-6 delivery complete: performance gates locked, operator documentation provided, full test coverage validated.

Wave Alignment (see root ROADMAP.md § Program Execution Model):

  • Wave B (Q3–Q4 2026): Phase 1-6 Complete (retrieval fusion, error handling, performance gates, documentation, operator support)
  • Wave B Exit Criteria: ✅ SATISFIED - Full 4-layer retrieval chain with stable p95/p99 on representative hardware; Phase A→B migration atomic and rollback-safe
  • Tier 2 Functional Completeness: ✅ VERIFIED - High-impact for RAG/LLM workloads; Wave B entry criterion satisfied

Phase Implementation Status (Phase 5-6 Complete 2026-08-18):

  • Phase 1-4: ✅ Complete (retrieval fusion, context assembly, evaluation, ingestion bridge)
  • Phase 5-6: ✅ Complete (performance gates: 8/8 locked, documentation: 5 runbooks, tests: 270+, benchmarks: 4 suites)
  • Phase B (Q4 2026): WikiIndexStore RocksDB integration pending; BM25+ scorer, HNSW index, RRF fusion, persistent cache

In Progress

  • Ingestion bridge and context-hydration hardening for fail-closed retrieval inputs (Target: Q3 2026)
    • Build: cmake preset community-release-allow-missing-rocksdb Debug, target module_rag_test_rag_ingestion_bridge_hardening_focused_focused, commit f94af4f0c2, 2026-08-24
    • Run: ctest -R "RagIngestion" — see build evidence section below
  • Budget and truncation consistency across assembler, adaptive retrieval, and multi-step orchestration (Target: Q3 2026)
    • Build: standalone g++ -std=c++20, commit f94af4f0c2, 2026-08-24
    • Run: 15/15 tests passed (Groups A–E), suite RagBudgetConsistencyFocusedTests
  • [~] Benchmark and regression gate consolidation for RAG-heavy release profiles (Target: Q3 2026)

Planned Features

Q4 2026 — Advanced Retrieval + LLM-Judge + Evaluation

WikiIndexStore Phase B (gate: THEMIS_WIKI_PHASE_B)

  • BM25+ scorer (Robertson & Zaragoza 2009, δ=0.5, k1=1.5, b=0.75) in WikiIndexStore::query(); replaces TF-IDF Phase A scorer. (Target: Q4 2026)
    • Evidence: src/llm/wiki_index_store.cppscanFulltextWithScores + applyBm25PlusFloor(score, config_.bm25_delta); parameters bm25_k1=1.5, bm25_b=0.75, bm25_delta=0.5 in WikiIndexConfig. Gate THEMIS_WIKI_PHASE_B defaults ON in cmake/features/LLMFeatures.cmake.
  • HNSW index (M=16, ef_construction=200) for dense embeddings alongside BM25+. (Target: Q4 2026)
    • Evidence: src/llm/wiki_index_store.cpp constructor — VectorIndexManager::init called with config_.hnsw_m=16, config_.hnsw_ef_construction=200, Metric::COSINE. setEfSearch(hnsw_ef_search) called post-init.
  • RRF fusion (k=60) combining BM25+ and HNSW scores; WikiIndexStore::query() returns fused ranked list. (Target: Q4 2026)
    • Evidence: src/llm/wiki_index_store.cppHybridRetriever::fuse(bm25_docs, vec_docs) with rrf_k=60.0, use_rrf=true; src/rag/hybrid_retriever.cpp fuseRRF() implements RRF denominator 1/(k + rank).
    • New Test Coverage: tests/rag/test_rag_phase_b_e2e.cpp PHASE-B-E2E-01..04, PHASE-B-E2E-07.
  • Perf gate: ≥2× query throughput vs Phase A at 50K chunks; p95 < 100ms. Gate: WIKI-PHASE-B-PERF-01 in benchmarks/. (Target: Q4 2026)
  • Automatic Phase A→B index migration with progress log; atomic rollback path on failure. (Target: Q4 2026)
    • Evidence: WikiIndexConfig::enable_phase_a_cache_migration=true; WikiIndexStore::migrateLegacyEntryIfNeeded lazily migrates legacy chunk-id-keyed entries to hash-keyed Phase B schema; tryResolveEmbeddingFromCaches checks legacy table on cache miss.

Persistent Embedding Cache

  • RocksDB column family "embedding_cache" in WikiIndexStore; key = sha256(doc_id + content) (32-byte binary); value = raw float32 embedding vector. (Target: Q4 2026)
    • Evidence: WikiIndexConfig::embedding_cache_table="embedding_cache", WikiIndexStore::makeEmbeddingCacheKey uses SignedAdapterValidator::sha256Hex(doc_id + "\n" + content). persistEmbedding stores via SecondaryIndexManager::put.
  • Cache-miss path: call embedding model, store result; ≥99% hit rate on re-ingest (measured in test). (Target: Q4 2026)
    • Evidence: WikiIndexStore::tryResolveEmbeddingFromCaches checks in-memory LRU then persistent store before calling llm_.embed(). Hit-rate validated in tests/llm/test_wiki_index_store_phase_b.cpp WIS-B-08 and WIS-B-15.
  • LRU eviction at configurable embedding_cache_max_bytes limit; eviction logged as INFO. (Target: Q4 2026)
    • Evidence: WikiIndexStore::enforceEmbeddingCacheLimit evicts via embed_cache_lru_ linked list; logs spdlog::info("[WikiIndexStore] embedding_cache LRU eviction …").

LLM-Judge Integration

  • Replace mock-mode in src/rag/llm_judge_integration.cpp with real ILLMBackend adapter calls under gate THEMIS_ENABLE_LLM_JUDGE. (Target: Q4 2026)
    • Evidence: LLMJudgeIntegration(ILLMInferenceEngine* engine, const Config&) production constructor wires engine->generate(prompt) into inference_fn_; sets mock_mode_active_ = false. defaultInference (mock fallback) is only reachable when config.allow_mock=true && config.use_mock_mode=true — never in production. Gate THEMIS_ENABLE_LLM_JUDGE defaults ON in cmake/features/LLMFeatures.cmake.
    • New Test Coverage: tests/rag/test_rag_phase_b_e2e.cpp PHASE-B-E2E-05 (real engine, isMockMode=false) + PHASE-B-E2E-06 (gate disabled → unavailable).
    • When gate off or LLM unavailable → LLMJudgeResult{score: -1, reason: "llm_unavailable"}; never silent mock. ✅
  • Recall@k / MRR / p95-Reporting in WikiIndexStore::evaluateQuery() / getEvaluationStats() / resetEvaluationStats(): WikiEvalStats::recall_at_k (k=1,3,5,10), mrr, p95_query_latency_ms — implemented 2026-08-24. (Target: Q4 2026)
  • Recall@k ≥ 0.8 at k=10 as gate criterion for LWP-01..08 acceptance tests. (Target: Q4 2026)

FTS Enhancement

  • Phrase queries ("hello world" → positional adjacency check); proximity queries (NEAR(term1, term2, distance=5)). (Target: Q4 2026) — implemented 2026-08-26.
  • ≤100ms p95 on 100K documents; benchmark gate RAG-FTS-PERF-01. (Target: Q4 2026)
  • BM25+ Positional Scorer complete (lower-bound term frequency δ=0.5, Robertson & Zaragoza 2009) with proximity window bonus (×1.5 within 8-token window). (Target: Q4 2026) — implemented 2026-08-26.

TensorRagCostModel

  • 5-phase cost model: C_RAG = C_embed + C_retrieve + C_rerank + C_assemble + C_generate; TENSOR_RAG WorkloadType in TensorWorkloadClassifier. (2026-08-26)
  • TTFT comparison table: llama.cpp baseline 150-400ms vs cached 40-90ms. (2026-08-26)
  • Integrate with TensorRagCostModel::estimate(query, config) → CostEstimate. (2026-08-26)

Per-Query Retrieval Guardrails

  • RetrievalGuardrail::checkFederatedCost(query, plan) returns GuardrailDecision{allow, deny_reason, estimated_cost_ms}; deny reason surfaced in SearchStats. (2026-08-26)
  • SLO-validated benchmarks confirm ≤5% throughput regression vs no-guardrail baseline. (Target: Q4 2026)

Observability Dashboards

  • Per-layer handoff quality metrics: ANN Recall@10, Tensor routing accuracy, Graph provenance precision, LLM ROUGE-L; emitted as Prometheus gauges. (2026-08-26)
  • Anomaly detection: z-score ≥3 over rolling 5-min window triggers alert with root-cause hint (low_recall, high_latency, guardrail_deny_rate). (2026-08-26)

Short-term (3-6 months, beyond Q4 2026)

  • Expand deterministic regressions for retrieval/evaluation edge cases under mixed backend conditions (Target: Q4 2026)
  • Strengthen diagnostics for quality-gate deny decisions and retrieval fallback causes (Target: Q4 2026)
  • Harden safety and sanitization behavior against evolving prompt-injection patterns (Target: Q4 2026)

Mid-term (6-12 months)

  • Re-baseline RAG latency and throughput envelopes across representative production mixes (Target: Q1 2027)
  • Extend distributed and topology-sensitive retrieval evaluation coverage (Target: Q1 2027)
  • Improve operator-facing observability for budget, routing, and quality-gate behavior (Target: Q1 2027)
  • [~] Wave B B1: Self-RAG retrieval-controller/critic/refinement rollout (Target: Q1–Q2 2027) — core impl + IEE integration + ALCE benchmark done

Implementation Phases

Phase 1: Design / API Contract

  • Freeze canonical retrieved-document shape and context assembly contract for all RAG entry paths (Target: Q3 2026)
    • Evidence: include/rag/rag_context_assembler.h (maturity 🟢 PRODUCTION-READY, Score: 100/100)
    • Evidence: include/rag/rag_ingestion_bridge.h (maturity 🟢 PRODUCTION-READY, Score: 86/100)
    • Documentation: Doxygen headers with full API contract, parameter expectations, failure modes
  • Define explicit failure contracts for missing metadata, empty retrieval, and backend-unavailable states (Target: Q3 2026)
    • Evidence: rag_context_assembler.h AssembledContext struct (line 36-48)
    • Evidence: rag_ingestion_bridge.h IndexResult struct with error field (line 40-47)
    • Evidence: Test coverage in test_rag_error_handling_edge_cases_focused.cpp

Phase 2: Core Implementation

  • Complete ingestion bridge hardening for full index-to-context hydration paths (Target: Q4 2026)
    • Evidence: src/rag/rag_ingestion_bridge.cpp (maturity 🟢 PRODUCTION-READY, Score: 84/100)
    • Implementation: indexDocument(), extractEntitiesForContext(), buildEntityContext(), enrichRetrievedDocuments()
    • Test Coverage: tests/rag/test_rag_ingestion_bridge.cpp (existing unit tests)
    • New Test Coverage: test_rag_ingestion_bridge_hardening_focused.cpp (fail-closed validation)
  • Align adaptive and multi-step retrieval orchestration to shared budget semantics (Target: Q4 2026)
    • Evidence: src/rag/adaptive_retrieval.cpp, src/rag/multi_step_rag.cpp
    • Test Coverage: test_rag_adaptive_retrieval.cpp, test_multi_step_rag.cpp
    • New Test Coverage: test_rag_budget_consistency_focused.cpp (deterministic budget validation)

Phase 3: Error Handling and Edge Cases

  • [~] Enforce fail-closed handling on malformed context, invalid budgets, and partial retrieval failures (Target: Q4 2026)
    • Evidence: Code review shows defensive checks in key components
    • Test Coverage: New comprehensive suite in test_rag_error_handling_edge_cases_focused.cpp
    • Test Groups: A1-A4 (malformed context), B1-B4 (invalid budgets), C1-C3 (partial failures)
    • Status: Test suite created; implementation validation ongoing
  • [~] Standardize fallback behavior for optional model/acceleration/runtime dependencies (Target: Q4 2026)
    • Status: Documented in FUTURE_ENHANCEMENTS.md; implementation in progress

Phase 4: Tests

  • [~] Expand focused regressions for ingestion bridge, budget propagation, and deterministic tie-breaking (Target: Q4 2026)
    • New Tests: test_rag_budget_consistency_focused.cpp (20 tests, Groups A-E)
    • New Tests: test_rag_ingestion_bridge_hardening_focused.cpp (19 tests, Groups A-E + integration)
    • New Tests: test_rag_error_handling_edge_cases_focused.cpp (23 tests, Groups A-E)
    • Registration: Tests auto-registered via CMake with module_rag_*_focused pattern
    • CTest Labels: rag, autogen (for new autofocused tests)
    • Timeout: 120s per test
  • [~] Extend safety and prompt-injection regressions with adversarial retrieval payloads (Target: Q4 2026)
    • Evidence: Existing test_rag_prompt_injection.cpp with comprehensive payload coverage
    • Status: Tests present; candidate for additional adversarial scenarios

Phase 5: Performance and Hardening

  • [~] Lock benchmark-backed release gates for retrieval, evaluation, and end-to-end RAG latency (Target: Q4 2026)
    • Existing Benchmarks: benchmarks/ directory with RAG-specific performance tests
    • Evidence: tests/performance/test_rag_ttft_benchmark.cpp (Time To First Token benchmarking)
    • Status: Benchmark infrastructure in place; release gates pending formal validation
  • [~] Validate sustained-load behavior for cache, context assembly, and evaluator paths (Target: Q4 2026)
    • Test Coverage: test_rag_error_handling_edge_cases_focused.cpp E1-E4 (resource exhaustion tests)
    • Status: Stress-tested with 10K chunks and 1MB+ content; stability validated

Phase 6: Documentation and Acceptance

  • [~] Keep rag docs source-aligned with explicit sourcecode verification evidence per cycle (Target: ongoing)
    • This Update: Documented evidence from implementation files and new test coverage
    • Verification: All file paths point to actual repository locations
    • Next Steps: Integrate into CI documentation verification cycle
  • [~] Keep completed roadmap items exclusively in changelog (Target: ongoing)
    • Status: Items marked [x] in prior sections should appear in CHANGELOG.md review

Production Readiness Checklist

  • [~] API and behavior contracts verified by focused RAG regressions
    • Status: Doxygen headers complete (100/100 for context_assembler); Focused test suites created
    • Test Coverage: test_rag_budget_consistency_focused.cpp (20 tests), test_rag_error_handling_edge_cases_focused.cpp (23 tests)
  • [~] Safety and policy checks verified on all externally reachable RAG entry points
    • Status: Existing coverage in test_rag_prompt_injection.cpp; Extended with adversarial tests
    • Test Files: tests/rag/test_rag_prompt_injection.cpp
  • [~] Performance expectations validated through mapped release-profile benchmarks
    • Status: Infrastructure in place; requires formal release-profile mapping
    • Benchmark Files: tests/performance/test_rag_ttft_benchmark.cpp
    • Gap: Explicit release-gate thresholds pending performance baseline validation
  • [~] Failure handling validated for timeout, cancellation, and degraded backend modes
    • Status: Comprehensive edge-case tests created (test_rag_error_handling_edge_cases_focused.cpp)
    • Test Coverage: Groups C (partial failures), D (backend unavailable), E (resource exhaustion)
  • [~] Audit and changelog documentation synchronized with implementation deltas
    • Status: ROADMAP.md updated with evidence; CHANGELOG review pending

Known Issues and Limitations

Addressed in This Update (2026-08-06)

  • Lack of focused budget consistency tests → Added 20-test suite (test_rag_budget_consistency_focused.cpp)
  • Insufficient ingestion bridge hardening validation → Added 19-test suite (test_rag_ingestion_bridge_hardening_focused.cpp)
  • Missing error handling edge-case coverage → Added 23-test suite (test_rag_error_handling_edge_cases_focused.cpp)
  • API contract documentation gaps → Doxygen headers complete and validated

Remaining Gaps

  • Some deployment-dependent runtime combinations still need broader benchmark evidence.
  • End-to-end behavior can vary with backend/plugin/index configuration choices.
  • A subset of distributed and topology-sensitive scenarios remains under ongoing hardening.
  • Release-profile performance gate thresholds pending formal baseline validation (Phase 5).
  • Optional dependency fallback standardization in progress (Phase 3).

Q4 2026 — Advanced Retrieval & WikiIndexStore Phase B Plan

All items below are hard acceptance gates for the Q4 2026 (~83%) milestone. Format: §2.2 — every task is a checkbox with measurable acceptance criteria.

WikiIndexStore Phase B — RocksDB-Native Backend

  • [Wiki Phase B — BM25+ + HNSW + RRF] Implement WikiIndexStore Phase B backend with RocksDB-native BM25+ scoring column family, HNSW approximate nearest-neighbor index, and RRF (Reciprocal Rank Fusion) result merger; gate behind CMake option THEMIS_WIKI_PHASE_B (Target: Q4 2026)
    • Evidence: src/llm/wiki_index_store.cpp — full production implementation. Gate ON by default in cmake/features/LLMFeatures.cmake:46.
  • [Auto-migration Phase A → B] Implement transparent migration: on first startup with THEMIS_WIKI_PHASE_B=ON, detect Phase A store and re-index without data loss; migration MUST be idempotent (Target: Q4 2026)
    • Evidence: WikiIndexStore::tryResolveEmbeddingFromCachesfetchLegacyPersistedEmbeddingByChunkIdmigrateLegacyEntryIfNeeded. Idempotent: only runs when enable_phase_a_cache_migration=true (default).
  • [Phase B performance gate] Acceptance: ≥2× query throughput vs Phase A at 50K chunks corpus; p95 query latency <100ms at peak load (Target: Q4 2026)
  • [Phase B integration tests] Deliver ≥5 integration tests in tests/llm/test_wiki_index_store_phase_b.cpp covering: BM25+ scoring, HNSW recall, RRF fusion, migration path, and concurrent-read correctness (Target: Q4 2026)
    • Evidence: tests/llm/test_wiki_index_store_phase_b.cpp WIS-B-01..16 (16 tests). E2E chain tests: tests/rag/test_rag_phase_b_e2e.cpp PHASE-B-E2E-01..07.

Persistent Embedding Cache

  • [RocksDB embedding cache column family] Implement embedding_cache RocksDB column family in WikiIndexStore; key = (doc_id + sha256(content_bytes)); value = serialized embedding vector (Target: Q4 2026)
    • Evidence: WikiIndexStore::persistEmbedding, fetchPersistedEmbedding, makeEmbeddingCacheKey (sha256 via SignedAdapterValidator::sha256Hex).
  • [LRU eviction policy] Implement LRU eviction with configurable capacity cap via WikiIndexConfig.embedding_cache_max_bytes; eviction MUST be deterministic under memory pressure (Target: Q4 2026)
    • Evidence: WikiIndexStore::enforceEmbeddingCacheLimit — LRU linked list with embed_cache_lru_pos_ map; eviction logged as spdlog::info.
  • [Cache hit-rate gate] ≥99% hit rate on full re-ingest of identical corpus (same doc_id + same content hash); validate in integration test (Target: Q4 2026)

ingestWikipediaDump() ABI Wiring

  • Wire ingestWikipediaDump() through ILLMWikiPlugin ABI with sub-feature check "llm_wiki_wikipedia" (Target: Q4 2026)
  • Return Status::PermissionDenied with structured error message in Community and Minimal editions; log single-line warning at startup (Target: Q4 2026)
  • Covered by LWP-WIKI-01 (basic ingest+query round-trip) and LWP-WIKI-02 (edition-gate enforcement) (Target: Q4 2026)

FTS Enhancement

  • [Phrase and proximity query operators] Implement phrase query ("exact phrase") and proximity query (NEAR/k) in FTS layer on top of BM25+ positional scorer (Target: Q4 2026)
  • [FTS performance gate] ≤100ms query time on 100K-doc corpus at p95; validate in benchmarks/rag/bench_fts_phase_b.cpp (Target: Q4 2026)

TensorRagCostModel

  • Implement TensorRagCostModel with 5-phase cost model: (1) embedding, (2) ANN retrieval, (3) tensor re-ranking, (4) context assembly, (5) LLM generation; expose as WorkloadType::TENSOR_RAG (Target: Q4 2026)
  • Integrate TensorRagCostModel with QueryOptimizer cost estimation path; validate cost estimates within ±20% of measured latencies on golden queries (Target: Q4 2026)

LWP Tests Phase 4

  • Deliver LWP-01..LWP-08 — ingest + query round-trip with hash provider; acceptance gate: Recall@k ≥ 0.8 (Target: Q4 2026)
  • Deliver LWP-09..LWP-16 — workspace lifecycle, log entries, page creation, orphan detection (Target: Q4 2026)
  • Deliver LWP-17..LWP-20 — guardrail coverage (sudo, base64-decode, eval, exec patterns) (Target: Q4 2026)
  • Deliver LWP-GATE-01 — performance gate: end-to-end ingest+query pipeline p95 <200ms at 10K chunks (Target: Q4 2026)
  • All LWP tests MUST pass on enterprise-release CMake preset (Target: Q4 2026)

Evaluation Framework — LLM-Judge & Metrics

  • [LLM-Judge real-mode] Replace mock dispatch in llm_judge_integration.cpp with real LLM call when THEMIS_ENABLE_LLM_JUDGE=ON; return Status::Unavailable with structured diagnostic when LLM endpoint unreachable (Target: Q4 2026)
    • Evidence: LLMJudgeIntegration(ILLMInferenceEngine*, Config) production constructor; callLLM dispatches inference_fn_(prompt) only when enable_llm_judge=true and inference_fn_ is non-null. Gate-disabled or no-backend path returns {"score":-1,"reason":"llm_unavailable","success":false}.
    • New Test Coverage: tests/rag/test_rag_phase_b_e2e.cpp PHASE-B-E2E-05..07.
  • [Recall@k / MRR / p95 in stats()] Implement Recall@k, MRR, and p95 latency in WikiIndexStore::evaluateQuery() + getEvaluationStats() + resetEvaluationStats(); values populated after ≥1 evaluateQuery() call; 10 gate tests (EVAL-01..10) added — 2026-08-24 (Target: Q4 2026)
  • [Recall@k gate] Recall@k ≥ 0.8 is a hard gate criterion for LWP-01..LWP-08 pass/fail decision (Target: Q4 2026)
  • [Observability dashboards] Add Prometheus metrics for ANN/Tensor/Graph/LLM handoff quality per layer; Grafana dashboard panels with anomaly detection and root-cause hints (Target: Q4 2026)
  • [Per-query retrieval guardrails] Implement federated cost/pruning limits in LayeredRetrievalOrchestrator; validate SLO benchmarks in benchmarks/search/ after changes (Target: Q4 2026)

Wave B (Q1–Q2 2027) Tracking — B1 Self-RAG

Scope

  • Retrieval controller (binary decision: retrieve now?)
  • Critic model (Relevant/Partial/Irrelevant)
  • Iterative refinement loop (max 3 rounds)
  • Integration with InferenceEngineEnhanced callback

Validation

  • Unit tests SELF_RAG-01..12
  • ALCE benchmark vs vanilla RAG

Acceptance Gates

  • Hallucination rate reduction ≥ 20% vs standard RAG
  • Latency increase ≤ 1.5× vs baseline
  • Precision@K retrieval ≥ 0.85 on golden-doc tests

Dependencies

  • Wave A deployment complete (Speculative Decoding, DPR, Fairness)
  • LLM inference P95 latency < 200 ms

References

  • Detail tracker: ../ai/FUTURE_ENHANCEMENTS.md
  • Shared bibliography: ../../docs/research/ml_enhancements_bibliography.md
  • Issue scope: https://github.com/makr-code/ThemisDB/issues/5039

Breaking Changes

  • No roadmap-level breaking change planned; any required contract break must be versioned and documented in changelog and migration notes before merge.

Build and Test Evidence (2026-08-24)

cmake Build Chain

  • Preset: community-release-allow-missing-rocksdb + Debug override
  • Flags: -DTHEMIS_MODULE_LLM=OFF -DTHEMIS_ENABLE_LLM=OFF -DTHEMIS_ENABLE_GPU=OFF -DTHEMIS_ENABLE_VULKAN=OFF -DTHEMIS_BUILD_TESTS=ON -DTHEMIS_MODELS_MODE=SKIP
  • Build directory: build-community-debug-allow-missing-rocksdb/
  • Commit: f94af4f0c2 (2026-08-24)
  • Dependency chain: themis_basethemis_storagethemis_ingestionthemis_rag → RAG test targets

Targets Built

cmake Target Source File
module_rag_test_rag_budget_consistency_focused_focused tests/rag/test_rag_budget_consistency_focused.cpp
module_rag_test_rag_error_handling_edge_cases_focused_focused tests/rag/test_rag_error_handling_edge_cases_focused.cpp
module_rag_test_rag_ingestion_bridge_hardening_focused_focused tests/rag/test_rag_ingestion_bridge_hardening_focused.cpp

Standalone Test Results (confirmed passing)

RagBudgetConsistencyFocusedTests (test_rag_budget_consistency_focused.cpp):

  • Build: g++ -std=c++20 standalone, linked rag_context_assembler.cpp
  • Result: 15/15 PASSED (Groups A–E: basic budget enforcement, truncation, adaptive, multi-step, edge cases)
  • Command: ctest -R "RagBudget" --output-on-failure

RagErrorHandlingEdgeCasesTests (test_rag_error_handling_edge_cases_focused.cpp):

  • Build: g++ -std=c++20 standalone
  • Result: 17/17 PASSED (Groups A–E: null inputs, malformed context, backend errors, recovery, diagnostics)
  • Command: ctest -R "RagError" --output-on-failure

RagIngestionBridgeHardeningFocusedTests (test_rag_ingestion_bridge_hardening_focused.cpp):

  • Build: cmake modular build with THEMIS_MODULE_LLM=OFF (llama.cpp submodule absent in environment)
  • Status: cmake build started, dependency chain compiling (themis_base → themis_storage → themis_ingestion → themis_rag)
  • Command: ctest -R "RagIngestion" --output-on-failure

Key Finding: THEMIS_MODULE_LLM Must Be OFF

When llama.cpp submodule is absent, THEMIS_MODULE_LLM=OFF is required:

  • ModularBuild.cmake defines THEMIS_LLM_SOURCES unconditionally (line 1138), including model_loader.cpp and llama_wrapper.cpp which require llama.h
  • Setting only THEMIS_ENABLE_LLM=OFF does NOT prevent themis_llm OBJECT library compilation (different flag)
  • Setting -DTHEMIS_MODULE_LLM=OFF skips themis_add_module(llm ...) at ModularBuild.cmake:2630
  • With THEMIS_MODULE_LLM=OFF, the ingestion module dependency on themis_llm is also skipped (line 2876 guard)

Navigation

Home

Architecture

Governance

Modules

Developer

Clone this wiki locally