Module ai Roadmap

AI Module Roadmap

Current Status

Production runtime exists for prompt validation, endpoint invocation, JSON mapping, and structured fail-closed error handling.

Module Documentation Enhancements (2026-07-19)

• Added comprehensive Doxygen documentation to all implementation functions
• Documented AIPluginGenerator::validatePrompt() with detailed validation rules
• Documented AIPluginGenerator::generatePlugin() with complete execution pipeline
• Documented all CAI ethics integration functions with semantics and contracts
• Added inline documentation for error handling, thread-safety, and retry policies
• Lines added: ai_plugin_generator.cpp +133, cai_ethics_integration.cpp +130
• Total module documentation lines: ~1,473 (up from 1,210)

In Progress

• Validation hardening for non-description prompt fields (Target: Q3 2026)
• Endpoint safety hardening (allow-list, response-size limits) (Target: Q3 2026)
• Performance gate consolidation for AI generation proxy benchmarks (Target: Q3 2026)

Planned Features

Short-term (3-6 months)

• Enforce schema-level validation for all generated payload fields (Target: Q4 2026)
• Introduce deterministic retry/backoff policy for transient endpoint failures (Target: Q4 2026)
• Add explicit redaction policy for diagnostic output fields (Target: Q4 2026)

Mid-term (6-12 months)

• Integrate optional sandbox verification gate for generated code artifacts (artifact materialization + optional callback verification enforced in generator path) (Target: Q1 2027)
• Add dedicated benchmark target for AI plugin generation path (Target: Q1 2027)
• Expand observability counters for error classes and endpoint quality signals (Target: Q1 2027)

Long-term (Q3 2027+)

• Wave C C1: Constitutional AI (CAI) safety module with 21 built-in principles, critic-revision loop, EthicsEvaluator integration, and CAI-01..15 + CAI-BENCH-01 coverage — include/ai/cai_ethics_integration.h, tests/test_cai_safety_module.cpp
• Wave C C2: Federated learning coordinator with secure aggregation, Byzantine-robust averaging, DP tuning, and FEDERATED-01..15 + FEDERATED-BENCH-01 coverage — tests/test_federated_privacy_training.cpp
• Human safety benchmark program for C1 (500 samples, 3 annotators) and convergence benchmark for C2 (10-node setup) — tests/test_cai_safety_module.cpp (CAI-BENCH-01), tests/test_federated_privacy_training.cpp (FEDERATED-BENCH-01)
• Integrate optional sandbox verification gate for generated code artifacts (Target: Q1 2027)
• Add dedicated benchmark target for AI plugin generation path (Target: Q1 2027)
• Expand observability counters for error classes and endpoint quality signals (Target: Q1 2027)
• Wave B B1: Self-RAG design/implementation/benchmark package (Target: Q1–Q2 2027) — core impl + IEE integration + ALCE acceptance-gate coverage done
• Wave B B2: RotatE knowledge-graph completion integration package (Target: Q1–Q2 2027) — core impl + KGC-01..15 tests + TransE-baseline acceptance coverage done
• Wave B B3: Multi-task LoRA fine-tuning package (Target: Q1–Q2 2027) — core impl + ablation/benchmark + acceptance-gate coverage done

Implementation Phases

Phase 1: Design / API Contract

• Stable API for prompt/config/result types in public header
• Validation-first behavior contract defined and implemented

Phase 2: Core Implementation

• Endpoint invocation path implemented with configurable transport
• JSON response mapping to GeneratedPlugin implemented

Phase 3: Error Handling and Edge Cases

• Non-2xx, transport, and parse failures normalized to structured errors
• Extended validation for capability/dependency fields (Target: Q3 2026)

Phase 4: Tests

• Focused unit coverage for constructor, validation, and endpoint/error paths
• Integration suite with deterministic endpoint fixtures (Target: Q3 2026)

Phase 5: Performance and Hardening

• Add module-specific benchmark instead of proxy-only tracking (Target: Q1 2027)
• Enforce endpoint allow-list and payload size bounds (Target: Q4 2026)

Phase 6: Documentation and Acceptance

• Core module docs aligned with source-verifiable behavior
• Completed work tracked in changelog; roadmap remains forward-looking

Production Readiness Checklist

• Validation-first execution path documented and verified
• Structured error handling for endpoint and parse failures verified
• Proxy benchmark mapping documented in performance expectations
• Dedicated benchmark target registered
• Hardening follow-ups closed for endpoint safety controls

Known Issues and Limitations

• Wave C C1/C2 production-runtime integration now covers AIPluginGenerator and LLMAQLHandler (executeInfer, executeInferStreaming, executeRAG, executeChat) via opt-in safety-gate and telemetry hooks.
• Schema-level validation for all LLM output fields is enforced: code fields ≤ 1 MiB, security_report ≤ 64 KiB, version ≤ 64 chars (defaults to 0.1.0), manifest.description truncated at 8192 chars, oversized build_dependencies entries silently dropped.

Wave C Timeline and Dependencies

• Start: Q3 2027 (early July)
• Target: End Q4 2027 (mid-December)
• Estimated Effort: 16–24 weeks total (depending on federated security requirements)

Dependencies

• Wave A + Wave B stability checks tracked in focused regression suites and release-gate docs (CTEST.md, Wave A #5038, Wave B #5039)
• Constitutional AI principles formalized in ethics framework (src/ai/cai_ethics_integration.cpp, tests/test_cai_safety_module.cpp)
• Multi-node federated benchmark infra/security review tracking established (FEDERATED-BENCH-01 coverage + issue traceability #5040/#5039)

References

• src/ai/FUTURE_ENHANCEMENTS.md#wave-c--strategic-ml-enhancements-q3-2027
• docs/research/ml_enhancements_bibliography.md
• #5040 (Wave C Issue)
• #5038 (Wave A Issue)
• #5039 (Wave B Issue)

Wave C Research Publication Opportunity

• Joint paper: ThemisDB Integration of Research-Backed ML Features
• Target venue window: MLSys 2028 / FAccT 2028
• Dedicated benchmark coverage exists via benchmarks/bench_ai_plugin_generator.cpp / benchmarks/ai/bench_ai_plugin_generator.cpp.
• Advanced field-level prompt validation remains incomplete.
• Sandbox artifact materialization and optional callback verification are enforced when enable_sandbox_gate is enabled; external sandbox engines remain deployment-specific.
• Wave B ML enhancement implementation and acceptance-gate coverage are complete; production promotion remains gated on Wave A deployment completion and latency prerequisites.

Wave B (Q1–Q2 2027) Tracking

B1: Self-RAG (Self-Retrieving, Auto-Critique)

• Retrieval controller (binary classify: retrieve now?)
• Critic model (Relevant/Partial/Irrelevant)
• Iterative refinement loop (max 3 rounds)
• Unit tests SELF_RAG-01..12
• InferenceEngineEnhanced callback integration
• ALCE benchmark vs vanilla RAG

B2: Knowledge Graph Completion (RotatE)

• RotatE embedding model (relation-as-rotation)
• Triple loss with negative sampling
• Link-prediction head
• Unit tests KGC-01..15
• TransE baseline benchmark
• KnowledgeGraphReasoner integration

B3: Multi-Task LoRA Fine-Tuning

• Shared LoRA base + task-specific projections
• Domain-gating mechanism
• Joint loss with configurable task weighting
• Unit tests MTL-01..10
• Shared-vs-single-task baseline ablation
• 3-task benchmark evaluation

Acceptance Gates

• Hallucination rate reduction ≥ 20% vs standard RAG
• Self-RAG latency increase ≤ 1.5× vs baseline
• Precision@K retrieval ≥ 0.85 on golden-doc tests
• RotatE MRR ≥ 0.35 and Hits@10 ≥ 0.55 on deterministic acceptance fixture
• RotatE inference latency ≤ 50 ms for top-20 predictions
• Multi-task LoRA average task performance ≥ +8% vs single-task
• Multi-task LoRA training time increase ≤ 15%

Dependencies

• Wave A deployment complete (Speculative Decoding, DPR, Fairness)
• LLM inference P95 latency < 200 ms
• KnowledgeGraphReasoner stable + benchmark suite passing

References

• Research bibliography: ../../docs/research/ml_enhancements_bibliography.md
• Future enhancements detail: FUTURE_ENHANCEMENTS.md
• Issue scope: https://github.com/makr-code/ThemisDB/issues/5039

Module Validation Evidence (2026-07-19)

Documentation Status

• @file Doxygen Headers: 100% coverage (4/4 files)

  • include/ai/ai_plugin_generator.h — hardened implementation metadata
  • include/ai/cai_ethics_integration.h — hardened implementation metadata
  • src/ai/ai_plugin_generator.cpp — hardened implementation metadata
  • src/ai/cai_ethics_integration.cpp — hardened implementation metadata

• Function/Method Documentation: Enhanced 2026-07-19

  • Added comprehensive Doxygen comments to all public methods
  • Added parameter/return/error documentation to key functions
  • Documented thread-safety contracts and error handling semantics
  • Implementation files received the primary Doxygen expansion (+263 lines across 2 .cpp files)
  • Header files received follow-up contract clarifications for transport, validation, and latency semantics

Code Quality Metrics

• Lines of Code:

  • Source: 946 lines (ai_plugin_generator.cpp: 634, cai_ethics_integration.cpp: 312)
  • Headers: 476 lines (ai_plugin_generator.h: 283, cai_ethics_integration.h: 193)
  • Total: 1,422 lines (module core)

• API Stability:

  • Public API stable (AIPluginGenerator, CAIEthicsIntegration)
  • Config structures fully documented with field semantics
  • Callback types fully documented with signatures and contract details
  • Zero breaking changes in current implementation

Feature Coverage

• ✅ Phase 1: Stable API for prompt/config/result types
• ✅ Phase 2: Endpoint invocation with configurable transport
• ✅ Phase 3: Structured error handling and edge cases
• ✅ Phase 4: Focused unit test coverage (2 test files: test_ai_decision_auditor.cpp, test_ai_plugin_generator.cpp)
• ✅ Phase 5: Observable counters implemented (Stats struct with 7 counters)
• ✅ Phase 6: Documentation aligned with implementation

Hardening Completeness

• ✅ Prompt validation (description length, token list sizes, format validation)
• ✅ Request sanitization (ASCII control character stripping)
• ✅ Endpoint allow-list enforcement
• ✅ Request/response size limits (256 KiB / 8 MiB)
• ✅ Retryable endpoint invocation (3 attempts, exponential backoff)
• ✅ Response parsing with malformed input rejection
• ✅ Output field validation (code size, manifest fields)
• ✅ Optional C1 CAI safety gate (Wave C feature)
• ✅ Optional sandbox artifact materialization + callback verification enforced in the generator path
• ✅ Optional C2 federated telemetry (Wave C feature)
• ✅ Observability counters for error classes

Production Readiness Status

• Maturity: 🟡 Hardened implementation (all HIGH-severity gaps reviewed; full production validation still pending environment-complete build/test)
• Configuration: Validatable via CMakeLists.txt, CMakePresets.json
• Testing: Focused test targets auto-discovered (module_ai_*_focused.exe)
• Error Handling: Fail-closed with structured Error result types
• Logging: Redaction-aware with configurable truncation (120 chars max)
• Thread Safety: Document-specified (not thread-safe for concurrent generatePlugin)
• Memory Safety: RAII compliance, smart pointers for ownership
• Security: Input validation, output bounds checking, endpoint allow-listing

Known Limitations (MODULE_GAPS.md reference)

• HIGH-severity scanner findings were reviewed and either remediated or reclassified with source-backed explanations (2026-07-19)
• MEDIUM-priority scope/documentation gaps remain tracked in MODULE_GAPS.md
• Dedicated benchmark target for the AI plugin generator path is registered and tracked in benchmark docs
• Sandbox verification now includes built-in artifact materialization, read-back verification, and optional callback enforcement

Acceptance Criteria Status

• Validation-first execution path implemented and verified
• Structured error handling for all failure points
• Hardening follow-ups complete for endpoint safety, payload validation, observability counters, benchmark coverage, and sandbox artifact verification
• API documentation complete and comprehensive
• Implementation semantics documented at function level
• Roadmap and Future Enhancements synchronized (2026-07-19)
• All HIGH-severity gaps reviewed with either code remediation or source-backed disposition notes (2026-07-19)

Breaking Changes

No breaking API change planned. Any signature/semantic contract change requires explicit migration notes and changelog entry.