Skip to content

Harden ggml/LLM integration paths and remove simulation-backed LLM fallbacks - #6089

Merged
makr-code merged 18 commits into
developfrom
copilot/core-modules-gaps-analysis-again
Aug 31, 2026
Merged

makr-code merged 18 commits into
developfrom
copilot/core-modules-gaps-analysis-again

Conversation

Copilot AI commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

This PR advances the next storage/LLM block by wiring the ggml TT type registration into the embedded LLM startup path and tightening LLM runtime behavior where production flows still degraded into heuristics or simulation-backed execution. It also updates the source-of-truth module status docs to match the new runtime state.

  • Storage / ggml bootstrap

    • Register GGML_TYPE_TT once during EmbeddedLLM startup under THEMIS_ENABLE_GGML_BRIDGE
    • Use std::call_once so TT registration is process-wide and deterministic
    • Narrow the remaining storage gap to allocator/prefetch hook wiring instead of leaving type registration implicit
  • Speculative decode fallback cleanup

    • In inference_engine_enhanced.cpp, stop converting remote speculative draft text into byte-modulo token IDs when no tokenizer bridge is available
    • Retry the local draft model instead of silently degrading into heuristic tokenization
    • Broaden tokenizer bridge reuse so local draft execution can auto-use llama.cpp-backed tokenization from draft or target plugins
  • Distributed training hardening

    • In distributed_training_coordinator.cpp, remove fabricated gradient/health behavior when no ShardRouter transport exists
    • Fail closed for gradient collection/broadcast without transport
    • Mark shards unavailable instead of reporting simulated healthy progress
  • Production validator hardening

    • In production_validator.cpp, require a real inference engine for stress runs
    • Remove synthetic local stress-response generation from the production validation path
    • Keep the “no engine attached” state explicit instead of blending disabled validation with successful execution
  • Focused test coverage

    • Extend speculative bridge coverage for tokenizer-bridge-driven local draft execution
    • Add validator coverage for the fail-closed stress-test contract when no inference engine is attached
    • Mirror the validator change in legacy coverage
  • Docs / source-validated status

    • Update src/llm/ROADMAP.md to reflect the closed speculative and fail-closed validator/training slices
    • Update src/storage/ROADMAP.md and src/MODULE_GAP_ANALYSIS_WAVE2.md so storage/LLM gap tracking matches the current source state

Example of the new fail-closed contract in the validator path:

if (!inference_engine_) {
    result.passed = false;
    result.error_message =
        "Stress test requires an attached inference engine; synthetic local fallback is disabled.";
    return result;
}

Copilot AI and others added 10 commits August 31, 2026 06:00
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

There are correctness/safety issues in the new/modified hardening code paths (notably ggml buffer copying assumptions and overly strict Postgres bound-parameter control-character rejection) that should be addressed before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR hardens several production execution paths by removing simulation/heuristic fallbacks across LLM speculative decoding, distributed training coordination, validator stress testing, shard RPC routing, and storage restore/ggml bridging, with accompanying test and roadmap/status updates to reflect the new fail-closed behavior.

Changes:

  • Harden storage + ggml integration by registering GGML_TYPE_TT once during EmbeddedLLM startup and mapping TT tensors into real ggml allocations when a ggml_context* is provided.
  • Remove simulation/heuristic fallbacks: speculative remote draft text no longer degrades into byte-modulo token IDs; distributed training and validator stress test paths now fail-closed when required transports/engines are absent.
  • Expand/adjust tests and update module and root roadmap/status documents to match the newly enforced runtime contracts.
File summaries
File Description
tests/test_tensor_phase3.cpp Updates TARGRetrieval entropy test expectations to match exact entropy path.
tests/tensor/test_tensor_phase3.cpp Same entropy expectation updates for tensor test suite variant.
tests/test_speculative_draft_fn_bridge.cpp Adds SPEC-TL-05 coverage for tokenizer bridge overriding heuristic in local draft path.
tests/legacy/speculative/test_speculative_draft_fn_bridge.cpp Mirrors SPEC-TL-05 coverage in legacy suite.
tests/test_shard_rpc_grpc.cpp Switches unit tests to explicit inproc:// endpoints for in-process routing.
tests/legacy/shard/test_shard_rpc_grpc.cpp Mirrors explicit inproc:// endpoints for legacy shard RPC tests.
tests/test_production_validator.cpp Adds fail-closed coverage when stress test runs without an inference engine.
tests/legacy/production/test_production_validator.cpp Mirrors validator fail-closed stress test coverage in legacy suite.
tests/test_new_aql_functions.cpp Extends AQL function tests for new ethics/process-mining behaviors.
tests/test_distributed_txn_api_handler.cpp Asserts explicit error status when shard endpoints are missing (fail-closed read-only path).
src/utils/input_validator.cpp Introduces validateJsonSchema() and routes critical paths to schema validation (fail-closed).
include/utils/input_validator.h Adds validateJsonSchema() declaration and keeps validateJsonStub() alias for compatibility.
src/server/http_server.cpp Migrates selected routes from validateJsonStub() to validateJsonSchema().
src/server/postgres_session.cpp Tightens prepared-statement literal binding with typed parsing and placeholder-safe replacement.
src/storage/ggml_tensor_bridge.cpp Uses ggml_context* in map path and copies decompressed TT data into ggml tensor when available.
src/storage/backup_manager.cpp Makes decompress/decrypt restore paths fail-closed when required dependencies are absent.
src/storage/ROADMAP.md Updates storage roadmap status to reflect fail-closed restore and ggml TT wiring progress.
src/sharding/shard_rpc_client.cpp Makes in-process routing explicit (inproc:// / loopback://) rather than implicit loopback detection.
src/sharding/distributed_transaction.cpp Validates shard endpoints and fails closed when endpoints are missing/placeholder.
src/sharding/cloud_sdk_integration.cpp Hardens cloud callback initialization by validating fixed provider arguments and required config.
src/sharding/cloud_backup.cpp Attempts SDK callback bootstrap when callbacks are missing and fails closed if still unavailable.
src/sharding/ROADMAP.md Updates sharding roadmap to reflect callback bootstrap + endpoint validation hardening.
src/server/ROADMAP.md Notes remaining server runtime gaps and marks request-validation schema hardening delivered.
src/rag/targ_retrieval.cpp Switches default entropy computation to full-vocabulary softmax entropy (exact).
include/rag/targ_retrieval.h Updates API docs to match built-in full-vocabulary entropy behavior.
src/rag/ROADMAP.md Updates RAG roadmap status to reflect exact entropy behavior.
src/llm/inference_engine_enhanced.cpp Adds tokenizer-bridge reuse (llama-backed) and removes remote byte-modulo fallback by retrying local draft.
src/llm/llama_wrapper.cpp Adds tokenizeForBridge() to expose llama.cpp tokenizer for bridge usage.
include/llm/llama_wrapper.h Declares and documents tokenizeForBridge() bridge API.
src/llm/embedded_llm.cpp Registers ggml TT type once during embedded LLM startup under THEMIS_ENABLE_GGML_BRIDGE.
src/llm/distributed_training_coordinator.cpp Removes simulated gradients/health behavior; marks shards unavailable and fails closed without transport.
src/llm/production_validator.cpp Requires attached inference engine for stress tests; removes synthetic fallback responses.
src/llm/ROADMAP.md Updates LLM roadmap to reflect closed simulation-backed paths and remaining residual fallback debt.
src/llm/QUICK_STATUS_BOARD.txt Marks quick board as historical snapshot superseded by roadmap.
src/query/functions/process_mining_functions.cpp Replaces process-mining stub behaviors with context/scanner-backed implementations.
src/query/functions/ethics_functions.cpp Replaces ethics stub behaviors with collection/graph/vector-backed execution over FunctionContext.
src/query/ROADMAP.md Updates query roadmap to reflect delivered ethics + process-mining closures.
src/MODULE_GAP_ANALYSIS_WAVE2.md Updates wave gap analysis to reflect revalidation and corrected prioritization.
ROADMAP.md Updates root roadmap “Last Updated” date and refreshes module status classification text.
Review details
  • Files reviewed: 39/39 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines 290 to 294
"(Subsequent occurrences for this schema are suppressed.)",
schema_name, schema_dir_, schema_name);
}

}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit: the warning message now reads InputValidator::validateJsonSchema instead of validateJsonStub.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in c97b478: the missing-schema warning now correctly references InputValidator::validateJsonSchema.

Comment on lines +316 to +323
#ifdef THEMIS_HAS_GGML
if (handle.impl_->real_ggml_tensor && handle.impl_->real_ggml_tensor->data &&
!handle.impl_->fake_tensor.data.empty()) {
std::memcpy(handle.impl_->real_ggml_tensor->data,
handle.impl_->fake_tensor.data.data(),
handle.impl_->fake_tensor.data.size() * sizeof(float));
}
#endif

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit. Before memcpy, the code now validates that:

  1. The tensor type is GGML_TYPE_F32
  2. The allocated capacity (ne[0] * sizeof(float)) is at least as large as the data to copy

If either check fails, the memcpy is skipped, an error is logged, and the handle is returned invalid.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in c97b478: added type/capacity guards before ggml memcpy so incompatible allocator outputs are rejected instead of copied.

Comment on lines +56 to +60
bool containsControlCharacters(const std::string& input) {
return std::any_of(input.begin(), input.end(), [](unsigned char c) {
return c == '\0' || std::iscntrl(c);
});
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit. containsControlCharacters now exempts \t, \n, and \r from the control-character check, so valid bound text parameters containing tabs/newlines/carriage-returns are accepted.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in c97b478: control-character validation now allows tab/newline/carriage-return for bound text parameters while still rejecting unsafe control bytes.

Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Copilot AI and others added 2 commits August 31, 2026 11:31
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
@makr-code
makr-code marked this pull request as ready for review August 31, 2026 13:07
@github-actions github-actions Bot added type:documentation Documentation change type:test Test-only change area:llm LLM / prompt engineering subsystem area:storage Storage subsystem area:sharding Sharding subsystem type:refactor Code refactoring without functional change area:search Search / retrieval / RAG subsystem wave:A Wave A module: access_model, search, sharding, replication, utils, updates, process, audit release_critical Change in a release-critical module; requires sign-off wave:B Wave B module: acceleration, llm, retrieval, tensor, gpu quality/doxygen-failed Doxygen governance gate failed on changed source code labels Aug 31, 2026
@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Target: pr

Doxygen Governance Gate

  • Verdict: FAIL
  • Base branch: develop
  • Changed C/C++ files: 49
  • Scoped modules: llama_cpp, llm, query, rag, server, sharding, storage, utils
  • Release lane: no
  • Phase 6 modules in scope: none
  • Coverage enforced: no
  • Coverage threshold: 95.0%
  • Coverage result: skipped
  • Coverage waiver active: no
  • Approved waivers: none
  • Structural findings: 90
  • Advisory findings: 0
  • Doxygen warnings: 12
  • XML generated: yes

Changed source files

  • benchmarks/llm/bench_llm_judge_integration.cpp
  • include/llama_cpp/llama_cpp_plugin.h
  • include/llm/gpu_memory_manager.h
  • include/llm/inference_engine_enhanced.h
  • include/llm/llama_wrapper.h
  • include/rag/llm_judge_integration.h
  • include/rag/response_parser.h
  • include/rag/targ_retrieval.h
  • include/storage/ggml_tensor_bridge.h
  • include/utils/input_validator.h
  • src/llama_cpp/llama_cpp_plugin.cpp
  • src/llm/distributed_training_coordinator.cpp
  • src/llm/embedded_llm.cpp
  • src/llm/gpu_memory_manager.cpp
  • src/llm/inference_engine_enhanced.cpp
  • src/llm/llama_wrapper.cpp
  • src/llm/production_validator.cpp
  • src/query/functions/ethics_functions.cpp
  • src/query/functions/process_mining_functions.cpp
  • src/rag/llm_judge_integration.cpp
  • ... 29 more

Blocking structural findings

  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:97 — Public declaration 'allocateGPU' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:101 — Public declaration 'allocateGPU' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:104 — Public declaration 'freeGPU' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:105 — Public declaration 'freeCPU' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:108 — Public declaration 'freeModel' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:111 — Public declaration 'freeModel' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:128 — Public declaration 'canAllocate' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:132 — Public declaration 'defragment' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:184 — Public declaration 'markGPUUnhealthy' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:41 — Class 'GPUMemoryManager' is missing a Doxygen comment
  • include/llm/gpu_memory_manager.h:185 — Public declaration 'markGPUHealthy' is missing a Doxygen comment
  • ... 70 more

Doxygen warnings

  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:72: warning: @copybrief or @copydoc target 'llm::ILLMPlugin::unloadLoRA' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:69: warning: @copybrief or @copydoc target 'llm::ILLMPlugin::loadLoRA' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:74: warning: @copybrief or @copydoc target 'llm::ILLMPlugin::listLoRAs' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:69: warning: @copybrief or @copydoc target 'llm::ILLMPlugin::loadLoRA' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:72: warning: @copybrief or @copydoc target 'llm::ILLMPlugin::unloadLoRA' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:74: warning: @copybrief or @copydoc target 'llm::ILLMPlugin::listLoRAs' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:69: warning: @copybrief or @copydoc target 'llm::ILLMPlugin::loadLoRA' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:70: warning: @copydetails or @copydoc target 'llm::ILLMPlugin::loadLoRA' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:72: warning: @copybrief or @copydoc target 'llm::ILLMPlugin::unloadLoRA' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:73: warning: @copydetails or @copydoc target 'llm::ILLMPlugin::unloadLoRA' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:74: warning: @copybrief or @copydoc target 'llm::ILLMPlugin::listLoRAs' not found
  • /home/runner/work/ThemisDB/ThemisDB/include/llama_cpp/llama_cpp_plugin.h:75: warning: @copydetails or @copydoc target 'llm::ILLMPlugin::listLoRAs' not found

source_workflow: Gate: PR Doxygen Governance
source_run: 33395862865
source_sha: a7e225d

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

✅ Wave C Policy Gate: Edition & License Validation

Result: ✅ Edition & License Validation PASSED
Workflow run: #33395863265
Triggered by: @Copilot

Checks

  • ✅ Edition matrix consistency check
  • ✅ License feature gate validation
  • ✅ Enterprise/military marker detection
  • ℹ️ Target branch: develop

See .github/workflows/gate-pr-edition-license.yml for details.

@github-actions github-actions Bot added this to the Backlog milestone Aug 31, 2026
@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

✅ Wave C Policy Gate: Community Fail-Closed Validation

Result: ✅ Community Fail-Closed Validation PASSED
Workflow run: #33395863265
Triggered by: @Copilot

Checks

  • ✅ No silent fallback patterns
  • ✅ License disclaimers present
  • ✅ No unguarded telemetry/tracking
  • ✅ No private plugin references in community/minimal scope
  • ✅ Community-only build config validation
  • ✅ No enterprise secrets leakage

See .github/workflows/gate-pr-community-failclosed.yml for details.

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

✅ Wave C Policy Gate: Hash & SBOM Integrity

Result: ✅ Hash & SBOM Integrity PASSED
Workflow run: #33395863265
Triggered by: @Copilot

Checks

  • ✅ Dependency hash integrity check
  • ✅ SBOM generation and registry consistency
  • ✅ Edition-correct SBOM composition
  • ✅ Private plugin variance detection
  • ℹ️ Approved SBOM registry: docs/governance/SBOM_APPROVED_VERSIONS.md

See .github/workflows/gate-pr-hash-sbom.yml for details.

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

✅ Wave C Policy Gate: Private Plugin Boundary Enforcement

Result: ✅ Private Plugin Boundary Enforcement PASSED
Workflow run: #33395863265
Triggered by: @Copilot

Checks

  • ✅ Private plugin leakage detection
  • ✅ Plugin manifest visibility validation
  • ✅ Submodule commit-pin enforcement (no branch refs for private)

See .github/workflows/gate-pr-plugin-boundary.yml for details.

Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Copilot AI requested a review from makr-code August 31, 2026 13:21
@github-actions github-actions Bot modified the milestones: Backlog, Q4 2026 Aug 31, 2026
@makr-code
makr-code merged commit 949b88e into develop Aug 31, 2026
48 of 52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:llm LLM / prompt engineering subsystem area:search Search / retrieval / RAG subsystem area:sharding Sharding subsystem area:storage Storage subsystem quality/doxygen-failed Doxygen governance gate failed on changed source code release_critical Change in a release-critical module; requires sign-off type:documentation Documentation change type:refactor Code refactoring without functional change type:test Test-only change wave:A Wave A module: access_model, search, sharding, replication, utils, updates, process, audit wave:B Wave B module: acceleration, llm, retrieval, tensor, gpu

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants