# ThemisDB
**High-performance multi-model database with native AI/LLM integration**
[](CHANGELOG.md)
[](docs/de/legal/license.md)
[](ROADMAP.md)
[](ROADMAP.md)
[](CONTRIBUTING.md)
[](docs/en/badges/loc.md)
[](docs/en/badges/loc.md)
[[๐ Documentation|Home]] ยท [[๐ Quick Start|QUICKSTART]] ยท [[๐ ๏ธ Setup|SETUP]] ยท [[โ ๏ธ Status|ROADMAP]] ยท [[๐ Audit Hub|AUDIT]] ยท [[๐ Support|SUPPORT]] ยท [[Release Notes|CHANGELOG]]
## Usage
## Note to AI vibe coding
The code in this repository was partially or fully generated by an AI tool (e.g., AI-assisted coding, vibe coding, or automated code generation). While efforts were made to ensure correctness and functionality, AI-generated code may: Lack modular consistency: Cross-module dependencies, naming conventions, or architectural patterns may be inconsistent or suboptimal. Contain subtle bugs: Logic errors, edge-case oversights, or inefficient patterns may persist.
Require manual review: Human oversight is essential for refactoring, testing, and alignment with project standards.
Be outdated: AI models may not reflect the latest best practices, libraries, or security standards.
Use this code at your own risk. Contributions, feedback, and improvements are welcome to address these limitations.
---
## โ ๏ธ IMPORTANT: Module Status Snapshot (72/72)
**This is an active development project.** Current synchronized status snapshot (source-based):
- โ
**19 modules** are `PRODUCTION_CANDIDATE`
- ๐ก **48 modules** are `HARDENING`
- ๐ด **2 modules** are `EXPERIMENTAL` (`llama_cpp`, `stable_diffusion`)
- โช **3 modules** are `THIN/PLACEHOLDER` (`ai_working`, `distributed_tensor`, `retrieval`)
### ๐ข Graph Module Status Snapshot
**Status:** โ
**PRODUCTION_CANDIDATE** โ L0 verification reports **0 real gaps** and current work is Phase 3 hardening / optimizer follow-up.
**Current focus:** GPU/distributed traversal parity, benchmark stabilization, diagnostics consistency, and hybrid retrieval rollout hardening.
**Canonical source:** [[src/graph/ROADMAP.md|ROADMAP]] and [[ROADMAP.md|ROADMAP]]
**See [[ROADMAP.md|ROADMAP]] for the full 72-module table.**
Documentation source precedence and update cadence are defined in [[DOCUMENTATION_GOVERNANCE.md|DOCUMENTATION-GOVERNANCE]].
Evidence artifacts:
- [[ROADMAP.md|ROADMAP]]
- [benchmarks/wave7/release_gate_manifest_w7.json](benchmarks/wave7/release_gate_manifest_w7.json)
- [[tests/integration/WAVE5_TEST_COVERAGE.md|WAVE5-TEST-COVERAGE]]
- [[tests/integration/WAVE6_TEST_COVERAGE.md|WAVE6-TEST-COVERAGE]]
- [[src/auth/ROADMAP.md|ROADMAP]]
- [[docs/security/GA_SANITIZER_EVIDENCE_BUNDLE.md|GA-SANITIZER-EVIDENCE-BUNDLE]]
- [[security/pentest/GA_PENTEST_EVIDENCE_BUNDLE.md|GA-PENTEST-EVIDENCE-BUNDLE]]
## Documentation Sync (2026-08-09)
- Root-level markdown documentation was refreshed against current `src/` module docs and root governance files.
- Current source-backed release-readiness highlights:
- **19 production-candidate modules** verified: access_model, analytics, graph, image_analysis, index, llm, llm_streaming, network, observability, prompt_engineering, query, rag, replication, server, sharding, temporal, training, transaction, vector_search.
- Process module Phase 1-6 complete and production-ready (2026-08-06): 101 files, 33,106+ LOC, 87 acceptance criteria passed.
- Auth module Phase 1-6 hardening complete (`src/auth/ROADMAP.md`): principal-contract freeze, 12 new error codes, RFP/FED/ASY focused tests, and AHP benchmark gates.
- Failover module Phase 2-3 complete: state machine, split-brain prevention, DR orchestration with diagnostics.
- Wave 5 and Wave 6 hardening remain the retained regression baseline for release-critical journeys.
- Wave 7 PASS evidence and sharding Phase 6 sign-off remain linked into the active GA hardening path.
- Batch C sanitizer / pentest evidence is closed; Batch D human governance sign-off is still pending at `docs/governance/GA_PROMOTION_SIGN_OFF.md` ยง9.
- Broken historical root evidence references were replaced with current source-verifiable artefacts.
### Scanner Baseline Update (2026-06-11)
- Aktueller Gap-Scan-Stand wird ueber die Worklist gepflegt:
- `ai_working/gap_scan_report_ollama_gemma4.md`
- `ai_working/gap_scan_report_ollama_gemma4.smoke.md`
- Scope-Regel: `themis_core` actionable, `third_party` nur informativ.
- Aktives Tracking-Issue fuer den aktuellen Baseline-Scope:
- `#5475` (`[P0-HIGH] INCLUDE Module - Current Gap Worklist Tracking (2026-06-11)`)
- Konsolidierungsstatus GitHub-Issues:
- Historische v3-P0- und Cross-Module-Tracker wurden geschlossen (superseded by `#5475`).
- Duplikat-Tracker `#5474` wurde geschlossen.
- Bewusst offen bleiben die Legacy-Umstellungs-Issues `#5363` bis `#5366`.
---
## What is ThemisDB?
ThemisDB is a **high-performance multi-model database engine in active development** that aims to combine relational, graph, vector, document, geospatial, and time-series storage in a single system with native AI/LLM integration.
**Current Status (2026-08-28, source-evidence based):** 72 modules are tracked in `src`; 19 are `PRODUCTION_CANDIDATE`, 48 are `HARDENING`, 2 are `EXPERIMENTAL`, and 3 are `THIN/PLACEHOLDER`. See [[ROADMAP.md|ROADMAP]] for detailed per-module status.
**Key capabilities at a glance:**
| Capability | Details |
|---|---|
| **Multi-model storage** | Relational ยท Graph ยท Vector (HNSW/FAISS) ยท Document ยท Geospatial ยท Time-series |
| **ACID transactions** | MVCC, SSI, 2PC, SAGA orchestration, HLC-based global ordering |
| **Distributed** | Raft consensus, mTLS replication, consistent-hash sharding, auto-failover |
| **AI/LLM native** | Embedded LLM inference (llama.cpp, ONNX), RAG pipeline, prompt engineering, LoRA fine-tuning |
| **Full-text search** | BM25 + vector hybrid search (RRF), faceted, conversational, multi-modal |
| **Observability** | Prometheus metrics, OpenTelemetry tracing, PagerDuty/Slack alerting |
| **Security** | AES-256-GCM field encryption, RLS, Zero-Trust policy, eIDAS timestamping, HSM/Vault |
| **Editions** | MINIMAL ยท COMMUNITY ยท ENTERPRISE ยท MILITARY ยท HYPERSCALER |
---
## Canonical Onboarding Path
For a consistent onboarding flow, use these pages in order:
1. [[QUICKSTART.md|QUICKSTART]] โ install + first successful run
2. [[SETUP.md|SETUP]] โ complete local development environment
3. [[SUPPORT.md|SUPPORT]] โ support and escalation paths
4. [[RELEASE_STRATEGY.md|RELEASE-STRATEGY]] โ release lanes and version lifecycle
5. [[INDEX.md|INDEX]] โ full root navigation map
---
## Installation
### Docker (fastest)
```bash
docker pull ghcr.io/makr-code/themisdb:latest
docker run -d --name themisdb -p 8765:8765 -p 8766:8766 ghcr.io/makr-code/themisdb:latest
```
Connect via the wire protocol on port `8766` or the REST/HTTP API on port `8765`.
### Build from source
Source-backed preset prerequisites:
- `linux-release` expects Ninja and a bootstrapped `vcpkg` checkout at `./vcpkg`.
- `community-release` requires a system RocksDB development package (`librocksdb-dev` or equivalent).
> Build reality check: on developer machines the most reliable path is a repo-local `vcpkg` checkout plus `THEMIS_AUTO_BOOTSTRAP_DEPS=ON` during configure. Some Windows and fresh-clone setups still require that bootstrap path to be present before the toolchain is valid; this is a local environment requirement and not a stable source-level contract.
> Avoid hard-coded absolute paths such as `C:/Projects/ThemisDB/...` in documentation. The portable pattern is to reference the repo root and use local `CMakeUserPresets.json` overrides or environment variables for machine-specific values.
```bash
git clone https://github.com/makr-code/ThemisDB.git
cd ThemisDB
# Install dependencies and configure build environment
./scripts/setup-pre-commit.sh # Linux/macOS
# CMake-native dependency bootstrap (cross-platform)
cmake --preset linux-release -DTHEMIS_AUTO_BOOTSTRAP_DEPS=ON
cmake --build --preset linux-release
# Community edition (requires RocksDB system package):
# cmake --preset community-release && cmake --build --preset community-release
# Windows (run from VS Developer Command Prompt):
# cmake --preset windows-release && cmake --build --preset windows-release
```
See [[QUICKSTART.md|QUICKSTART]] for a step-by-step guide, and [[SETUP.md|SETUP]] for a full development-environment walkthrough.
---
## Editions
ThemisDB is available in five editions, selected at CMake build time:
| Edition | Use case | Branch | Build flag |
|---|---|---|---|
| **MINIMAL** | Embedded / resource-constrained | `minimal` | `-DTHEMIS_EDITION=MINIMAL` |
| **COMMUNITY** | Open-source, self-hosted | `community` | `-DTHEMIS_EDITION=COMMUNITY` |
| **ENTERPRISE** | Commercial, SLA-backed | `enterprise` | `-DTHEMIS_EDITION=ENTERPRISE` |
| **MILITARY** | Hardened / air-gapped | `military` | `-DTHEMIS_EDITION=MILITARY` |
| **HYPERSCALER** | Cloud/OEM, Kubernetes operator | `hyperscaler` | `-DTHEMIS_EDITION=HYPERSCALER` |
Feature sets are nested: MINIMAL โ COMMUNITY โ ENTERPRISE โ HYPERSCALER.
See [[RELEASE_STRATEGY.md|RELEASE-STRATEGY]] for the full feature comparison and edition matrix.
GitHub workflow and branch governance references:
- [[BRANCHING_STRATEGY.md|BRANCHING-STRATEGY]] - canonical branch model (`develop`, `minimal`, `community`, `enterprise`, `hyperscaler`, `military`)
- [[.github/GOVERNANCE.md|GOVERNANCE]] - labels, milestones, and issue/PR metadata standards
- [[.github/pull_request_template.md|pull-request-template]] - required PR evidence sections
---
## Usage
After startup, verify health and run a first query:
```bash
curl http://localhost:8765/health
curl -X POST http://localhost:8765/v2/query \
-H 'Content-Type: application/json' \
-d '{"query":"SELECT 1 AS hello"}'
```
---
## Architecture
ThemisDB is organised into tracked source modules under `src/`, grouped into four logical layers:
```
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ API Layer REST ยท GraphQL ยท gRPC ยท Wire V2 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Query Layer AQL ยท Optimizer ยท Planner ยท Cache โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Storage Layer RocksDB ยท MVCC ยท WAL ยท Sharding โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Distributed Raft ยท Replication ยท Failover ยท CDC โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
```
โ Full architecture reference: [[ARCHITECTURE.md|ARCHITECTURE]]
โ Module list and status: [[ROADMAP.md|ROADMAP]]
### Distributed Transactions
ThemisDB supports distributed transactions across shards using a family of commit protocols.
Three concrete coordinator classes share the `IRecoverableTwoPhaseCoordinator` recovery contract:
| Coordinator | Protocols | Use Case |
|---|---|---|
| `TwoPhaseCommitCoordinator` | 2PC | Standalone 2PC in sharding module |
| `CrossShardTransactionCoordinator` | 2PC ยท 3PC ยท SAGA ยท Percolator ยท Calvin | Multi-protocol orchestration |
| `DistributedTransactionCoordinator` | 2PC (+ optional Percolator fast-path) | TrueTime-based distributed 2PC coordinator |
`WALLoggingHelper` (`include/sharding/wal_logging_helper.h`) is used by
`TwoPhaseCommitCoordinator` and `DistributedTransactionCoordinator` (both `WALManager`-backed).
`CrossShardTransactionCoordinator` uses its own `TransactionWAL` backend.
โ Full architecture reference: [[docs/architecture/transaction_coordinators.md|transaction-coordinators]]
โ Interface design: [[docs/ITRANSACTION_COORDINATOR.md|ITRANSACTION-COORDINATOR]]
### Security Tiering Quick Reference
For security and hardening reviews, use the tier model (T0 Trusted Core -> T5 Plugin Boundary) as the default classification.
| What | Where |
|---|---|
| Tier model and trust boundaries | [[ARCHITECTURE.md|ARCHITECTURE]] |
| Normative security rules per tier | [[SECURITY.md|SECURITY]] |
| Contributor checklist for tier/boundary evidence | [[CONTRIBUTING.md|CONTRIBUTING]] |
| PR template section (required for runtime changes) | [[.github/pull_request_template.md|pull-request-template]] |
| Tier-to-test verification mapping | [[CTEST.md|CTEST]] |
Rule of thumb: architecture is layered, but security acceptance is tier-based.
---
## Documentation
| Document | Description |
|---|---|
| [[QUICKSTART.md|QUICKSTART]] | Get running in minutes |
| [[SETUP.md|SETUP]] | Full development environment setup |
| [[ARCHITECTURE.md|ARCHITECTURE]] | System design and module overview |
| [[VERSIONING.md|VERSIONING]] | Versioning policy and release cadence |
| [[RELEASE_STRATEGY.md|RELEASE-STRATEGY]] | Branch model, edition matrix, CI/CD pipeline |
| [[CHANGELOG.md|CHANGELOG]] | Release notes (Keep a Changelog format) |
| [[AUDIT.md|AUDIT]] | Root audit navigation pointer to canonical `/audit/**` sources |
| [[docs/performance/PERFORMANCE_EXPECTATIONS.md|PERFORMANCE-EXPECTATIONS]] | Benchmarks and performance targets |
| [[docs/use-cases/LLM_WIKI_MVP.md|LLM-WIKI-MVP]] | LLM Wiki MVP setup, config, and usage |
| [[SOP.md|SOP]] | Standard operating procedures (release, hotfix, incident) |
| [[GOVERNANCE.md|GOVERNANCE]] | Project governance: roles, decision-making, contribution policy |
| [[MAINTAINERS.md|MAINTAINERS]] | Maintainer roster and module ownership |
| [[SECURITY.md|SECURITY]] | Security policy and vulnerability reporting |
| [[CONTRIBUTING.md|CONTRIBUTING]] | How to contribute |
| [[CODE_OF_CONDUCT.md|CODE-OF-CONDUCT]] | Community guidelines |
| [[SUPPORT.md|SUPPORT]] | Where to get help |
| [[INDEX.md|INDEX]] | Full project structure index |
| [docs/](docs/) | Extended documentation (API reference, guides, research) |
---
## Versioning
ThemisDB follows [Semantic Versioning 2.0.0](https://semver.org/). The current version is stored in the [`VERSION`](VERSION) file and in [[`CHANGELOG.md`|CHANGELOG]]. Pre-release identifiers use the form `-rcN` (release candidate) or `-alphaN` / `-betaN`.
See [[VERSIONING.md|VERSIONING]] for the full versioning policy.
---
## Contributing
Contributions are welcome! Please read [[CONTRIBUTING.md|CONTRIBUTING]] before submitting a pull request. All participants are expected to follow our [[Code of Conduct|CODE-OF-CONDUCT]].
Good first issues are tagged [`good first issue`](https://github.com/makr-code/ThemisDB/issues?q=is%3Aopen+label%3A%22good+first+issue%22) in the issue tracker.
---
## Security
To report a security vulnerability, **do not** open a public issue. Follow the responsible disclosure process in [[SECURITY.md|SECURITY]] or use [GitHub Security Advisories](https://github.com/makr-code/ThemisDB/security/advisories/new).
---
## License
ThemisDB is released under the [[MIT License with Government Clause|license]].
---
## Quality Assurance & Gap Scanning
ThemisDB includes **Gap Scanner V3 (GS3)**, a comprehensive multi-phase gap detection system with 46 specialized scanners organized across 4 phases.
### Quick Start with GS3
```bash
# List all 46 scanners
python tools/gs3.py list-scanners
# Run fast scan on source code
python tools/gs3.py scan src include tests --scan-mode fast --output results.json
# Generate Markdown report
python tools/gs3.py report results.json --format md --output report.md
# Generate JSON report
python tools/gs3.py report results.json --format json
```
### Scanner Organization
| Phase | Category | Count | Focus |
|-------|----------|-------|-------|
| **Phase 1** | AI, Core C++, Checks | 18 | Baseline detection (AI-Vibe, memory, concurrency) |
| **Phase 2** | Safety | 5 | Exception safety, input validation, type safety |
| **Phase 3** | Security | 7 | Cryptography, data leaks, hardening |
| **Phase 4** | Design & Quality | 16 | Architecture rules, documentation standards |
### Documentation
- **[[tools/GS3_CLI_GUIDE.md|GS3-CLI-GUIDE]]** โ Complete CLI reference and usage guide
- **[[tools/scanners/GS3_COMPLETE_GUIDE.md|GS3-COMPLETE-GUIDE]]** โ System architecture and scanner design
- **[[tools/GS3_PROJECT_COMPLETION_REPORT.md|GS3-PROJECT-COMPLETION-REPORT]]** โ Project deliverables and metrics
- **[[tools/legacy/LEGACY_SCANNER_MAPPING.md|LEGACY-SCANNER-MAPPING]]** โ Legacy code archival and migration info
### Key Features
- **46 specialized scanners** for AI-Vibe, C++, security, and design gaps
- **Dual-axis classification**: Severity (CRITICAL/HIGH/MEDIUM/LOW) ร Impact (CRITICAL/HIGH/MEDIUM/LOW/THIRD_PARTY)
- **Auto-discovery**: Scanners automatically discovered from `tools/scanners/`
- **Multiple output formats**: JSON (machine-readable) and Markdown (human-readable)
- **Scan modes**: Fast (quick pass) and Thorough (detailed analysis)
- **Phase-based execution**: Sequential scanning through phases 1-4
### CI/CD Integration
```bash
# Fast scan for PR validation (< 3 minutes)
python tools/gs3.py scan src --scan-mode fast --output pr_scan.json
# Fail on critical blockers
if grep -q '"severity":"CRITICAL".*"impact_level":"CRITICAL"' pr_scan.json; then
echo "FAILED: Critical blockers detected"
exit 1
fi
```
See [[tools/GS3_CLI_GUIDE.md|GS3-CLI-GUIDE]] for more CI/CD examples.
---
## Module Documentation
> Per-module documentation lives in `src//README.md` and `include//`. This section is a navigation index.
---
Zuletzt geprueft (Root-Sync): 2026-08-28