-
Notifications
You must be signed in to change notification settings - Fork 1
content_filesystem_api
makr-code edited this page Dec 21, 2025
·
1 revision
Stand: 5. Dezember 2025
Version: 1.0.0
Kategorie: Content
Datum: 19. Nov 2025 Status: Implementiert (Core), Server-Endpoints optional
Einfaches Binär-Content-Storage über RocksDB mit konsistenten Metadaten, optionaler SHA‑256 Integrität und Range‑Reads.
- Datenlayout:
-
content:<pk>:meta→ CBOR‑codiertes JSON{ pk, mime, size, sha256_hex } -
content:<pk>:blob→ Binärdaten
-
- Transaktionen: Einzel‑Writes (Meta dann Blob); für Multi‑Objekt‑Ingest können WriteBatch/Txn genutzt werden.
class ContentFS {
public:
ContentFS(RocksDBWrapper& db);
struct Status { bool ok; std::string message; };
Status put(const std::string& pk,
const std::vector<uint8_t>& data,
const std::string& mime,
const std::optional<std::string>& sha256_expected_hex = std::nullopt);
std::pair<Status, std::vector<uint8_t>> get(const std::string& pk) const;
std::pair<Status, std::vector<uint8_t>> getRange(const std::string& pk, uint64_t offset, uint64_t length) const;
std::pair<Status, ContentMeta> head(const std::string& pk) const;
Status remove(const std::string& pk);
static std::string sha256Hex(const std::vector<uint8_t>& data);
};RocksDBWrapper db(cfg); db.open();
ContentFS fs(db);
std::vector<uint8_t> bytes = load_file("image.png");
auto hex = ContentFS::sha256Hex(bytes);
auto st = fs.put("img_123", bytes, "image/png", hex);
auto [hst, meta] = fs.head("img_123");
auto [gst, full] = fs.get("img_123");
auto [rst, part] = fs.getRange("img_123", 0, 1024);-
tests/test_content_fs.cpp: Roundtrip, Range‑Reads, Checksum‑Mismatch, Delete → 4/4 PASS (MSVC Debug)
- Kleine bis mittlere Blobs (KB–MB) performant direkt in RocksDB/BlobDB.
- Für sehr große Objekte (>100MB) optionales Sharding/Streaming als Erweiterung (Future Work).
-
POST /content– Upload (Body), Header:Content-Type, optionalX-Checksum-SHA256. -
GET /content/{pk}– Download; unterstütztRange:. -
HEAD /content/{pk}– Metadaten. -
DELETE /content/{pk}– Löschen.
Hinweis: Endpoints sind aktuell nicht verdrahtet; Implementierung kann über themis_server Handler folgen.
- Architecture-ACCESS-MODEL-IMPLEMENTATION-SUMMARY
- Architecture-ADR-003-pg-dump-sql-parser
- Architecture-BASEENTITY-PRINCIPLE
- Architecture-CACHE-STORAGE-INTEGRATION
- Architecture-CMAKE-ARCHITECTURE
- Architecture-CMAKE-FLAGS-REFERENCE
- Architecture-CMAKE-MODULAR-ARCHITECTURE
- Architecture-CONCERNS-ARCHITECTURE-DIAGRAM
- Architecture-CONCERNS-IMPLEMENTATION-SUMMARY
- Architecture-CONTENT-MODEL
- Architecture-COPILOT-THEMISDB-GRAPH-RAG-BACKEND-ARCHITECTURE
- Architecture-CRYPTO-AND-KEYS
- Architecture-FEATURE-FLAGS-REFERENCE
- Architecture-GPU-ARCHITECTURE-REVIEW-TEMPLATE
- Architecture-HTTP-SHUTDOWN-HARDENING
- Architecture-MIGRATION-GUIDE-CONCERNS
- Architecture-MIGRATION-GUIDE-v13-v14
- Architecture-MODULARIZATION-GUIDE
- Architecture-MODULAR-ARCHITECTURE-ROADMAP
- Architecture-MODULE-ARCHITECTURE-INDEX
- Architecture-P1D01-ISSMPLUGIN-DESIGN-REVIEW
- Architecture-P1-D01-ISSMPLUGIN-DESIGN-REVIEW
- Architecture-P1-D08-MAMBA-GOVERNANCE-CONTRACT
- Architecture-P1-P2-IMPLEMENTATION-COMPLETION-INDEX
- Architecture-PHASE0-COMPLETION-ASSESSMENT
- Architecture-PHASE3-QUERYENGINE-DI-ARCHITECTURE
- Architecture-PHASE4-INDEX-MANAGER-DI
- Architecture-POSTGRESQL-WIRE-PROTOCOL
- Architecture-QUERYENGINE-IMPLEMENTATION-GUIDE
- Architecture-QUERY-SCHEDULING
- Architecture-RAFT-CONSENSUS-DESIGN
- Architecture-README
- Architecture-README-SSM-HYBRID-IMPLEMENTATION
- Architecture-REFACTORING-SUMMARY
- Architecture-RESOURCE-POOLING
- Architecture-SOURCE-DIRECTORY-GUIDE
- Architecture-THEMIS-CORE-GUIDE
- Architecture-UNIFIED-ACCESS-MODEL
- Architecture-WAL-GRPC-MTLS-CONFIGURATION
- Architecture-WIRE-PROTOCOL-RETRY
- Architecture-boltzmann-observability-draft
- Architecture-experimental-logarithmic-vector-storage
- Architecture-llm-wiki-mvp-adr
- Architecture-rewrite-engine-architecture
- Architecture-rope-api-architecture
- Architecture-ssm-gguf-mamba-status
- Architecture-ssm-hybrid-analysis
- Architecture-ssm-hybrid-rollout-plan
- Architecture-ssm-plugin-interface-design-review
- Architecture-transaction-coordinators
- Architecture-wiki-secondary-index
- Architecture-wire-protocol
- Governance-DISABLED-STUB-POLICY
- Governance-DOCS-PR-POLICY
- Governance-GA-PROMOTION-SIGN-OFF
- Governance-GITHUB-MILESTONES-SETUP
- Governance-MATURITY-CLAIM-VERIFICATION-CHECKLIST
- Governance-MATURITY-EVIDENCE-REGISTRY
- Governance-MERGE-GATE-BOT-CONFIG
- Governance-MERGE-GATE-STATUS-LIVE
- Governance-PHASE3-ENFORCEMENT-RUNBOOK
- Governance-PHASE-1-CLOSURE-REPORT
- Governance-PHASE-CLOSURE-POLICY
- Governance-PHASE-DEPENDENCY-GRAPH
- Governance-PLUGIN-SUBMODULE-ROLLBACK
- Governance-PRODUCTION-READY-2026-DELIVERY-PLAN
- Governance-PR-VERSION-TARGETING
- Governance-PR-VERSION-TARGETING-BACKFILL
- Governance-QUERY-MODULE-STATUS
- Governance-README
- Governance-RELEASE-PROMOTION-GATE-POLICY
- Governance-RELEASE-VALIDATION-CHECKLIST
- Governance-SECURITY-MODULE-5671-EVIDENCE-SUMMARY
- Governance-SHARDING-P6-RESIDUAL-RISK-ACCEPTANCE
- Governance-SOURCECODE-COMPLIANCE-GOVERNANCE
- Governance-UPDATES-DEVELOPMENT-STATUS-SIGN-OFF
- Governance-WAVE-C-IMPLEMENTATION-COMPLETE
- Module-acceleration-Roadmap
- Module-access-model-Roadmap
- Module-ai-Roadmap
- Module-analytics-Roadmap
- Module-api-Roadmap
- Module-aql-Roadmap
- Module-auth-Roadmap
- Module-base-Roadmap
- Module-cache-Roadmap
- Module-cdc-Roadmap
- Module-chaos-Roadmap
- Module-chimera-Roadmap
- Module-config-Roadmap
- Module-content-Roadmap
- Module-core-Roadmap
- Module-distributed-knowledge-Roadmap
- Module-distributed-tensor-Roadmap
- Module-document-Roadmap
- Module-ethics-ai-Roadmap
- Module-evaluation-Roadmap
- Module-execution-Roadmap
- Module-exporters-Roadmap
- Module-failover-Roadmap
- Module-geo-Roadmap
- Module-governance-Roadmap
- Module-gpu-Roadmap
- Module-graph-Roadmap
- Module-image-analysis-Roadmap
- Module-importers-Roadmap
- Module-index-Roadmap
- Module-ingestion-Roadmap
- Module-llama-cpp-Roadmap
- Module-llm-Roadmap
- Module-llm-streaming-Roadmap
- Module-llm-wiki-Roadmap
- Module-maintenance-Roadmap
- Module-metadata-Roadmap
- Module-network-Roadmap
- Module-observability-Roadmap
- Module-onnx-clip-Roadmap
- Module-performance-Roadmap
- Module-plugins-Roadmap
- Module-process-Roadmap
- Module-projects-Roadmap
- Module-prompt-engineering-Roadmap
- Module-query-Roadmap
- Module-rag-Roadmap
- Module-replication-Roadmap
- Module-retrieval-Roadmap
- Module-rpc-grpc-Roadmap
- Module-scheduler-Roadmap
- Module-scraper-Roadmap
- Module-search-Roadmap
- Module-security-Roadmap
- Module-server-Roadmap
- Module-sharding-Roadmap
- Module-stable-diffusion-Roadmap
- Module-storage-Roadmap
- Module-temporal-Roadmap
- Module-tensor-Roadmap
- Module-themis-Roadmap
- Module-timeseries-Roadmap
- Module-toolbox-Roadmap
- Module-training-Roadmap
- Module-transaction-Roadmap
- Module-updates-Roadmap
- Module-user-storage-encrypted-Roadmap
- Module-utils-Roadmap
- Module-vector-search-Roadmap
- Module-voice-Roadmap
- Module-whisper-Roadmap