docs: module dep map follow-ups — circular-dep ROADMAP tracking, CI lint boundary, cross-batch consumer audit, transaction renumbering - #6284
Conversation
…path modules - src/server/ARCHITECTURE.md: add ## Module Dependencies + ## Integration Points; validated date 2026-09-09 - src/query/ARCHITECTURE.md: add ## 14 Module Dependencies + ## 15 Integration Points; Last Updated 2026-09-09 - src/transaction/ARCHITECTURE.md: add ## Module Dependencies + ## 9 Integration Points (Detailed); Last Updated 2026-09-09 - src/sharding/ARCHITECTURE.md: add ## Module Dependencies + ## Integration Points; validated date 2026-09-09 - src/storage/ARCHITECTURE.md: add ## Module Dependencies + ## Integration Points; validated date 2026-09-09 - src/replication/ARCHITECTURE.md: add ## Module Dependencies + ## Integration Points; validated date 2026-09-09 Upstream dependency tables sourced from confirmed include/src scan (2026-09-09). Circular deps (llm↔server, query↔llm, sharding↔transaction) documented with contract + thread-safety + failure-mode notes. Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
…g layer Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
- src/llm/ARCHITECTURE.md: add Module Dependencies (15 upstream, 5 downstream), Integration Points (5 critical), Known Design Issues (llm↔server circular via mcp_tool_bridge.cpp→mcp_server.h; llm↔query bidirectional via lora_functions.cpp; distributed-collectives open item); Last Updated → 2026-09-09 - src/rag/ARCHITECTURE.md: add Module Dependencies (11 upstream, 3 downstream), Integration Points (5: ContextWindowBudget, ILLMPlugin, IVectorIndex, KnowledgeGraph, Recall@k open); Last Updated → 2026-09-09 - src/search/ARCHITECTURE.md: add Last Updated + status header, Module Dependencies (5 upstream, 3 downstream), Integration Points (5: HybridSearch v2.0, LlmReranker injected, ANNFrontdoor, LayeredRetrievalOrchestrator v3.0, DistributedHybridSearch v2.2); Last Updated → 2026-09-09 - src/index/ARCHITECTURE.md: add Last Updated + status header, Module Dependencies (6 upstream, 3 downstream), Integration Points (4: IVectorIndex/ISecondaryIndex/ IGraphIndex contracts, ANNFrontdoor frozen ABI, GPU RAII guards, LoRA adaptation); Last Updated → 2026-09-09 - src/acceleration/ARCHITECTURE.md: add Module Dependencies (leaf module, 0 upstream ThemisDB deps; 4 downstream consumers), Integration Points (5: ComputeBackend dispatch, DeviceManager probing, CUDA RAII guards, Multi-GPU/NCCL-RCCL, Plugin+shader integrity); Last Updated → 2026-09-09 SOT domain: module-behavior + architecture-governance Level: level1 (src/<module>/) Milestone: DOC-WEEKLY-2026-37 Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
…9-09) Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
… modules Modules updated (26 files this commit; analytics/graph/geo/content/ingestion already committed in f025b14): Layer 0/foundation: - src/utils/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/base/ARCHITECTURE.md — add Module Dependencies + Integration Points Layer 0+/framework: - src/core/ARCHITECTURE.md — Last Updated 2026-09-09; add § Module Dependencies + expand Integration Points - src/network/ARCHITECTURE.md — Last Updated 2026-09-09; add § Module Dependencies + expand Integration Points - src/plugins/ARCHITECTURE.md — add Module Dependencies + Integration Points Layer 1: - src/metadata/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/temporal/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/timeseries/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/tensor/ARCHITECTURE.md — add Module Dependencies + Integration Points Layer 2: - src/scheduler/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/distributed_knowledge/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/governance/ARCHITECTURE.md — add Module Dependencies + Integration Points Layer 4: - src/document/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/process/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/prompt_engineering/ARCHITECTURE.md — add Module Dependencies + Integration Points Layer 5: - src/training/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/retrieval/ARCHITECTURE.md — add Module Dependencies + Integration Points (scaffold note) - src/ethics_ai/ARCHITECTURE.md — add Module Dependencies + Integration Points - src/llm_wiki/ARCHITECTURE.md — expand Dependencies + Integration Points - src/distributed_tensor/ARCHITECTURE.md — expand Integration Surfaces + add § Module Dependencies + Critical Integration blocks Layer 5 plugins: - src/whisper/ARCHITECTURE.md — Last Updated 2026-09-09; add Module Dependencies + Integration Points - src/stable_diffusion/ARCHITECTURE.md — Last Updated 2026-09-09; add Module Dependencies + Integration Points - src/llama_cpp/ARCHITECTURE.md — Last Updated 2026-09-09; add Module Dependencies + Integration Points - src/onnx_clip/ARCHITECTURE.md — Last Updated 2026-09-09; expand Integration Points + add Module Dependencies Docs-only / no source: - src/llm_streaming/ARCHITECTURE.md — add Implementation Status (no source) + Module Dependencies + Integration Points - src/vector_search/ARCHITECTURE.md — add Implementation Status (no source) + Module Dependencies + Integration Points SOT domain: module-behavior / architecture-governance Level: level1 Milestone: DOC-WEEKLY-2026-37 Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
…): transaction ARCHITECTURE.md section renumbering Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
…o Batch 1-3 dep tables Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
This PR is a documentation- and governance-focused follow-up that tightens architecture enforcement (CI lint boundary) and expands module dependency documentation, while also adding formal tracking for known circular dependencies.
Changes:
- Enforce
ingestion → toolboxas a prohibited dependency in the architecture scanner. - Add/extend per-module dependency and integration-point sections across many
ARCHITECTURE.mdfiles (incl. “docs-only module” disclaimers). - Update
ROADMAP.mdwith explicit circular-dependency tracking and a resolution checklist; renumbertransactionarchitecture sections.
Reviewed changes
Copilot reviewed 53 out of 53 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/scanners/gs3_step04_design_architecture.py | Adds a prohibited dependency to enforce the ingestion/toolbox boundary in CI lint. |
| ROADMAP.md | Adds an explicit circular-dependency blocker row and a detailed tracking subsection. |
| src/ARCHITECTURE.md | Adds a unified dependency map, layers, and documented circular dependencies. |
| src/whisper/ARCHITECTURE.md | Updates “Last Updated” and adds dependency/integration documentation. |
| src/vector_search/ARCHITECTURE.md | Adds “no source implementation” disclaimer plus dependencies/integration notes. |
| src/utils/ARCHITECTURE.md | Adds dependencies and integration points for utils. |
| src/transaction/ARCHITECTURE.md | Renumbers sections and inserts a dependencies + integration section. |
| src/training/ARCHITECTURE.md | Adds dependencies and integration points for training module. |
| src/toolbox/ARCHITECTURE.md | Documents the dependency direction rule and adds dependencies/integration points. |
| src/timeseries/ARCHITECTURE.md | Adds dependencies and integration points for timeseries. |
| src/tensor/ARCHITECTURE.md | Adds dependencies and integration points for tensor. |
| src/temporal/ARCHITECTURE.md | Adds dependencies and integration points for temporal. |
| src/storage/ARCHITECTURE.md | Adds dependency tables and integration points for storage. |
| src/stable_diffusion/ARCHITECTURE.md | Adds dependencies and integration points for stable_diffusion plugin. |
| src/sharding/ARCHITECTURE.md | Adds dependency tables and integration points for sharding. |
| src/server/ARCHITECTURE.md | Adds dependency tables and integration points for server. |
| src/security/ARCHITECTURE.md | Adds dependency tables and integration points for security. |
| src/search/ARCHITECTURE.md | Adds dependency tables and detailed integration-point contracts for search. |
| src/scheduler/ARCHITECTURE.md | Adds dependency tables and integration points for scheduler. |
| src/retrieval/ARCHITECTURE.md | Adds dependencies/integration notes (scaffold/aspirational disclaimer). |
| src/replication/ARCHITECTURE.md | Adds dependencies and integration points for replication. |
| src/rag/ARCHITECTURE.md | Adds dependencies and integration points for RAG. |
| src/query/ARCHITECTURE.md | Adds a dependencies section and new integration points section. |
| src/prompt_engineering/ARCHITECTURE.md | Adds dependencies and integration points for prompt_engineering. |
| src/process/ARCHITECTURE.md | Adds dependencies and integration points for process. |
| src/plugins/ARCHITECTURE.md | Adds dependencies and integration points for plugins. |
| src/onnx_clip/ARCHITECTURE.md | Adds dependencies and integration points for onnx_clip plugin. |
| src/observability/ARCHITECTURE.md | Adds dependencies and integration points for observability. |
| src/network/ARCHITECTURE.md | Adds dependencies and integration points for network. |
| src/metadata/ARCHITECTURE.md | Adds dependencies and integration points for metadata. |
| src/maintenance/ARCHITECTURE.md | Adds dependencies and integration points for maintenance. |
| src/llm_wiki/ARCHITECTURE.md | Adds dependencies and integration points for llm_wiki. |
| src/llm_streaming/ARCHITECTURE.md | Adds “no source implementation” disclaimer plus dependencies/integration notes. |
| src/llm/ARCHITECTURE.md | Adds a module dependency table, integration points, and known circular-dep section. |
| src/llama_cpp/ARCHITECTURE.md | Adds dependencies and integration points for llama_cpp plugin. |
| src/ingestion/ARCHITECTURE.md | Adds dependencies/integration points for ingestion (incl. toolbox mention). |
| src/index/ARCHITECTURE.md | Adds dependency tables and integration points for index. |
| src/graph/ARCHITECTURE.md | Adds dependency tables and integration points for graph. |
| src/governance/ARCHITECTURE.md | Adds dependencies and integration points for governance. |
| src/geo/ARCHITECTURE.md | Adds dependencies and integration points for geo. |
| src/failover/ARCHITECTURE.md | Adds dependencies and integration points for failover. |
| src/ethics_ai/ARCHITECTURE.md | Adds dependencies and integration points for ethics_ai. |
| src/document/ARCHITECTURE.md | Adds dependencies and integration points for document. |
| src/distributed_tensor/ARCHITECTURE.md | Adds dependencies and integration points for distributed_tensor. |
| src/distributed_knowledge/ARCHITECTURE.md | Adds dependencies and integration points for distributed_knowledge. |
| src/core/ARCHITECTURE.md | Adds dependencies and integration points for core. |
| src/content/ARCHITECTURE.md | Adds dependencies and integration points for content. |
| src/cdc/ARCHITECTURE.md | Adds dependencies and integration points for cdc. |
| src/cache/ARCHITECTURE.md | Adds dependencies and integration points for cache. |
| src/base/ARCHITECTURE.md | Adds dependencies and integration points for base. |
| src/auth/ARCHITECTURE.md | Adds dependencies and integration points for auth. |
| src/analytics/ARCHITECTURE.md | Adds dependencies and integration points for analytics. |
| src/acceleration/ARCHITECTURE.md | Documents acceleration as a leaf module; adds dependencies/integration points. |
Suppressed comments (2)
src/whisper/ARCHITECTURE.md:1
- The metadata block previously included
**Module Path:** \src/whisper/`but it’s removed in this update. If other module architecture docs rely on this field for consistency/searchability, restore theModule Path` line to avoid losing structured metadata.
src/stable_diffusion/ARCHITECTURE.md:1 - The metadata block previously included
**Module Path:** \src/stable_diffusion/`but it’s removed in this update. Consider restoringModule Path` to keep module docs consistent and to preserve a stable “where is this module” anchor.
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| | **0 — Foundation** | `base`, `core`, `performance`, `utils` | Stateless utilities; no ThemisDB upstream deps | | ||
| | **1 — Persistent Storage** | `storage`, `cache`, `metadata`, `timeseries`, `temporal` | RocksDB wrapper, MVCC, blob/columnar, schema management | | ||
| | **2 — Distributed Infrastructure** | `sharding`, `replication`, `network`, `transaction`, `cdc`, `failover`, `maintenance` | Consensus (Raft/Paxos/Gossip), WAL shipping, 2PC, ACID, changefeed | | ||
| | **3 — Indexing & Search** | `index`, `search`, `geo`, `graph` | HNSW/vector/spatial/graph indexes, BM25+vector hybrid search | | ||
| | **4 — Data Processing** | `query`, `aql`, `analytics`, `execution`, `content`, `ingestion`, `toolbox`, `document` | AQL/SQL pipeline, operator execution, multimodal ingestion | | ||
| | **5 — AI / LLM** | `llm`, `rag`, `retrieval`, `llm_wiki`, `training`, `prompt_engineering`, `llama_cpp`, `onnx_clip`, `whisper`, `stable_diffusion`, `distributed_knowledge`, `distributed_tensor` | Inference, RAG evaluation, LoRA, embeddings, vision, multi-modal | | ||
| | **6 — Application & Protocol** | `server`, `api`, `auth`, `security`, `plugins`, `rpc_grpc`, `acceleration` | HTTP/gRPC servers, 50+ API handlers, GPU acceleration, plugin system | |
There was a problem hiding this comment.
Fixed in edb737043e. I moved acceleration out of the application/protocol layer into its own leaf runtime services layer so the dependency direction matches the module architecture doc.
| | observability | `include/observability/` | Inference span telemetry, throughput metrics | | ||
| | security | `include/security/` | Prompt sanitisation, credential-isolation at inference boundary | | ||
| | utils | `include/utils/` | Common utilities (logging, string helpers) | | ||
| | llama_cpp | `stable-diffusion.cpp/` (external) | Underlying llama.cpp inference backend | |
There was a problem hiding this comment.
Fixed in edb737043e. The llama_cpp dependency row now points at the ThemisDB llama_cpp plugin surface instead of the unrelated external path.
| @@ -6,8 +6,7 @@ | |||
| <!-- Links: README.md · ROADMAP.md · FUTURE_ENHANCEMENTS.md --> | |||
|
|
|||
| **Version:** 1.0 | |||
There was a problem hiding this comment.
Fixed in edb737043e. I restored the Module Path metadata line in src/llama_cpp/ARCHITECTURE.md.
| @@ -6,8 +6,7 @@ | |||
| # ONNX CLIP Plugin — Architecture Guide | |||
|
|
|||
| **Version:** 0.0.1 | |||
There was a problem hiding this comment.
Fixed in edb737043e. I restored the Module Path metadata line in src/onnx_clip/ARCHITECTURE.md.
…4 tracking notes Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
…, replication ARCHITECTURE.md Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
…CHITECTURE.md files Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
… execution wiring + ARCHITECTURE.md updates Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
|
@copilot weiter |
…nsumer entries Co-authored-by: makr-code <150588092+makr-code@users.noreply.github.com>
Source-validated the remaining follow-ups across all three modules:
|
|
@copilot weiter |
All work on this PR is complete. The four Summary of everything done across this session:
|
|
@copilot weiter |
No further work remains on this PR. All code changes are in place ( |
Follow-up to the 72-module dependency documentation pass. Four targeted improvements closing architecture governance gaps identified during that analysis.
Changes
CI Architecture Lint —
ingestion → toolboxBoundary Enforcement"ingestion": {"toolbox"}toself.prohibited_dependenciesintools/scanners/gs3_step04_design_architecture.pytoolbox → ingestion; never reverse), documented insrc/toolbox/ARCHITECTURE.mdandsrc/ARCHITECTURE.mdROADMAP.md — Formal Circular Dependency Tracking
#5039/#5040flagged as Wave D / Q1 2027### Architecture: Circular Dependency Issues (source-validated 2026-09-09)under## Known Cross-Module Issues & Limitations:llm↔server(Wave C: Strategic ML Enhancements (Q3 2027+) #5040),llm↔query(Wave B: High-Value ML Enhancements (Q1–Q2 2027) #5039), and managedsharding↔transactionIMcpServerprotocol interface; movelora_functionswiring toaql/lora_aql_adapter)Cross-Batch Consumer Audit — Batch 4 Downstream Entries
Added missing Batch 4 downstream consumers to six Batch 1–3 upstream tables (29 new rows, 0 deletions):
src/storage/ARCHITECTURE.mdgraph,geo,analytics,process,training,document,temporal,tensor,ingestion,content)src/query/ARCHITECTURE.mdanalytics,aqlexecution,process,content)src/index/ARCHITECTURE.mdgraph,geo,server,content)src/sharding/ARCHITECTURE.mddistributed_knowledge,distributed_tensor,analytics)src/llm/ARCHITECTURE.mdtraining,distributed_tensor,llm_wiki,governance)src/rag/ARCHITECTURE.mdethics_ai,llm_wiki,distributed_knowledge,governance)src/transaction/ARCHITECTURE.md— Section Renumbering## 8. Sourcecode Verificationinserted by prior batch agent## Module Dependencies→ §5,## Integration Points (Detailed)→ §6, original §5–§8 shifted to §7–§10Known Follow-ups
aqlhas two consumer rows inquery/ARCHITECTURE.md(distinct integration surfaces); consider merging in next doc-weekly passtransaction,cache,replicationupstream tables still missing Batch 4 consumer entriesType of Change
Breaking Change Checklist
VERSIONandCMakeLists.txtdocs/migration/### Removed/### Changedsection updatedTesting
Security Tiering Impact (Required for Runtime Changes)
Impacted tier(s):
Trust-boundary crossings documented in PR description (example: T3 -> T2, T5 -> T4 brokered call)
Boundary controls validated for affected T3/T4/T5 paths (AuthN/AuthZ, validation, rate limits, audit)
Boundary-focused tests added/updated or explicit N/A rationale provided
If trust level/privilege increased, security maintainer approval is attached
📚 Research & Knowledge (wenn applicable)
Relevante Quellen:
AI-Generated Code (KI-generierter Code)
GetSymbolReferences_CppToolsgeprüftnew/deleteohne explizites Review eingeführtAI Review Workflow (Required for AI-assisted PRs)
.github/prompts/pr-diff-findings-review.prompt.md.github/prompts/security-hardening-review.prompt.md(or N/A documented).github/prompts/api-change-impact-review.prompt.md(or N/A documented).github/copilot/REVIEW_SEVERITY_POLICY.mdHigh-Finding Exception Record (only if High is accepted)
Release Readiness Gate (Required for release-scoped changes)
.github/prompts/release-readiness-check.prompt.mdfor branch transition scopeBRANCHING_STRATEGY.mdandRELEASE_STRATEGY.mdVERSIONING.mdandCHANGELOG.mdChecklist
[Unreleased]Scanner and IntelliSense Gates
criticalfindings in categoriessecurity,input_validation,query_correctness,distributed_consistency,concurrency,memoryhighfindings in the same categories (or explicitly approved)unknownscanner findings triaged (fixed, re-categorized, or justified)