Module metadata Roadmap

Metadata Module Roadmap

Current Status

Production metadata runtime exists across schema discovery, consistency tooling, lineage/export behavior, distributed metadata surfaces, and information-schema/statistics support.

In Progress

• [~] re-baseline p95/p99 envelopes for metadata access and cache operations (Target: Q1 2027)
• [~] broaden benchmark depth beyond cache-centric metadata hot paths (Target: Q1 2027)
• [~] harden long-running reliability under sustained schema mutation pressure (Target: Q1 2027)

Planned Features

Mid-term (6-12 months)

• expand deep concurrent-access stress coverage beyond focused unit tests (Target: Q2 2027)
• add operator runbook for metadata incident triage with actionable diagnostic steps (Target: Q2 2027)

Implementation Phases

Phase 1: Design / API Contract

• freeze schema/consistency/export contracts for active major line (Target: Q3 2026)
• define explicit error taxonomy for schema/export/distributed metadata failure classes (Target: Q3 2026)

Phase 2: Core Implementation

• complete hardening for metadata orchestration internals (Target: Q4 2026)
• align consistency/lineage/export behavior to bounded runtime contracts (Target: Q4 2026)

Phase 3: Error Handling and Edge Cases

• standardize fail-safe behavior for malformed schema and export failure scenarios (Target: Q4 2026)
• unify diagnostics across schema/consistency/export incidents (Target: Q4 2026)

Phase 4: Tests

• expand focused regressions for metadata consistency and export edge scenarios (Target: Q4 2026)
• extend deterministic stress fixtures for metadata cache and schema mutation operations (Target: Q4 2026)
• Phase A: schema churn stress + SchemaVersionManager lock contract (MCH-S01..S08, MCH-L01..L04)
• Phase B: consistency edge cases + lineage traversal + export failure paths (MCH-C01..C08, MCH-LN01..LN04, MCH-EX01..EX04)
• Phase C: distributed catalog diagnostics + RBAC diagnostics (MCH-DC01..DC04, MCH-SEC01..SEC04)

Phase 5: Performance and Hardening

• lock benchmark-backed release gates for metadata hot paths (Target: Q4 2026)
• validate p95/p99 and throughput behavior against release baselines (Target: Q4 2026)
• Phase D: broaden benchmarks to consistency/lineage hot paths (GATE-MCL-01..04)
• fix blocking_no_timeout gaps in schema_manager: removed unbounded shared-lock re-acquisition in getTable/getAllRelationships/getDatabaseMetadata; read under existing write lock instead (2026-08-04)
• fix circular_lock_ordering / deadlock_risk in statistics_collector: release refresh_mutex_ before acquiring cache_mutex_ in refreshLoop_ (2026-08-04)

Phase 6: Documentation and Acceptance

• core metadata module docs aligned to source-verifiable behavior
• roadmap/future planning separated from historical changelog entries
• benchmark README extended with full gate table (GATE-MET-01..04, GATE-MCL-01..04)
• fix schema_consistency_checker: removed const_cast in runCheck() — results_mutex_ already declared mutable (2026-08-04)

Production Readiness Checklist

• core metadata surfaces documented and source-verified
• module-level security and failure behavior documented
• benchmark mapping documented in performance expectations
• Phase 2/3 hardening closed: ConsistencyIssue diagnostics, ColumnRef contracts, fail-safe edge paths (MCH-01..MCH-08)
• Phase A–D tests and benchmarks delivered and registered (MCH-S/L/C/LN/EX/DC/SEC + GATE-MCL-01..04)
• release benchmark stabilization complete
• CRITICAL gap closure: blocking_no_timeout (schema_manager), circular_lock_ordering (statistics_collector), const_cast (schema_consistency_checker) — resolved 2026-08-04

Known Issues and Limitations

• runtime behavior depends on schema scale, cache state, and configured integration/export paths.
• deep concurrent-access stress beyond focused unit level still expanding (Q2 2027).
• operator runbook for metadata incident triage not yet written (Q2 2027).
• remaining gap scanner findings (scope_mismatch, string_concat_loop, copy_overhead, range_temporary) are MEDIUM/LOW severity and deferred to Q2 2027 maintenance sweep.

Breaking Changes

No breaking metadata contract planned. Any contract-breaking change requires migration notes and changelog entry before merge.

Program Execution Model — Wave Context

This module is a contributing module in the program-level Wave A → B → C → D execution model. It does not own a primary wave deliverable but must remain release_critical-green throughout all waves and must deliver Wave D operability improvements in Q1 2027. See [[../../ROADMAP.md|ROADMAP]] for the full wave model and exit criteria.

Wave D Contribution for metadata

• Deliver or validate distributed tracing, high-cardinality stress coverage, exporter reliability, and operator remediation hints as applicable to this module (Target: Q1 2027)
• Contribute to or validate long-duration soak test coverage for this module's primary paths (Target: Q1 2027)
• Ensure runbook coverage for operator-critical scenarios in this module (Target: Q1 2027)

Cross-Wave Requirements

• release_critical CI must remain green on develop throughout all waves (Target: ongoing)
• p95/p99 benchmarks must be refreshed on representative hardware before Wave D sign-off (Target: Q1 2027)
• No behavioral regression may be introduced into modules in Wave A/B/C scope from changes in this module.

Program-Level Success Criteria (contribution)

• This module's distributed/acceleration paths fail closed (Target: Q1 2027)
• Benchmark-backed p95/p99 baselines exist on representative hardware (Target: Q1 2027)
• Operator-critical paths have diagnostics, alerts, and runbooks (Target: Q1 2027)