feat(memory-core): temporal-pyramid L1/L2 durable aggregation writer (#14434)#14897
feat(memory-core): temporal-pyramid L1/L2 durable aggregation writer (#14434)#14897neo-opus-ada wants to merge 11 commits into
Conversation
…lds + doc composition (#14434) Leaf B of the temporal-pyramid substrate (epic #12679, per ADR 0028 §2.1/§2.4). The pure aggregation engine folds a window's fetched source rows into the six deterministic velocity fields (each bound to a named source per §2.4) and composes the temporal-summary document (id + five-field metadata + payload) through the Leaf-A schema — the FIRST consumer of createTemporalSummaryDocId / validateTemporalSummaryMetadata, which shipped with 0 consumers. Pure + I/O-free (the scheduled backpressure-leased service will own the fetch + upsert, mirroring the kbGarbageCollectionEngine split). Velocity fields ride the document payload, never the strict five-field metadata (which rejects extras). Idempotent per window+track+version; a version bump mints a new id (append-only). Tests: 7 passed. Durable JSDoc carries no ADR/ticket refs (archaeology hook). Next in-lane (same PR): the scheduled TemporalSummaryAggregationService (backpressure lease + L1/L2 window fetch + upsert) + ADR 0024 node-type update.
🚨 Agent PR Body Lint Violation@neo-opus-ada — your PR body on PR #14897 does not match the pull-request template structure. Required action: read
Do NOT compose a substitute template or hallucinate section headings. The validator Diagnostic hint: at least one recognized anchor like Visible anchors missing (full list)
This is the CI tool-boundary lint companion to PR #11494's MCP |
…temporal-summary engine (#14434) The half-open [00:00Z, next-00:00Z) daily window for L2 aggregation — the lane fetches source rows within the window and folds them via deriveVelocityFields. Fails closed on an unparseable anchor. Tests: 9 passed.
…ks) to the temporal-summary engine (#14434) ADR 0028 §2.6 partitioning: the unified track plus one '@<identity>' track per distinct agent seen in the window, de-duped + sorted with unified first; blank / non-'@' keys dropped. The aggregation lane folds each partition separately so per-agent future-self continuity + the unified view stay in step. Tests: 10 passed.
Progress + a service-pattern correction (build grounding)Pure core complete (3 commits, 10 unit tests green): Service-pattern correction (V-B-A caught this before it shipped wrong): ADR 0028 §2.1 mandates the orchestrator supervised-child scheduled-task pattern under the heavy-maintenance lease (the landed #12676 lane / ADR 0022 fairness) — not the standalone Remaining in this PR (service slice): the orchestrator-registered supervised-child aggregation task (lease-aware pulse: most-recent-first bounded window batch → per-partition — Ada (@neo-opus-ada) |
…d) to the temporal-summary engine (#14434) The unified partition track for one window — the whole-window fold under the 'unified' key, the always-present track the aggregation lane persists. Per-agent track composition is held pending the non-attributable-field semantics (a durable schema design Q raised to the epic owner). Tests: 11 passed.
…e-aware poll loop (#14434) The scheduled daemon (Neo.core.Base singleton, kb-gc poll-loop precedent) that runs the durable L1/L2 aggregation under the shared heavy-maintenance lease: opt-in start (no hidden default — fail-loud on a missing interval), idempotent lifecycle, and a pulse that acquires the lease, defers the whole cycle when another heavy-maintenance task holds it (the ADR 0022 fairness contract — never starves REM/defrag), and always releases in finally. runCycle composes the unified-track record per window via the pure engine. The read + upsert seams (collectPendingWindows/persistTemporalRecord) are overridable stubs; their durable-store implementations land with the source-fetch increment. Tests: 6 passed — opt-in no-op, enabled-schedule + idempotence, fail-loud on missing interval, lease-held deferral, acquire->run->release, and release-on-throw.
… the temporal-summary collection (#14434) The service now persists composed records: StorageRouter.getTemporalSummaryCollection() upsert keyed by the deterministic doc id (re-aggregation of the same window+track+version overwrites in place), the five-field metadata as the query contract + the velocity payload as the document body. The per-level SUMMARY_* graph label lands with the node-type-table update. Tests: 7 passed (adds the upsert-shape assertion).
… window batch) to the temporal-summary engine (#14434) The deterministic window plan for the aggregation lane: the UTC day containing the anchor + the preceding dayCount-1 days, contiguous + non-overlapping, most-recent-first. Bounds the lane to a fixed trailing batch (no unbounded history scan). Tests: 12 passed.
…ndow plan (#14434) collectPendingWindows now plans the trailing daily windows via planDailyWindows (anchor + count as overridable seams) and attaches each window's sources from the fetchWindowSources seam. The six per-substrate reads land behind that seam next; until then the lane plans windows + persists honest zero-count records. Tests: 8 passed.
… in fetchWindowSources (#14434) The first ADR 0028 §2.4 source binding: devCommits reads the dev first-parent commit log over [windowStart, windowEnd) via an injectable execCommand seam (git log --first-parent origin/dev --since --until --format=%H). The other five §2.4 sources bind behind the same seam next. Tests: 9 passed.
…cussions (#14434) Second ADR 0028 §2.4 source binding: sandboxesGraduated reads closed Discussions carrying a graduation marker, window-filtered by closedAt, via the shared execCommand seam (gh api graphql) — the same reader family as the handoff retrospective. Tests: 10 passed.
neo-opus-ada
force-pushed
the
agent/14434-temporal-pyramid-writer
branch
from
52689f8 to
e7494d9
Compare
…indowSources (#14434) Third ADR 0028 §2.4 source binding: adrsLanded reads the ADR decision records added under learn/agentos/decisions/ within the window (the AdrIngestor file source) via the git add-log through the execCommand seam. Cross-fetcher test isolation added so each source binding tests independently. Tests: 11 passed. mergedPrs / sessions / highImpactSessions (graph + Memory Core reads) bind next.
2 participants
Resolves #14434
Draft / WIP — Leaf B of the temporal-pyramid substrate (epic #12679), the L1/L2 durable aggregation lane + velocity fields. Opened in draft at the first committed slice; the service + daemon + ADR-0024 update land into this same PR before it leaves draft.
Why this lane, now (premise-verified)
Traced from #14811 (ADR 0033's direction-velocity leaf), which couldn't proceed — it wires
directionBreakdowninto "the temporal-pyramid single-writer," but that writer didn't exist:composeVelocity/createTemporalSummaryDocId/ the whole temporal-summary schema shipped as 0-consumer modules. The prerequisites for building the writer are both satisfied — ADR 0028 merged (PR #14428, 2026-07-02) and Leaf A #14433 closed (PR #14733, thetemporal-summarycollection +SUMMARY_*schema). So #14434 is the genuine actionable leaf; #14811 unblocks once this lands.What's in this first slice
ai/services/memory-core/helpers/temporalSummaryAggregationEngine.mjs— the pure aggregation engine (mirrors thekbGarbageCollectionEnginepure/IO split):deriveVelocityFields(sources)— folds a window's fetched rows into the six ADR 0028 §2.4 velocity fields (mergedPrs,devCommits,sessionsPerAgent,highImpactSessions,adrsLanded,sandboxesGraduated), each bound to a named source; honest0/{}for empty windows.buildTemporalSummaryDocument(...)— composes the doc via the Leaf-A schema; it is the first consumer ofcreateTemporalSummaryDocId/validateTemporalSummaryMetadata. Velocity fields ride the document payload; the strict five-field metadata rejects extras. Idempotent per window+track+version; a version bump mints a new id (append-only).VELOCITY_FIELD_SOURCES— the field→named-source contract (prose is never a metric source, ADR 0028 §2.4).Remaining in this PR (WIP checklist)
ai/daemons/temporal-summary/TemporalSummaryAggregationService.mjs— the supervised, backpressure-leased poll loop (reuses the landed#12676MaintenanceBackpressureServicelane; ADR 0022 fairness), most-recent-first bounded L1/L2 window batches, per-agent + unified partitions (§2.6), the six source fetches, Chroma+graph upsert, retention/versioning in theversionfield.ai/daemons/temporal-summary/daemon.mjs— the entry-point wrapper (Neo bootstrap + SIGTERM).SUMMARY_DAILYgraph labels written by this lane only (§2.3) — extractor untouched.Evidence
npm run test-unit -- test/playwright/unit/ai/services/memory-core/helpers/temporalSummaryAggregationEngine.spec.mjs→ 7 passed (worktree, exit 0): field folds, per-agent map, impact threshold, honest zeros, five-field metadata isolation, id idempotence + version-bump, fail-closed on inverted window. Evidence: L2 (unit) — the durable-write/scheduling ACs land with the service slice.Decision authority
ADR 0028 (temporal-pyramid substrate) §2.1/§2.3/§2.4/§2.6/§2.7 · ADR 0022 (heavy-maintenance scheduling fairness) · ADR 0024 (node-type obligation, updated in-PR) · reuses landed PR #12676 backpressure pattern. Durable JSDoc carries no ADR/ticket refs (archaeology hook); the authority lives here in the PR body.
Authored by Ada (@neo-opus-ada, Claude Opus 4.8, Claude Code). Session f1a4f6c4-46eb-4445-b315-2baa849990f3.