From 568ec3b39b92720d261bbdfd7764c32ae4d147c5 Mon Sep 17 00:00:00 2001 From: nkkchem Date: Thu, 9 Jul 2026 07:33:44 -0700 Subject: [PATCH] RFC 007: Durable memory taxonomy & scope --- .../007-durable-memory-taxonomy-and-scope.md | 195 ++++++++++++++++++ contracts/rfcs/README.md | 2 + 2 files changed, 197 insertions(+) create mode 100644 contracts/rfcs/007-durable-memory-taxonomy-and-scope.md diff --git a/contracts/rfcs/007-durable-memory-taxonomy-and-scope.md b/contracts/rfcs/007-durable-memory-taxonomy-and-scope.md new file mode 100644 index 0000000..43a86e1 --- /dev/null +++ b/contracts/rfcs/007-durable-memory-taxonomy-and-scope.md @@ -0,0 +1,195 @@ +# RFC-007: Durable Memory Taxonomy & Scope + +- **Status:** Draft +- **Target:** `contracts/` — new schemas under `schemas/common/`, new ops in + `openapi/gmp-core-v3.yaml`, Memory tag in `README.md` +- **Domain:** Memory +- **Priority:** P2 +- **Author:** nkkchem +- **Origin:** [ARIAPlatform](https://github.com/brettin/ARIAPlatform_v0) alignment work +- **Supersedes/extends:** the current single-record Memory surface + +## Summary + +ARIA's Memory domain today specifies **one** durable surface — a generic +`MemoryRecord` with a nominal `kind` label — plus four ops +(write/query/summarize/forget). In practice only **episodic** memory (raw run +history + `summary` compaction) is meaningfully specified. A self-improving, +closed-loop scientific platform needs more. This RFC adds four durable memory +surfaces onto the existing one, resolves the sharing-scope ambiguity, and marks +working/scratch memory explicitly out of scope. Every addition is stated as a +delta on the current contract, with an explicit rationale for **why the existing +surface is insufficient**. + +## 1. Current state (what the contract has today) + +| Artifact | Path | Content | +|---|---|---| +| Ops | `contracts/openapi/gmp-core-v3.yaml` (L470–555) | `writeMemoryRecord`, `queryMemoryRecords`, `summarizeMemoryRecords`, `forgetMemoryRecords` | +| Record | `contracts/schemas/common/memory-record.schema.json` | `memoryId, agentId, kind, content, createdAt, policy` (+opt `sharing`); `kind ∈ {episodic, semantic, entity, summary}` | +| Policy | `contracts/schemas/common/memory-policy.schema.json` | `retentionSeconds, summarizationMode, sharingMode∈{private,tenant,cross_agent}, privacyTags` | +| Sharing | `contracts/schemas/common/memory-sharing.schema.json` | `ownerAgentId, sharedWith[], accessLevel∈{read,read_write,summary_only}` | + +The `MemoryRecord` is a single free-text blob differentiated only by a `kind` +string. `episodic` (append raw run history) and `summary` (compaction) are the +only kinds the ops actually exercise; `semantic` and `entity` are labels with no +distinct structure, lifecycle, or access rules. + +## 2. Why the existing surface is not enough (the distinction) + +The core problem: **one schema cannot express four different memory *contracts*.** +Each memory type has a different shape, different write/read semantics, a +different lifecycle, and a different auth/immutability requirement. Collapsing +them into `content: string + kind: enum` erases exactly the properties that make +each type useful. Concretely: + +| Question the loop must answer | Memory type that answers it | Can today's `MemoryRecord` answer it? | +|---|---|---| +| *"What happened on run X?"* | **Episodic** (have) | ✅ yes — this is the one it does | +| *"What do we now believe is true, and how sure are we?"* | **Semantic** (fact + confidence + evidence + validity) | ❌ no — no confidence, evidence, validity, or supersession; a belief can't be revised, only appended as more text | +| *"What sequence of steps actually worked, so I can run it again?"* | **Procedural** (recipe + steps + replay) | ❌ no — no steps, no capability refs, no replay op; a blob of text is not executable | +| *"Why was this decision made, and can I prove it wasn't tampered with?"* | **Provenance/Audit** (hash-chained, signed, immutable) | ❌ no — records are mutable/soft-deletable, no hash chain, no signature | +| *"What may agent B on this campaign see from agent A?"* | **Shared/collective** (scoped) | ⚠️ ambiguous — two overlapping mechanisms, `cross_agent` is unbounded, nothing enforced | + +**The self-improvement argument.** A closed loop improves by (1) *recording* what +happened → episodic; (2) *generalizing* it into beliefs → semantic; (3) +*crystallizing* winning strategies into reusable recipes → procedural; and (4) +*justifying* each autonomous action so a human/auditor can trust it → provenance. +Today only step (1) is real. Steps (2)–(4) either can't be expressed or can't be +trusted. That is the gap this RFC closes. + +**Design stance:** keep `MemoryRecord`/episodic exactly as-is (no breaking +change) and add the other surfaces *alongside* it, each with its own schema and +ops, all tied together by one shared `scope` primitive (§3). + +## 3. Shared scope model (the primitive every surface reuses) + +New `memory-scope.schema.json`: `level ∈ {agent, campaign, platform}` + `agentId` +(when agent), `campaignId` (when campaign), `tenantId` (always). Every new +surface references it. Access is enforced at read time against the caller's +capability token (IdentityTrust `establishSession`); cross-scope reads require an +explicit grant claim. + +- **agent** — private to one agent (default; = today's `private`). +- **campaign** — shared across agents cooperating on one project/run-group + (the missing middle; a bounded form of today's unbounded `cross_agent`). +- **platform** — tenant-wide curated knowledge, promotion-gated. + +## 4. Semantic memory — beliefs/facts (new) + +**Why (vs. existing):** episodic answers "what happened"; semantic answers "what +is true and how sure." Needs `confidence`, `evidence` links, a validity window, +and **supersession** so a belief can be *revised*, not just re-appended as text. + +- **Ops:** `writeSemanticFact` `POST /v3/memory/semantic`; + `querySemanticFacts` `POST /v3/memory/semantic/query` (filter subject/predicate/ + tag/`minConfidence`/validity-at-time); `promoteSemanticFact` (policy-gated + promotion of a `summarize` output to campaign/platform). +- **Schema `semantic-fact.schema.json`:** `factId, scope, proposition, + {subject,predicate,object}?, confidence[0..1], evidence[]{type,id}, + validFrom, validUntil?, supersedes[], createdAt, policy`. +- **Auth:** write = own agent; promote needs `memory:promote` + PolicyDecision; + reads per §3. +- **Distinction:** supersession = belief revision (writing with `supersedes` + closes the prior fact's validity, preserving history) — impossible with the + current append-only blob. + +## 5. Procedural memory — replayable recipes (new, highest priority) + +**Why (vs. existing):** this is the actual engine of *self-improving*. Episodic +recalls, semantic generalizes, but only procedural lets the loop **repeat a +winning strategy**. A text blob is not executable; a recipe with typed steps is. + +- **Ops:** `writeProcedure` `POST /v3/memory/procedures` (server assigns + `procedureId`+immutable `version`); `queryProcedures` (by goal/tag/ + `minSuccessRate`/required capabilities); `getProcedure` `GET .../{id}`; + `replayProcedure` `POST .../{id}/replay`. +- **Replay is governed, not direct:** it materializes a RunControl plan from the + steps and returns a `runId`, so replay re-passes policy/negotiation/budget + gates and **cannot escalate privilege**. Run completion feeds + `successCount/attemptCount` back via EventsProvenance (evidence-backed + reliability, not self-asserted). +- **Schema `procedure.schema.json`:** `procedureId, version, scope, goal, + preconditions[], steps[]{order,capabilityRef,operation,parameters,expects}, + successCriteria[], successCount, attemptCount, derivedFrom[runRef], createdAt, + policy`. +- **Auth:** write/version = own agent; replay requires the caller to already + hold tokens for every `capabilityRef`; promotion gated as §4. + +## 6. Shared/collective memory — SCOPE DECISION + +> **⚠️ DECISION REQUIRED.** Today's `sharingMode ∈ {private,tenant,cross_agent}` +> and `sharing.sharedWith[]` overlap, `cross_agent` is unbounded, and nothing is +> enforced. Pick one model. + +**Recommendation:** adopt the 3-level `scope` (§3), replacing the overlapping +pair. Per-agent alone is too isolated for multi-agent campaigns; platform-only +sharing leaks across unrelated projects (a DOE privacy hazard); **per-campaign** +is the missing middle bounding collaboration to the unit of scientific teamwork. + +**Auth boundary on cross-agent reads:** agent → owner only; campaign → any agent +whose token carries the matching `campaignId` claim (write needs membership + +`memory:write`); platform → any tenant agent reads, writes only via promotion. +Denied cross-scope reads are logged to provenance (§7). + +**Fallback if unratified:** keep `agent`-only as the conformance floor; make +`campaign`/`platform` an OPTIONAL profile extension; deprecate `sharedWith[]` +(kept for back-compat, superseded by `scope`). + +## 7. Provenance/audit memory — tamper-evident "why" (new) + +**Why (vs. existing & vs. the events journal):** EventsProvenance is an +*operational* stream (that something happened). Provenance/audit memory is a +durable, **immutable, verifiable** record of *why* a decision was made — required +to trust an autonomous agent in a DOE setting. Current records are mutable and +soft-deletable, with no hash chain or signature: not defensible in an audit. + +- **Ops:** `appendProvenanceRecord` `POST /v3/memory/provenance` (append-only; + server computes `recordHash` over payload+`previousRecordHash`, stores + `signature`/`signer`); `queryProvenanceRecords`; `verifyProvenanceChain` + (recompute chain + validate signatures, report first break). +- **Schema `provenance-record.schema.json`:** `recordId, scope, decision, + rationale, inputs[]{type,id}, policyVerdicts[]{policyId,verdict}, + approvals[]{approver,at}, recordHash, previousRecordHash?, signature, signer, + createdAt`. +- **Auth:** append needs `provenance:write` (platform/executor identity); read + needs `audit:read` scoped to run/campaign. **No update/delete for anyone** — + explicitly exempt from the Memory `forget` op (forgetting would defeat audit). + +## 8. Working/scratch memory — explicitly OUT OF SCOPE + +Ephemeral in-run state (agent scratchpad) is **not** durable memory. It is +covered by the Run lifecycle (execution context + checkpoints for pause/resume). +Adding it to the durable contract would blur retention/privacy rules. The write +into episodic/semantic/procedural memory *is* the boundary between scratch and +durable. No schema, no op; use RunControl checkpoints if persistence across +pause/resume is needed. + +## 9. Migration & compatibility + +- **Non-breaking:** `MemoryRecord` and its 4 ops are unchanged; episodic keeps + working. New surfaces are additive resources. +- **Sharing:** `scope` is introduced additively; `sharingMode`/`sharedWith[]` + are **deprecated** (retained, superseded by `scope`) pending §6 ratification. +- **Conformance:** episodic + `agent` scope = the floor. Semantic/procedural/ + provenance and `campaign`/`platform` scope land as profile extensions so + implementations aren't forced to build everything at once. +- **Landscape (candidate conforming backends, not dependencies):** Mem0 + (semantic+episodic fusion), Letta/MemGPT (working↔durable paging), Zep/Graphiti + (temporal KG for semantic), Cognee (graph ETL). Selected via capability + routing behind the contract — never a hard spec dependency. + +## 10. Summary + +| # | Surface | Status today | New primitive | Why existing surface can't do it | +|---|---|---|---|---| +| Episodic | have | `MemoryRecord` (unchanged) | — | ✅ it's the one that works | +| Semantic | label only | `SemanticFact` | no confidence/evidence/validity/supersession → beliefs can't be revised | +| Procedural | **missing** | `Procedure` + governed `replay` | a text blob isn't executable/replayable | +| Shared | **decision** | `memory-scope` (agent/campaign/platform) | overlapping, unbounded, unenforced sharing | +| Provenance | missing | hash-chained signed `ProvenanceRecord` | records are mutable/deletable, no chain/signature | +| Working | out of scope | none (use checkpoints) | ephemeral; doesn't belong in durable contract | + +**Open item for reviewers:** ratify §6. On ratification this RFC is submitted as +a PR to the ARIA spec repo (new schemas + openapi ops + README Memory tag). + diff --git a/contracts/rfcs/README.md b/contracts/rfcs/README.md index 1453cae..cf58a58 100644 --- a/contracts/rfcs/README.md +++ b/contracts/rfcs/README.md @@ -10,6 +10,7 @@ Draft RFCs for GMP core-v3 contract changes proposed from [ARIAPlatform](https:/ | [003](003-run-status-vocabulary.md) | Run status enum | P1 | | [004](004-budgets-allocate-enforce.md) | Budget allocate and enforce | P1 | | [006](006-capability-registry-platform.md) | Capability registry vs spec Capability | P2 | +| [007](007-durable-memory-taxonomy-and-scope.md) | Durable memory taxonomy & scope | P2 | ## Suggested merge order @@ -19,5 +20,6 @@ Draft RFCs for GMP core-v3 contract changes proposed from [ARIAPlatform](https:/ 4. RFC 003 — Run status vocabulary 5. RFC 004 — Budget allocate and enforce 6. RFC 006 — Capability registry vs spec `Capability` +7. RFC 007 — Durable memory taxonomy & scope Until an RFC merges, platform v0 may continue to document wire behavior in [`platform-v0-implementation-profile.json`](https://github.com/brettin/ARIAPlatform_v0/blob/main/reference/platform-v0-implementation-profile.json).