[DM-036] Implement collective-memory source and reviewed-publication adapters

[nicoechaniz]

Outcome

Implement two deliberately separate collective-memory adapters:

1. an inbound collective-source adapter that imports an immutable,
   attributable generation into Daimon source quarantine; and
2. an outbound reviewed-publication adapter that publishes only an
   explicitly consented derived artifact after independent exact-byte review.

They must not share credentials, queues, receipts, authority, or a live SQLite
database. Collective knowledge never becomes autobiographical /me memory,
and outbound approval is never inferred from inbound-source trust.

Blocked by

• DM-015 (closed): source publication, grant, and quarantine semantics.
• DM-018: closed adapter invocation/result/manifest contracts.
• DM-023: authoritative ledger/projection synchronization and correction rules.
• [DM-036 dependency] Add provenance-safe snapshot/export and reviewed Matrix publication boundary Mar-IA-no/collective-memory#2: supported provenance-safe export/publication
  boundary. Until it lands, only synthetic adapter fixtures may be used.

Audited upstream boundary

At Mar-IA-no/collective-memory main
ca0486307aea6ea8ec65f70ffd5ba32fd56b8850 (MIT):

• librarian.sh is single-writer under flock, stages a map generation, then
  atomically swaps the published snapshot symlink;
• tier1.py rebuilds index.db.tmp and publishes it with os.replace;
• ui_builder.py publishes a generation directory and swaps .mapa/ui;
• /ui/v2/* reads a regenerable ui_v2.db projection in SQLite read-only mode;
• current manifests/generation APIs are optimized for navigation. They do not
  form a complete immutable exchange contract carrying original authors,
  license, consent, classification, source lineage, supersession, and
  tombstones, and there is no supported external publication transaction.

Therefore DM-036 must not copy index.db, ui_v2.db, WAL/SHM files, scrape an
undocumented schema, write corpus paths directly, or treat a UI generation as
personal-memory authority.

Inbound collective-source contract

• Consume only a supported versioned API or crash-consistent immutable
  export bundle. Bind producer instance/release, export generation,
  manifest hash, creation time, policy version, and every artifact's stable
  logical ID, content hash/length/media type, authors/principals, original
  source refs, license, consent scope, classification, supersession, and
  active/tombstone state.
• Verify canonical encoding, all hashes/lengths, bounded counts/sizes,
  declared media types, unique IDs, predecessor/generation continuity, and
  configured producer authenticity before any state effect. Reject unknown
  fields/versions, partial bundles, traversal/symlinks, executable refs,
  dangling lineage, hash collisions, regressions, and mixed generations.
• Convert accepted artifacts into DM-015 source publications/claims and
  quarantine records with exact origin and importer version. Never emit a
  personal-memory assertion, category, /me fact, membership, consent, or
  capability merely because collective-memory contains the text.
• Plan/preview is non-mutating. Apply is idempotent by exact producer,
  artifact ID, generation, and content hash. A changed successor requires
  the accepted predecessor; a tombstone retracts only that imported
  projection while retaining ledger/source history.
• A failed fetch, timeout, invalid bundle, policy rejection, crash, or
  interrupted apply keeps the prior accepted generation active and records
  a closed redacted error. Recovery deterministically completes or rolls
  back; it never leaves a mixed generation.
• Source pull credentials/configuration are read-only and cannot call the
  outbound publisher. Remote bytes are inert and cannot select adapters,
  targets, commands, paths, database handles, keys, grants, or reviewers.

Outbound reviewed-publication contract

• Accept only a separately rendered, policy-approved derived artifact with
  exact Matrix source event/checkpoint refs, classification, consent grant,
  review policy, target logical ID, predecessor receipt, content hash,
  media type, and bounded inert bytes. Raw personal-memory rows, prompts,
  session logs, keys, grants, or databases are never publishable inputs.
• Require explicit subject/owner consent plus an independent signed human
  approval bound to the exact final rendered bytes, target, classification,
  policy, and source checkpoint. An inbound producer/source attestation is
  not an outbound approval and the Librarian/model cannot self-approve.
• Deterministically plan an allowlisted logical target, render all metadata,
  links, title/frontmatter and body, then scan the final bytes for
  secrets/private material before applying. No request-supplied host path,
  URL, command, template, database handle, arbitrary filename, or SQL is
  resolved or executed.
• Apply through the supported collective-memory transaction under its
  process-wide writer boundary. Artifact, manifest/audit entry, index/map/UI
  generation, and durable content-addressed receipt publish all-old or
  all-new. Failure and restart cannot expose an unreceipted/mixed view.
• Identical retries return the same receipt. Drift, changed bytes with the
  same idempotency identity, stale predecessor/review/consent, or an
  untracked target fail closed. Updates and revocations are explicit
  successor versions/tombstones; no broad delete or silent overwrite.
• Publication receipts retain exact Matrix and collective-memory source/
  destination IDs, before/after hashes, generation, review/consent refs,
  transaction status, and supersession/tombstone relation without exposing
  private paths or credentials.

Separation invariants

• Inbound and outbound use distinct adapter IDs, manifests, capability
  declarations, identities/keys, configuration, queues, idempotency spaces,
  stores, logs, metrics, and receipts.
• Granting inbound read access cannot confer outbound write, review, or
  consent authority. Revoking one direction does not silently revoke or
  activate the other.
• Matrix remains authority for /me, ledger history, memory corrections,
  classification, grants, reviews, and adapter policy. Collective-memory
  remains authority for its own corpus/generations. Each side stores only a
  rebuildable projection plus receipts for the other.
• Adapter outputs use DM-018 logical content references and closed errors;
  no host path, SQLite connection, model prompt, credential, or private
  deployment identifier crosses the contract.

Required tests

• Contract/schema vectors for valid inbound export, outbound plan/apply and
  receipts, including canonical ordering and deterministic hashes.
• Inbound: initial generation, identical retry, successor, tombstone,
  offline catch-up, pagination/bounds, failed refresh and rebuild from the
  authoritative Matrix ledger/source log.
• Inbound negatives: tampered/mixed/partial generation, duplicate ID,
  same ID+hash with different bytes, predecessor regression/fork, missing
  author/license/consent/source, unknown field/version/media type,
  traversal/symlink, oversized input, timeout and response loss.
• Outbound: deterministic preview, approved publish, retry, successor and
  reviewed tombstone with exact receipt/provenance verification.
• Outbound negatives: approval for different bytes/target/checkpoint,
  expired/revoked consent, self-review, final-render secret in title,
  frontmatter/link/body, target drift, untracked revoke, concurrent writers,
  response loss, and failure/crash at every transaction stage.
• Direction-confusion tests prove inbound credentials/events cannot publish,
  outbound receipts cannot assert source trust, and collective text cannot
  enter personal-memory projections without the separate DM-023 policy path.
• Real-I/O integration against an isolated collective-memory instance:
  export -> Matrix quarantine/accept -> reviewed derived publish -> atomic
  reindex/search/Atlas visibility -> tombstone/reindex. Verify old-or-new
  views, SQLite integrity, no shared database/WAL, no network/model call in
  commit, and no effect on an unrelated corpus.
• Preserve all Matrix and collective-memory tests; scan fixtures, receipts,
  logs, bundles, and repository diff for secrets/private content.

Reuse/adaptation decision

• Reuse the behavior of collective-memory's current flock + staging +
  atomic snapshot/index/UI swaps, but consume only the new supported exchange
  boundary from [DM-036 dependency] Add provenance-safe snapshot/export and reviewed Matrix publication boundary Mar-IA-no/collective-memory#2.
• Reuse the reviewed HMK publication flow merged as
  91f581c89b45fcbd5e0b30f04a951a98bed42351 (reviewed implementation head
  bad0501) only as outbound policy/test evidence. Adapt it to DM-018 contracts,
  final-render scanning, separate direction authority, stable upstream receipts,
  and rejection of untracked revocation; do not copy live state or databases.
• Existing Proposal: reviewed HMK publication artifacts as a downstream corpus Mar-IA-no/collective-memory#1 records the earlier HMK-as-downstream
  proposal; [DM-001] Publish the maintained foundation and V0 ontology #2 is the concrete bidirectional boundary required by DM-036.

Evidence inventory: CONCURRENT-WORK-AUDIT.md.

Security and rollback

Treat every remote/local artifact as untrusted inert content. Never include
private memory, root/operational keys, provider credentials, raw DB/WAL files,
host inventory, prompts, sessions, or arbitrary paths. Before live deployment,
rollback removes the unused adapters/schemas. After accepted imports or
publications exist, rollback disables capabilities and writes explicit successor
receipts/tombstones while preserving ledger/source/review history and high-water
marks; it never deletes external data by inference.

Deployment: N/A. Synthetic and isolated local instances only until the later
consent/canary cards authorize a real corpus.

Expected PR

One focused Matrix PR after dependencies land: separate source/publisher adapter
implementations, closed schemas/manifests, durable state/receipts, policy gates,
real-I/O isolated integration and adversarial crash/concurrency tests, plus the
operator runbook. No live collective-memory or CompAII deployment change.

[nicoechaniz]

Pre-V0 integration boundary confirmed: existing hermes-memory-kit work provides the reviewed outbound publication half only. DM-036 must implement inbound collective knowledge as an attributed external/source quarantine path and outbound publication as a separate consented, reviewed adapter with final-artifact credential scanning, receipts, revocation, and tombstones. Neither direction may treat collective-memory rows as personal lived experience.

[nicoechaniz]

DM-036 integration candidate published as draft PR #115, stacked on the current DM-083 branch.

Exact boundary:

• PR head: c85f2fcda38ec79e78c1cca9ef10175c2088232f
• semantic merge: 56f79a1a9128284ef53c4289343162ba84020ad7
• collective-memory contract: 3e3b39416917f8e3c2bc5ca69362b20296205938
• registry: 101 required scenarios, digest 12f01cb1704e9943b7f2069ba41fa3f7f9b153c8cf5a22f506e404ac918b963f

Qualification passed: 23 DM-036 contract/crash/real-I/O tests locally and on isolated daimonmatrix scratch state; complete Matrix suite 566 passed/19 skipped; workflow-equivalent Ruff/mypy/generator gates clean; wheel and sdist reproducible and secret-free; installed conformance ran twice byte-identically with release_ready=true.

The issue remains correctly blocked and unclaimed: Mar-IA-no/collective-memory#3 still needs independent review/merge. PR #115 therefore has no closing keyword or invented Claim-ID and is explicitly a candidate only. After the upstream dependency lands, the next step is a signed Matrix claim, converting the PR to the canonical review state, and independent Matrix review. No deployment or live corpus mutation occurred.