Skip to content

Ingest validated Cockroach Crawler SourceRecords #7

Description

@AjnasNB

Current state

The Qarinah-side crawler interoperability bridge shipped on main in #15 (merge commit e6fc3dca9e30fc7e243f2ca35a05e9ec8aa373fe).

The end-to-end trust boundary is still incomplete: Cockroach Crawler does not yet expose the canonical public runtime validator/content-identity contract required here. That upstream work is tracked in AjnasNB/cockroach-crawler#36, so this issue remains open.

Remaining scope

Consume only crawler records accepted by the crawler-owned public runtime validator, then map them into evidence-linked Qarinah context without adding a Qarinah dependency to the crawler.

Acceptance criteria

  • Define a canonical SourceRecord runtime validator and content identity contract cockroach-crawler#36 ships a public, versioned normalize/validate SourceRecord API first.
  • The bridge accepts the validated canonical record shape and preserves its schema version, acquisition identity, content hash, source locator, revision identity, citations, authors, and bounded acquisition provenance.
  • Source content remains untrusted data and never becomes executable instructions, Maqam policy, approval, or authentication material.
  • contentHash drives deterministic duplicate/revision behavior according to the crawler-owned identity contract.
  • Re-acquisition, URL aliases, real revisions, hash mismatch, missing citations, malformed provenance, unsupported schema versions, and oversized records have deterministic fixtures.
  • Rejection occurs before ledger mutation; a failed import cannot leave partial nodes, relations, checkpoints, or content-pack entries.
  • Tests run offline on Node 20, 22, and 24 with no crawl, DNS lookup, account, key, model call, or network request.
  • Documentation names the separate ownership boundaries and does not claim that validation proves source truth or safety.

Boundaries

Do not duplicate the crawler validator inside Qarinah, make the crawler depend on Qarinah, execute retrieved instructions, or describe the integration as end-to-end shipped until both public contracts are merged and verified.

Metadata

Metadata

Assignees

No one assigned

    Labels

    integrationAgent, governance, crawler, or platform integrationroadmapPlanned beyond the initial foundationsecurity-boundaryRequires explicit threat-boundary review

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions