Feature: ingest redaction and cardinality governance

[awksedgreep]

Summary

Add a versioned ingest-governance policy that protects embedded deployments from sensitive-data retention, oversized values, and cardinality explosions before values enter durable blocks or indexes.

Candidate policy actions

• Drop configured resource, metadata, attribute, event, link, or label paths.
• Deterministically hash/tokenize selected values with explicit key management.
• Truncate values at documented UTF-8/byte boundaries.
• Reject or quarantine prohibited records under a bounded policy.
• Bound metric series and distinct selected values per configured scope.
• Report top cardinality contributors without exposing sensitive values.
• Apply allowlists/denylists to dynamic keys and headers.

Contract

• Policy parsing is strict, bounded, versioned, and fail-closed when enabled.
• Transformation happens before compression dictionaries, term/trigram indexes, trace attribute blooms, live-tail publication, and durable admission.
• Preserve native types unless the configured action explicitly replaces a value.
• Counters distinguish accepted, mutated, rejected, and policy-error records by non-sensitive reason.
• Policy identity is recorded with stored data or store metadata sufficiently to explain historical behavior.
• Policy replacement is atomic and has explicit effects on already stored data.

Cardinality safety

• Define exact scope and reset semantics for budgets.
• Avoid unbounded in-memory distinct-value tracking; use measured bounded structures with explicit false-positive behavior if approximate.
• Make partial-batch acceptance rules explicit for every ingestion protocol.

Acceptance criteria

• Tests prove removed plaintext does not reach public rows or any newly written index/block representation.
• Hashing, UTF-8 truncation, nested JSON paths, arrays, missing/null/type distinctions, and hostile structures are pinned.
• Transactions, batches, live tail, rollback, reopen, backup, and policy replacement remain exact.
• Resource overhead and cardinality enforcement are bounded under adversarial input.