Category: Ingestion & Data Management
Priority: Low
Phase: Phase 2 (Ingestion + CLI)
Problem
Idempotent ingestion is asserted but the mechanism is unstated; a naive full-scan to detect changes doesn't scale across hundreds of files.
Solution
Maintain a manifest table (file path, content hash, ingest timestamp, chunk count) in Postgres; only parse/embed files with a changed or missing hash.
Category: Ingestion & Data Management
Priority: Low
Phase: Phase 2 (Ingestion + CLI)
Problem
Idempotent ingestion is asserted but the mechanism is unstated; a naive full-scan to detect changes doesn't scale across hundreds of files.
Solution
Maintain a manifest table (file path, content hash, ingest timestamp, chunk count) in Postgres; only parse/embed files with a changed or missing hash.