Problem
Dayflow chat currently relies mostly on date/range-based SQLite retrieval. That works for direct questions like "what did I do today?", but it is weak for fuzzy or topic-based questions such as:
- "When was I working on auth bugs?"
- "What kept distracting me during coding?"
- "Find times I was stuck debugging."
- "What did I do related to Stripe last week?"
For these, the assistant has to pull broad timeline/observation windows and scan them in context, which gets slower, more expensive, and less reliable as user history grows.
Proposal
Add a local LanceDB-backed semantic index for chat retrieval while keeping SQLite as the source of truth.
Suggested approach:
- Keep SQLite as canonical storage for timeline cards, observations, screenshots metadata, categories, and durations.
- Add a LanceDB index beside the existing SQLite database.
- Embed semantic chunks from:
- timeline cards
- observation groups
- optional daily/weekly summaries later
- Store metadata with each vector:
- source_type
- source_id
- day
- start_ts / end_ts
- category
- app/site metadata
- For chat, route queries by intent:
- clear date/range question -> existing SQLite fetchTimeline/fetchObservations path
- fuzzy/topic question -> LanceDB semantic search, then hydrate matching IDs from SQLite
- quantitative question -> semantic search for candidate IDs, SQLite for exact aggregation
Why LanceDB
LanceDB looks like a good fit for Dayflow because it can run locally/embedded, supports metadata filtering and hybrid retrieval patterns, and avoids adding server infrastructure. It also keeps the privacy model aligned with Dayflow's local-first design.
Acceptance criteria
- Chat can retrieve relevant activity by semantic topic, not only explicit dates.
- SQLite remains the source of truth; LanceDB only stores searchable chunks and source IDs.
- Indexing runs incrementally when timeline cards/observations are created or updated.
- Retrieval supports date/category/app filters where available.
- Existing date-based chat behavior does not regress.
- Add a lightweight rebuild path for the semantic index from SQLite.
Problem
Dayflow chat currently relies mostly on date/range-based SQLite retrieval. That works for direct questions like "what did I do today?", but it is weak for fuzzy or topic-based questions such as:
For these, the assistant has to pull broad timeline/observation windows and scan them in context, which gets slower, more expensive, and less reliable as user history grows.
Proposal
Add a local LanceDB-backed semantic index for chat retrieval while keeping SQLite as the source of truth.
Suggested approach:
Why LanceDB
LanceDB looks like a good fit for Dayflow because it can run locally/embedded, supports metadata filtering and hybrid retrieval patterns, and avoids adding server infrastructure. It also keeps the privacy model aligned with Dayflow's local-first design.
Acceptance criteria