I would love to use CASS with Shelley (https://github.com/boldsoftware/shelley).
I could contribute the connector code, but I think you don't accept code contributions.
I am enclosing a design doc (AI-generated) - hopefully it can be easily turned into beads.
Thank you.
Summary
Shelley stores conversations and messages in one SQLite database rather than one
file per session. The database uses WAL mode and may be placed anywhere with
Shelley's global -db flag; the CLI default is shelley.db relative to the
Shelley process's working directory.
The clean implementation boundary is:
- add Shelley database detection, read-only querying, normalization, source
discovery, and direct token extraction to
franken_agent_detection
(FAD);
- bump CASS's pinned FAD revision;
- add CASS's explicit watch, source, raw-mirror policy, metadata-refresh,
capability, presentation, contract-test, and documentation integration.
No Shelley code or schema change is required for the first implementation.
Use the SQLite database directly; do not require a running Shelley server.
Suggested implementation slices
Treat this issue as three reviewable PRs rather than one monolithic patch:
- FAD connector: schema-aware local detection, safe SQLite reads, source
discovery, projection, limits, fixtures, and direct usage extraction.
- CASS batch integration: dependency/re-export/registry wiring, explicit
local source configuration, raw-mirror denial, capabilities, presentation,
metadata refresh, and end-to-end batch indexing.
- CASS live integration: DB/WAL/SHM watch routing, metadata-only change
handling, watch diagnostics, and operational regression tests.
The issue is complete after all three land. Live SSH/remote SQLite sync, exact
multi-call indirect-usage analytics, and a Shelley export API remain follow-ups.
Revisions checked
Checked on August 24, 2026.
CASS currently pins FAD 0.2.1 with selected SQLite connector features in
Cargo.toml
and repeats that exact contract in
build.rs.
Why this needs a dedicated connector
Shelley is not a safe alias for an existing SQLite-backed connector:
- its schema and message JSON are Shelley-specific;
- one database contains many conversations;
- its database location is configurable and its CLI default is relative to the
server process's working directory
(flag definition);
- it uses WAL mode with concurrent readers and a writer
(pool initialization);
- its database also contains non-conversation tables with API keys and other
sensitive configuration, so CASS must not raw-mirror the whole file.
Do not detect Shelley by a generic *.db or shelley.db filename alone.
Admission requires a Shelley schema signature.
Shelley database contract
Shelley's migrations are the authoritative schema history
(db/schema).
The current generated row shapes are in
db/generated/models.go.
Minimum supported schema
Support databases migrated through 003-add-message-sequence.sql. Require:
migrations(migration_name, ...)
conversations(
conversation_id, slug, user_initiated, created_at, updated_at
)
messages(
message_id, conversation_id, sequence_id, type,
llm_data, user_data, usage_data, created_at
)
Require the migration records for 001-conversations.sql, 002-messages.sql,
and 003-add-message-sequence.sql when identifying an automatically discovered
candidate. An explicitly configured candidate may be admitted by the exact
required table/column signature so sanitized fixtures and repaired databases
remain usable.
Read these newer columns when present:
conversations:
cwd, archived, parent_conversation_id, model, conversation_options,
current_generation, tags, is_draft, draft, queued_messages
messages:
display_data, excluded_from_context, generation, llm_api_url, model_name,
forked_from_message_id, user_email, other_usage_data
Treat unknown later migrations, columns, message types, and JSON fields as
forward-compatible additions. Missing required columns are an incompatible
schema error, not an empty successful scan.
Ordering and identity
-
messages.sequence_id is authoritative within a conversation. Order by
(sequence_id, message_id), never by timestamp alone. Set
NormalizedMessage.idx = sequence_id; do not reindex gaps away. Duplicate
sequence IDs are source corruption.
-
A Shelley conversation_id is stable within one database. It is not globally
namespaced across independent databases.
-
Use a database path namespace to avoid collisions when two local Shelley
databases contain the same conversation ID:
external_id = "shelley:" + first_16_hex(blake3(canonical_db_path))
+ ":" + conversation_id
-
Preserve the original Shelley conversation_id in metadata. Moving or
copying a database to another path intentionally creates a new namespace.
-
Use the canonical database path as NormalizedConversation.source_path.
Current CASS health, retry, trust, doctor, and source-opening paths expect a
real file. The namespaced external_id distinguishes conversations sharing
one source file.
Proposed FAD implementation
Add src/connectors/shelley.rs with ShelleyConnector.
Feature and registry changes
In FAD:
- add optional
blake3 and url dependencies and reuse FAD's existing
optional base64 dependency;
- define
shelley = ["connectors", "dep:frankensqlite", "frankensqlite/fts5", "dep:asupersync", "dep:blake3", "dep:base64", "dep:url"];
- include
shelley in all-connectors and add feature = "shelley" to the
cfg list that exposes the sqlite_sync module (there is no separate
sqlite_sync Cargo feature);
- export/register
ShelleyConnector in src/connectors/mod.rs and src/lib.rs;
- add
shelley to KNOWN_CONNECTORS, canonical slug aliases, environment
overrides, and local default probes; give default_probe_paths_tilde() an
empty Shelley arm until safe remote export/sync exists;
- route Shelley's global
entry_from_detect() path through schema-aware
validation, never generic path-existence-only detect_roots();
- update FAD's registry-consistency test/table and exempt Shelley's intentionally
empty default_probe_paths_tilde() arm from any non-empty-path assertion;
- add
shelley to extract_tokens_for_agent().
Use FAD's existing synchronous FrankenSQLite bridge, not rusqlite
(sqlite_sync.rs).
crush.rs is the simplest single-database template; opencode.rs is the best
reference for candidate expansion, schema variants, and preventing remote scans
from importing local defaults.
Candidate path rules
Use one candidate-expansion and schema-admission helper for detect(),
scan(), and discover_source_files(). Do not delegate Shelley detection to
FAD's generic path-existence report: ShelleyConnector::detect() must open and
validate candidates read-only so detection and scanning cannot disagree.
Priority:
- explicit file roots, accepting any filename after schema validation;
- explicit directory roots expanded to
<root>/shelley.db and, when the root
represents a home directory, <root>/.config/shelley/shelley.db;
- new connector-only override
CASS_SHELLEY_DB (treated as an explicit
candidate);
- optional CASS preset
~/.config/shelley/shelley.db—this is a deployment
convention, not Shelley's CLI default;
./shelley.db and ~/shelley.db, each still requiring schema validation.
Rules:
- never recursively search
$HOME for SQLite databases;
- never admit an arbitrary database by basename alone;
- when any supplied scan root is remote, do not add local default candidates;
- remote roots without a schema-admitted Shelley candidate are ordinary
non-matches because CASS offers every configured root to every connector;
- if a remote root does contain a validated Shelley database, reject it in the
first implementation with an actionable message: current CASS SSH sync does
not copy a live SQLite database and its WAL/SHM as one consistent, sanitized
bundle;
- canonicalize and deduplicate candidates before opening them.
Safe SQLite access
For each admitted database:
- open with
SQLITE_OPEN_READ_ONLY;
- set connection-local
PRAGMA query_only=ON, PRAGMA trusted_schema=OFF,
and a bounded busy timeout;
- verify
migrations, conversations, and messages are tables, not views;
- do not use
immutable=1 against a live database because committed rows
may exist only in the WAL;
- validate schema, select the conversation inventory, and read messages inside
one read transaction so the result is a coherent committed snapshot;
- never run migrations, checkpoints,
VACUUM, ANALYZE, repair, or any
write-capable pragma;
- close the read transaction promptly so Shelley can checkpoint its WAL.
A release-blocking test must prove that the exact SQLite stack used by FAD can
see a committed Shelley row present only in shelley.db-wal, while hashes,
sizes, mtimes, and inode identity of the database and sidecars remain unchanged.
If that cannot be guaranteed, an explicitly configured scan must fail with an
actionable connector error rather than silently indexing stale main-file
contents. Automatic probes should ignore schema non-matches; they must not hide
I/O or corruption failures for a path that already passed Shelley admission.
Source discovery
discover_source_files() should return:
- the main database as
DiscoveredSourceRole::SqliteDatabase;
- a present
-wal sidecar as required reconstruction metadata;
- a present
-shm sidecar as optional metadata.
The connector consumes the main database through SQLite, not by separately
parsing sidecar bytes. Discovery exists for provenance, diagnostics, and watch
coverage. CASS must apply the Shelley-specific raw-mirror exclusion described
below before capturing these files.
Query strategy
For a full scan, read every non-draft conversation and its messages. For an
incremental scan with since_ts:
- sweep the lightweight
conversations table on every DB/WAL event, because
archive, tag, and option updates do not reliably bump updated_at;
- select full transcripts for new conversations and conversations with recent
messages, using a small overlap window (for example two seconds);
- emit metadata-only refresh records for other already-indexed conversations,
marked metadata.shelley.metadata_only=true; CASS must refresh an existing
row from these records but never create a new empty conversation from one.
This catches metadata-only changes without reloading every message blob on every
turn. New/fresh conversations always emit a complete transcript.
Implement true callback-based scanning. Extend sqlite_sync with a bounded
row-callback helper and a safe read-transaction guard rather than loading the
entire database into one Vec.
Conversation projection
One Shelley conversations row becomes one normalized CASS conversation.
| Shelley field |
Projection |
conversation_id |
Namespaced external_id; raw ID retained in metadata. |
slug |
Title when non-empty; otherwise first non-empty user text, bounded to 120 characters. |
cwd |
workspace. |
created_at |
started_at. |
| latest message time |
ended_at; fall back to updated_at when there are no messages. |
archived |
Metadata; archived conversations remain searchable. |
user_initiated |
Preserve as independent metadata. Do not use it alone to classify subagents. |
non-null parent_conversation_id |
Separate subagent conversation with parent metadata; never flatten into the parent. |
model |
Current configured model metadata only; historical message model comes from each message. |
current_generation |
Conversation metadata; preserve each message's raw generation without baking mutable active/superseded labels into message extras. |
tags |
Bounded metadata. |
is_draft, draft |
Skip draft-only conversations and never index draft text. |
| no normalized messages |
Skip on a full scan; metadata-only refresh records may target an existing CASS row. |
queued_messages |
Never index; queued input is transient and not yet part of the message log. |
messages.excluded_from_context |
Preserve in message extra; index otherwise-visible stored content unless another explicit suppression rule applies. |
agent_working |
Omit; it is runtime state. |
conversation_options |
Preserve only allowlisted non-secret values such as thinking_level, disable_all_tools, and disable_notifications. Never retain hook URLs. |
Recommended metadata shape:
{
"source": "shelley",
"shelley": {
"conversation_id": "cABC123",
"database_namespace": "0123456789abcdef",
"user_initiated": true,
"parent_conversation_id": null,
"archived": false,
"current_generation": 2,
"configured_model": "opus",
"tags": [],
"message_count": 42,
"omitted_system_prompt_count": 1,
"omitted_carried_message_count": 8
}
}
Do not copy draft bodies, queued message bodies, raw participant email
addresses, hook URLs, or whole conversation rows into metadata.
Generations, compaction, forks, and subagents
- Preserve unique messages from all generations for archival search; do not
filter only to current_generation.
- Preserve each row's raw
generation and the conversation's
current_generation; do not persist a derived active/superseded label that
would become stale after later compaction.
- Shelley re-records recent messages into a new generation with
user_data.compaction_carried="true" and a fresh row timestamp. For carried
rows only, match the nearest unmatched prior-generation row by canonical
source type, LLM role, normalized content/tool structure, and allowlisted
user data—not by row timestamp. Count the omission in conversation
metadata. If no earlier match exists, retain it and mark it
unmatched_carried=true rather than dropping data.
- A distillation summary is real searchable content. Prefer
user_data.distillation_content when user_data.distilled="true".
- Forks remain separate conversations. Preserve
forked_from_message_id on
copied messages. Their text remains searchable in the fork, but their copied
usage_data must not be counted again.
- Subagents remain separate conversations with parent identifiers in metadata.
CASS's CASS_SKIP_SUBAGENTS predicate must be extended to recognize Shelley
parent metadata, not only Claude's subagents/ path layout.
Message and content projection
Shelley's canonical persisted message shape is llm.Message
(llm.Message and llm.Content).
Because the Go constants share one iota block, persisted content type numbers
are:
2 text
3 thinking
4 redacted thinking
5 tool use
6 tool result
7 server tool use
8 web-search tool result set
9 individual web-search result
Parse llm_data as the canonical source. Use user_data and display_data
only through explicit per-message-type allowlists. Maintain a transcript-wide
ToolUseID -> tool name map so a later tool-result row can resolve the call
recorded in an earlier assistant row.
Role mapping
| Shelley row/content |
Normalized role |
ordinary user row |
user |
agent row |
assistant |
| a user/tool row containing only tool-result content |
tool |
error |
system, preserving retry/refusal fields |
warning, gitinfo, modelchange |
labeled system event |
| distillation status or summary |
labeled system event/summary |
| unknown row with a valid LLM role |
role derived from llm_data.Role |
| otherwise unknown row |
system, preserving the original type |
slug |
omit from searchable turns; retain omission/usage metadata only |
Do not assume messages.type='tool' for tool results. Shelley normally stores a
tool-result LLM message with user role, so inspect content blocks.
Content blocks
Render blocks in source order:
| Shelley content |
Search projection |
| text |
Plain text. |
| thinking |
[Thinking]\n plus plaintext thinking. |
| redacted thinking |
[Redacted thinking]; never emit Data or Signature. |
| tool use |
[Tool call: NAME]\n plus bounded canonical JSON arguments; also emit NormalizedInvocation. |
| tool result |
[Tool result: NAME] plus recursively rendered bounded result content; preserve error/timing metadata. Resolve names by ToolUseID, with display_data only as a safe fallback. |
| server tool use |
Same policy as tool use, labeled server-side. |
| web-search result set |
Recursively render individual results. |
| web-search result |
Bounded title, sanitized URL, page age, and visible text. |
image (MediaType + Data) |
Placeholder with MIME type, dimensions, byte count, and digest; never emit base64. |
| unknown type |
Preserve safe visible text/thinking when present, otherwise a short unsupported-type marker. |
Never normalize these opaque/provider fields into content, metadata, snippets, or
invocation arguments:
Signature;
- OpenAI encrypted reasoning/continuation data;
EncryptedContent or EncryptedIndex;
- image/base64/data-URI payloads;
- raw citations/caller payloads;
- arbitrary
Content.Display embedded in llm_data;
- arbitrary recursive
display_data.
Shelley-only row types
system with no recognized event metadata is the generated system prompt and
tool schema. Skip it entirely and increment omitted_system_prompt_count.
warning: index allowlisted user_data.text; preserve suppression flags.
gitinfo: index allowlisted user_data.text; preserve worktree, branch,
commit, and subject.
modelchange: index allowlisted user_data.text; preserve model/reasoning
transitions and display names.
error: render LLM text and preserve error_type, retryable, refusal
category, and refusal explanation.
slug: no searchable content.
- a
user row with user_data.cwd_change=true: retain the user-visible text
and allowlisted from/to values.
- rows identified by
user_data.distilled or user_data.distill_status: retain
status markers and real summary content, while suppressing
matched carried duplicates as described above. Never dereference or retain
user_data.distillation_file; use only the stored distillation_content.
Size limits
Apply source limits before JSON unmarshalling and deterministic UTF-8-safe
output limits before returning normalized data:
- raw
llm_data, user_data, display_data, or usage field: 8 MiB before
deserialization; larger values become omission descriptors without entering
the recursive JSON renderer;
- recursive content/tool-result rendering depth: 32;
- searchable content per message: 1 MiB;
- invocation arguments: 128 KiB;
- message
extra: 256 KiB;
- conversation metadata: 256 KiB;
- individual URL: 8 KiB.
For truncation, retain a bounded head and tail plus omitted-byte count and a
content digest. Never retain the original oversized value elsewhere in
normalized JSON. Sanitize URLs to http/https, remove userinfo, query, and
fragment, and cap the result; preserve only a digest of the removed form when
needed. Apply this to web results, llm_api_url, usage_data.url, and every
other_usage_data URL. CASS's existing secret redactor remains the final
persistence boundary across titles, content, metadata, extras, and tool
arguments.
Tokens and models
Shelley's direct usage JSON shape is
llm.Usage:
input_tokens
cache_creation_input_tokens
cache_read_input_tokens
output_tokens
cost_usd
model
url
start_time
end_time
For direct assistant usage:
- model precedence:
messages.model_name, then usage_data.model;
- map cache creation/read fields exactly;
- infer provider only from a recognized model family or sanitized API host;
- set the data source to API when any provider-reported usage exists;
- store a bounded
cass.tool_call_count in message extra and read it in the
token extractor; CASS currently does not persist NormalizedInvocation;
- when
forked_from_message_id is non-null, return an explicit suppressed-usage
state, not an empty usage object that falls back to content-based estimation.
other_usage_data is an array of purposed indirect calls, potentially using
multiple models. Preserve it in bounded structured message metadata, but do not
collapse heterogeneous calls into one fake CASS usage record. The first connector
change may report direct usage only. Exact indirect analytics require a separate
CASS token-ledger change because the current schema permits one token_usage row
per normalized message.
Malformed usage JSON must not discard otherwise valid conversation content.
Add a Suppressed/NoEstimate token-data state (or equivalent explicit flag)
to FAD/CASS so copied fork rows are excluded from both API totals and estimated
totals.
Required CASS integration
CASS's batch registry automatically consumes FAD factories, but watch,
persistence policy, and presentation surfaces remain explicit
(connector registry).
Dependency and registry
- bump FAD revision/version/features in
Cargo.toml, Cargo.lock, and the exact
dependency contract in build.rs;
- add
src/connectors/shelley.rs re-export and pub mod shelley;
- add
ShelleyConnector to the explicit imports in src/indexer/mod.rs;
- add
ConnectorKind::Shelley with a stable compact serde key, plus every
from_slug, slug, and create_connector arm;
- add
shelley to stable capabilities ordering (factory discovery makes
inclusion automatic; the explicit edit preserves deterministic public order);
- add a Shelley message-merge policy that trusts stable
sequence_id values and
does not replay-fingerprint-deduplicate two distinct rows merely because
timestamp, role, author, and content match;
- when
CASS_SHELLEY_DB is non-empty, schedule one explicit Shelley scan even
if schema-aware detection returns false, so scan() can return the required
path-specific validation error. Do not mark an invalid database as detected.
Watch mode
Shelley commits may change only the WAL; watching the main file alone is
insufficient. For an admitted database path P:
- watch
parent(P) while retaining P as the scan target;
- route events for
P, P-wal, and P-shm to the Shelley connector regardless
of the database's basename;
- carry the original event kind and sidecar mtime through classification;
- normalize sidecar event paths back to
P before scanning without losing the
trigger timestamp;
- use CASS's existing debounce for event bursts;
- treat WAL create/modify/rename/remove and main DB create/modify/replace as
reindex signals, including events whose path no longer exists by
classification time;
- treat SHM lifecycle as a signal but ignore access-only metadata churn;
- bypass the unchanged-main-file watch-once optimization for Shelley;
- do not use only the main DB mtime as the incremental cursor;
- add Shelley handling to
explicit_watch_once_connector_hint() so a standalone
configured DB works without prior detection.
The connector's schema validation remains authoritative; path classification is
only a routing hint.
Existing-conversation metadata refresh
CASS currently appends messages to an existing conversation but primarily
updates tail state. That is insufficient for Shelley because slug, archive
state, tags, model, generation, and workspace can change after first indexing.
Add a metadata refresh for every successfully re-observed existing
conversation, in the same transaction as append/dedup:
- update title, workspace, source path, ended time, metadata JSON, and origin
host from the normalized conversation;
- do not change source ID, agent identity, or external ID;
- do not rewrite canonical message rows;
- regenerate/upsert that conversation's derived Tantivy/FTS documents so title,
workspace, and metadata-dependent search output do not remain stale;
- accept
metadata_only=true records only for an already-existing identity and
skip them on a miss.
A test must index a conversation before slug generation, update the Shelley row,
rescan, and observe the new title in canonical storage and search output without
creating a duplicate conversation.
Raw-mirror privacy policy
Do not pass a Shelley database or its WAL/SHM sidecars through CASS's ordinary
raw-mirror capture. The current raw mirror copies source bytes into an
unencrypted content-addressed store, while the Shelley database also contains
custom-model API keys, notification configuration, browser cache/session
material, abandoned request/response debug rows, system prompts, and all other
conversations.
Add a provider-aware raw-mirror policy:
provider=shelley + role in {sqlite_database, metadata_sidecar}
=> disabled_sensitive_container
Enforce this policy centrally and at every production capture entry point:
pre-parse discovery, explicit-root fallback, per-conversation capture, and
cass doctor --fix raw-mirror backfill. Record
metadata.cass.raw_mirror.status="disabled_sensitive_container" and teach
doctor/coverage checks to count it as an intentional exclusion rather than
missing data-loss protection.
A future sanitized Shelley export can be mirrored, but the live database must
not be.
Sources, diagnostics, and presentation
- optionally add
~/.config/shelley/shelley.db to macOS/Linux local presets,
clearly documented as a CASS deployment convention rather than Shelley's
default; keep ~/shelley.db lower priority;
- advertise
CASS_SHELLEY_DB in environment/capability documentation;
- auto-probed schema non-matches are ignored; an explicitly configured
unreadable, busy, corrupt, or incompatible database returns an actionable
error containing the path and reason, which CASS exposes through its existing
ingest diagnostics JSON/log fields;
- add Shelley to TUI autocomplete/color/icon mappings, known-agent contracts,
HTML display/CSS identity, optional Pages badge styling, and documentation;
- update the current 26-provider assertions/documentation to 27 in
tests/agent_detection_completeness.rs and
tests/spec_connector_enumeration_completeness.rs, plus capabilities
fixtures, robot/diag goldens, and README connector/environment tables.
Test plan
FAD
- conventional path, current-directory path,
CASS_SHELLEY_DB, explicit file
with an arbitrary filename, explicit directory, and duplicate/symlink roots;
- remote roots are rejected without importing or copying a local default
database;
- random SQLite and basename-only false positives are rejected;
- minimum supported schema, current migration 037 schema, additive future
columns/types, missing required columns, corrupt DB, and a migration race;
- a committed WAL-only message is visible and the reader does not mutate the
main database or sidecars; test with a quiescent-but-open Shelley-compatible
writer and cover present/missing SHM behavior;
- sequence ordering with identical timestamps and legitimate sequence gaps;
- duplicate
sequence_id values are reported as corruption;
- archived, parent/subagent, fork, generation, excluded-context, draft, queued,
and empty-conversation cases;
- user, assistant, error, warning, git info, model change, distillation, slug,
system prompt, tool use/result, thinking, redacted thinking, web search, image,
and unknown content cases;
- real Shelley-produced compaction where carried rows have fresh timestamps,
including a carried prior distillation summary;
- direct token/cache/model extraction and fork-copy API/estimated-usage
suppression;
- heterogeneous
other_usage_data preservation without fake aggregation;
- malformed JSON skips only the affected row or field and produces a diagnostic;
- pre-unmarshal source caps, recursive-depth limits, URL sanitization, and
base64/encrypted/signature/Content.Display exclusion;
- exact numeric persisted JSON with
Role 0/1 and Type 2 through 9;
- cross-row tool-use/result name resolution;
- parent ID and
user_initiated disagreement cases;
- discovery lists the DB and present sidecars.
CASS
- end-to-end
tests/connector_shelley.rs using a sanitized current-schema
SQLite fixture with migration records;
- connector enumeration/completeness and capability ordering;
- stable-sequence merge behavior: distinct Shelley rows with equal
timestamp/role/content are not replay-deduplicated;
- local preset and explicit-path classification; remote Shelley roots are
rejected;
- watch-state serialization and arbitrary-name DB/WAL/SHM event classification,
including removed/renamed sidecars;
- watch mode indexes a committed WAL-only append;
- no raw-mirror blob or manifest is created for Shelley;
- doctor reports raw-mirror exclusion as intentional;
- initial no-slug ingest followed by slug, archive, unarchive, tag, option, and
workspace updates that do not rely on updated_at changing;
CASS_SKIP_SUBAGENTS using non-null Shelley parent metadata in batch,
streaming, watch, and targeted-retry paths;
- search by
--agent shelley finds user, assistant, tool-result, thinking, and
distillation-summary content;
- model/token analytics use direct provider usage and do not count fork copies;
- TUI, HTML, Pages, capabilities, and diagnostics snapshots/goldens.
If a fixture is committed, add provenance, sanitization notes, and checksums to
tests/fixtures/connectors/MANIFEST.json.
Acceptance criteria
cass capabilities --json includes shelley.
- A valid explicitly configured Shelley database indexes with
agent_slug="shelley", including a database whose filename is not
shelley.db.
- The optional CASS preset
~/.config/shelley/shelley.db is detected
automatically when enabled; Shelley's actual CLI default remains
./shelley.db relative to its process working directory.
- A random SQLite database or filename-only false positive is not detected.
- Search finds user, assistant, tool-result, plaintext thinking, and
distillation-summary content in sequence order.
- Archived and subagent conversations are indexed separately with parent and
generation metadata.
- Drafts, queued messages, generated system prompts, participant emails,
hook URLs, image/base64 bytes, signatures, and encrypted continuation data do
not enter normalized searchable storage.
- Direct model/input/output/cache usage is available; forked copies do not
double-count it; heterogeneous indirect usage is preserved without being
misreported.
- A committed WAL-only turn is indexed without reader-induced mutation of the
database, WAL, or SHM files.
- No Shelley raw-mirror artifact is created.
- Later slug/archive/tag/workspace changes refresh the existing CASS row rather
than creating a duplicate.
- An explicitly configured invalid, corrupt, locked, or incompatible database
produces an actionable ingest error rather than silent zero-result success;
automatic schema non-matches remain ordinary non-detections.
Non-goals for the first change
- recursively discovering arbitrary Shelley databases;
- requiring or calling Shelley's HTTP API;
- changing Shelley's schema;
- writing, checkpointing, repairing, or migrating the source database;
- raw-mirroring the live Shelley database;
- live SSH/remote Shelley database sync before a consistent sanitized bundle or
export exists;
- indexing draft or queued message bodies;
- flattening parent and subagent histories into one transcript;
- exact multi-call
other_usage_data analytics in CASS's current one-row-per-
message token ledger;
- automatically purging an existing CASS archive when a Shelley conversation is
hard-deleted; a fresh archive will omit it, while cleanup of an existing CASS
archive remains explicit;
- native
cass resume support for Shelley.
Optional follow-up: versioned Shelley export
A later Shelley-owned connector-export-v1 command/API could provide a stable
instance ID, schema version, sanitized conversation projection, consistent
snapshot revision, per-conversation content hashes, and deletion tombstones.
That would enable safe remote sync and improve deletion-aware incremental
indexing without coupling CASS to Shelley's private schema. It is not required
for the local direct read-only SQLite connector above.
I would love to use CASS with Shelley (https://github.com/boldsoftware/shelley).
I could contribute the connector code, but I think you don't accept code contributions.
I am enclosing a design doc (AI-generated) - hopefully it can be easily turned into beads.
Thank you.
Summary
Shelley stores conversations and messages in one SQLite database rather than one
file per session. The database uses WAL mode and may be placed anywhere with
Shelley's global
-dbflag; the CLI default isshelley.dbrelative to theShelley process's working directory.
The clean implementation boundary is:
discovery, and direct token extraction to
franken_agent_detection(FAD);
capability, presentation, contract-test, and documentation integration.
No Shelley code or schema change is required for the first implementation.
Use the SQLite database directly; do not require a running Shelley server.
Suggested implementation slices
Treat this issue as three reviewable PRs rather than one monolithic patch:
discovery, projection, limits, fixtures, and direct usage extraction.
local source configuration, raw-mirror denial, capabilities, presentation,
metadata refresh, and end-to-end batch indexing.
handling, watch diagnostics, and operational regression tests.
The issue is complete after all three land. Live SSH/remote SQLite sync, exact
multi-call indirect-usage analytics, and a Shelley export API remain follow-ups.
Revisions checked
Checked on August 24, 2026.
maineaf44af282424dc8maine615da6833c97eb6CASS currently pins FAD
0.2.1with selected SQLite connector features inCargo.tomland repeats that exact contract in
build.rs.Why this needs a dedicated connector
Shelley is not a safe alias for an existing SQLite-backed connector:
server process's working directory
(flag definition);
(pool initialization);
sensitive configuration, so CASS must not raw-mirror the whole file.
Do not detect Shelley by a generic
*.dborshelley.dbfilename alone.Admission requires a Shelley schema signature.
Shelley database contract
Shelley's migrations are the authoritative schema history
(
db/schema).The current generated row shapes are in
db/generated/models.go.Minimum supported schema
Support databases migrated through
003-add-message-sequence.sql. Require:Require the migration records for
001-conversations.sql,002-messages.sql,and
003-add-message-sequence.sqlwhen identifying an automatically discoveredcandidate. An explicitly configured candidate may be admitted by the exact
required table/column signature so sanitized fixtures and repaired databases
remain usable.
Read these newer columns when present:
Treat unknown later migrations, columns, message types, and JSON fields as
forward-compatible additions. Missing required columns are an incompatible
schema error, not an empty successful scan.
Ordering and identity
messages.sequence_idis authoritative within a conversation. Order by(sequence_id, message_id), never by timestamp alone. SetNormalizedMessage.idx = sequence_id; do not reindex gaps away. Duplicatesequence IDs are source corruption.
A Shelley
conversation_idis stable within one database. It is not globallynamespaced across independent databases.
Use a database path namespace to avoid collisions when two local Shelley
databases contain the same conversation ID:
Preserve the original Shelley
conversation_idin metadata. Moving orcopying a database to another path intentionally creates a new namespace.
Use the canonical database path as
NormalizedConversation.source_path.Current CASS health, retry, trust, doctor, and source-opening paths expect a
real file. The namespaced
external_iddistinguishes conversations sharingone source file.
Proposed FAD implementation
Add
src/connectors/shelley.rswithShelleyConnector.Feature and registry changes
In FAD:
blake3andurldependencies and reuse FAD's existingoptional
base64dependency;shelley = ["connectors", "dep:frankensqlite", "frankensqlite/fts5", "dep:asupersync", "dep:blake3", "dep:base64", "dep:url"];shelleyinall-connectorsand addfeature = "shelley"to thecfg list that exposes the
sqlite_syncmodule (there is no separatesqlite_syncCargo feature);ShelleyConnectorinsrc/connectors/mod.rsandsrc/lib.rs;shelleytoKNOWN_CONNECTORS, canonical slug aliases, environmentoverrides, and local default probes; give
default_probe_paths_tilde()anempty Shelley arm until safe remote export/sync exists;
entry_from_detect()path through schema-awarevalidation, never generic path-existence-only
detect_roots();empty
default_probe_paths_tilde()arm from any non-empty-path assertion;shelleytoextract_tokens_for_agent().Use FAD's existing synchronous FrankenSQLite bridge, not
rusqlite(
sqlite_sync.rs).crush.rsis the simplest single-database template;opencode.rsis the bestreference for candidate expansion, schema variants, and preventing remote scans
from importing local defaults.
Candidate path rules
Use one candidate-expansion and schema-admission helper for
detect(),scan(), anddiscover_source_files(). Do not delegate Shelley detection toFAD's generic path-existence report:
ShelleyConnector::detect()must open andvalidate candidates read-only so detection and scanning cannot disagree.
Priority:
<root>/shelley.dband, when the rootrepresents a home directory,
<root>/.config/shelley/shelley.db;CASS_SHELLEY_DB(treated as an explicitcandidate);
~/.config/shelley/shelley.db—this is a deploymentconvention, not Shelley's CLI default;
./shelley.dband~/shelley.db, each still requiring schema validation.Rules:
$HOMEfor SQLite databases;non-matches because CASS offers every configured root to every connector;
first implementation with an actionable message: current CASS SSH sync does
not copy a live SQLite database and its WAL/SHM as one consistent, sanitized
bundle;
Safe SQLite access
For each admitted database:
SQLITE_OPEN_READ_ONLY;PRAGMA query_only=ON,PRAGMA trusted_schema=OFF,and a bounded busy timeout;
migrations,conversations, andmessagesare tables, not views;immutable=1against a live database because committed rowsmay exist only in the WAL;
one read transaction so the result is a coherent committed snapshot;
VACUUM,ANALYZE, repair, or anywrite-capable pragma;
A release-blocking test must prove that the exact SQLite stack used by FAD can
see a committed Shelley row present only in
shelley.db-wal, while hashes,sizes, mtimes, and inode identity of the database and sidecars remain unchanged.
If that cannot be guaranteed, an explicitly configured scan must fail with an
actionable connector error rather than silently indexing stale main-file
contents. Automatic probes should ignore schema non-matches; they must not hide
I/O or corruption failures for a path that already passed Shelley admission.
Source discovery
discover_source_files()should return:DiscoveredSourceRole::SqliteDatabase;-walsidecar as required reconstruction metadata;-shmsidecar as optional metadata.The connector consumes the main database through SQLite, not by separately
parsing sidecar bytes. Discovery exists for provenance, diagnostics, and watch
coverage. CASS must apply the Shelley-specific raw-mirror exclusion described
below before capturing these files.
Query strategy
For a full scan, read every non-draft conversation and its messages. For an
incremental scan with
since_ts:conversationstable on every DB/WAL event, becausearchive, tag, and option updates do not reliably bump
updated_at;messages, using a small overlap window (for example two seconds);
marked
metadata.shelley.metadata_only=true; CASS must refresh an existingrow from these records but never create a new empty conversation from one.
This catches metadata-only changes without reloading every message blob on every
turn. New/fresh conversations always emit a complete transcript.
Implement true callback-based scanning. Extend
sqlite_syncwith a boundedrow-callback helper and a safe read-transaction guard rather than loading the
entire database into one
Vec.Conversation projection
One Shelley
conversationsrow becomes one normalized CASS conversation.conversation_idexternal_id; raw ID retained in metadata.slugcwdworkspace.created_atstarted_at.ended_at; fall back toupdated_atwhen there are no messages.archiveduser_initiatedparent_conversation_idmodelcurrent_generationtagsis_draft,draftqueued_messagesmessages.excluded_from_contextextra; index otherwise-visible stored content unless another explicit suppression rule applies.agent_workingconversation_optionsthinking_level,disable_all_tools, anddisable_notifications. Never retain hook URLs.Recommended metadata shape:
{ "source": "shelley", "shelley": { "conversation_id": "cABC123", "database_namespace": "0123456789abcdef", "user_initiated": true, "parent_conversation_id": null, "archived": false, "current_generation": 2, "configured_model": "opus", "tags": [], "message_count": 42, "omitted_system_prompt_count": 1, "omitted_carried_message_count": 8 } }Do not copy draft bodies, queued message bodies, raw participant email
addresses, hook URLs, or whole conversation rows into metadata.
Generations, compaction, forks, and subagents
filter only to
current_generation.generationand the conversation'scurrent_generation; do not persist a derived active/superseded label thatwould become stale after later compaction.
user_data.compaction_carried="true"and a fresh row timestamp. For carriedrows only, match the nearest unmatched prior-generation row by canonical
source type, LLM role, normalized content/tool structure, and allowlisted
user data—not by row timestamp. Count the omission in conversation
metadata. If no earlier match exists, retain it and mark it
unmatched_carried=truerather than dropping data.user_data.distillation_contentwhenuser_data.distilled="true".forked_from_message_idoncopied messages. Their text remains searchable in the fork, but their copied
usage_datamust not be counted again.CASS's
CASS_SKIP_SUBAGENTSpredicate must be extended to recognize Shelleyparent metadata, not only Claude's
subagents/path layout.Message and content projection
Shelley's canonical persisted message shape is
llm.Message(
llm.Messageandllm.Content).Because the Go constants share one
iotablock, persisted content type numbersare:
Parse
llm_dataas the canonical source. Useuser_dataanddisplay_dataonly through explicit per-message-type allowlists. Maintain a transcript-wide
ToolUseID -> tool namemap so a later tool-result row can resolve the callrecorded in an earlier assistant row.
Role mapping
userrowuseragentrowassistanttoolerrorsystem, preserving retry/refusal fieldswarning,gitinfo,modelchangesystemeventsystemevent/summaryllm_data.Rolesystem, preserving the original typeslugDo not assume
messages.type='tool'for tool results. Shelley normally stores atool-result LLM message with user role, so inspect content blocks.
Content blocks
Render blocks in source order:
[Thinking]\nplus plaintext thinking.[Redacted thinking]; never emitDataorSignature.[Tool call: NAME]\nplus bounded canonical JSON arguments; also emitNormalizedInvocation.[Tool result: NAME]plus recursively rendered bounded result content; preserve error/timing metadata. Resolve names byToolUseID, withdisplay_dataonly as a safe fallback.MediaType+Data)Never normalize these opaque/provider fields into content, metadata, snippets, or
invocation arguments:
Signature;EncryptedContentorEncryptedIndex;Content.Displayembedded inllm_data;display_data.Shelley-only row types
systemwith no recognized event metadata is the generated system prompt andtool schema. Skip it entirely and increment
omitted_system_prompt_count.warning: index allowlisteduser_data.text; preserve suppression flags.gitinfo: index allowlisteduser_data.text; preserve worktree, branch,commit, and subject.
modelchange: index allowlisteduser_data.text; preserve model/reasoningtransitions and display names.
error: render LLM text and preserveerror_type,retryable, refusalcategory, and refusal explanation.
slug: no searchable content.userrow withuser_data.cwd_change=true: retain the user-visible textand allowlisted
from/tovalues.user_data.distilledoruser_data.distill_status: retainstatus markers and real summary content, while suppressing
matched carried duplicates as described above. Never dereference or retain
user_data.distillation_file; use only the storeddistillation_content.Size limits
Apply source limits before JSON unmarshalling and deterministic UTF-8-safe
output limits before returning normalized data:
llm_data,user_data,display_data, or usage field: 8 MiB beforedeserialization; larger values become omission descriptors without entering
the recursive JSON renderer;
extra: 256 KiB;For truncation, retain a bounded head and tail plus omitted-byte count and a
content digest. Never retain the original oversized value elsewhere in
normalized JSON. Sanitize URLs to
http/https, remove userinfo, query, andfragment, and cap the result; preserve only a digest of the removed form when
needed. Apply this to web results,
llm_api_url,usage_data.url, and everyother_usage_dataURL. CASS's existing secret redactor remains the finalpersistence boundary across titles, content, metadata, extras, and tool
arguments.
Tokens and models
Shelley's direct usage JSON shape is
llm.Usage:For direct assistant usage:
messages.model_name, thenusage_data.model;cass.tool_call_countin messageextraand read it in thetoken extractor; CASS currently does not persist
NormalizedInvocation;forked_from_message_idis non-null, return an explicit suppressed-usagestate, not an empty usage object that falls back to content-based estimation.
other_usage_datais an array of purposed indirect calls, potentially usingmultiple models. Preserve it in bounded structured message metadata, but do not
collapse heterogeneous calls into one fake CASS usage record. The first connector
change may report direct usage only. Exact indirect analytics require a separate
CASS token-ledger change because the current schema permits one
token_usagerowper normalized message.
Malformed usage JSON must not discard otherwise valid conversation content.
Add a
Suppressed/NoEstimatetoken-data state (or equivalent explicit flag)to FAD/CASS so copied fork rows are excluded from both API totals and estimated
totals.
Required CASS integration
CASS's batch registry automatically consumes FAD factories, but watch,
persistence policy, and presentation surfaces remain explicit
(connector registry).
Dependency and registry
Cargo.toml,Cargo.lock, and the exactdependency contract in
build.rs;src/connectors/shelley.rsre-export andpub mod shelley;ShelleyConnectorto the explicit imports insrc/indexer/mod.rs;ConnectorKind::Shelleywith a stable compact serde key, plus everyfrom_slug,slug, andcreate_connectorarm;shelleyto stable capabilities ordering (factory discovery makesinclusion automatic; the explicit edit preserves deterministic public order);
sequence_idvalues anddoes not replay-fingerprint-deduplicate two distinct rows merely because
timestamp, role, author, and content match;
CASS_SHELLEY_DBis non-empty, schedule one explicit Shelley scan evenif schema-aware detection returns false, so
scan()can return the requiredpath-specific validation error. Do not mark an invalid database as detected.
Watch mode
Shelley commits may change only the WAL; watching the main file alone is
insufficient. For an admitted database path
P:parent(P)while retainingPas the scan target;P,P-wal, andP-shmto the Shelley connector regardlessof the database's basename;
Pbefore scanning without losing thetrigger timestamp;
reindex signals, including events whose path no longer exists by
classification time;
explicit_watch_once_connector_hint()so a standaloneconfigured DB works without prior detection.
The connector's schema validation remains authoritative; path classification is
only a routing hint.
Existing-conversation metadata refresh
CASS currently appends messages to an existing conversation but primarily
updates tail state. That is insufficient for Shelley because
slug, archivestate, tags, model, generation, and workspace can change after first indexing.
Add a metadata refresh for every successfully re-observed existing
conversation, in the same transaction as append/dedup:
host from the normalized conversation;
workspace, and metadata-dependent search output do not remain stale;
metadata_only=truerecords only for an already-existing identity andskip them on a miss.
A test must index a conversation before slug generation, update the Shelley row,
rescan, and observe the new title in canonical storage and search output without
creating a duplicate conversation.
Raw-mirror privacy policy
Do not pass a Shelley database or its WAL/SHM sidecars through CASS's ordinary
raw-mirror capture. The current raw mirror copies source bytes into an
unencrypted content-addressed store, while the Shelley database also contains
custom-model API keys, notification configuration, browser cache/session
material, abandoned request/response debug rows, system prompts, and all other
conversations.
Add a provider-aware raw-mirror policy:
Enforce this policy centrally and at every production capture entry point:
pre-parse discovery, explicit-root fallback, per-conversation capture, and
cass doctor --fixraw-mirror backfill. Recordmetadata.cass.raw_mirror.status="disabled_sensitive_container"and teachdoctor/coverage checks to count it as an intentional exclusion rather than
missing data-loss protection.
A future sanitized Shelley export can be mirrored, but the live database must
not be.
Sources, diagnostics, and presentation
~/.config/shelley/shelley.dbto macOS/Linux local presets,clearly documented as a CASS deployment convention rather than Shelley's
default; keep
~/shelley.dblower priority;CASS_SHELLEY_DBin environment/capability documentation;unreadable, busy, corrupt, or incompatible database returns an actionable
error containing the path and reason, which CASS exposes through its existing
ingest diagnostics JSON/log fields;
HTML display/CSS identity, optional Pages badge styling, and documentation;
tests/agent_detection_completeness.rsandtests/spec_connector_enumeration_completeness.rs, plus capabilitiesfixtures, robot/diag goldens, and README connector/environment tables.
Test plan
FAD
CASS_SHELLEY_DB, explicit filewith an arbitrary filename, explicit directory, and duplicate/symlink roots;
database;
columns/types, missing required columns, corrupt DB, and a migration race;
main database or sidecars; test with a quiescent-but-open Shelley-compatible
writer and cover present/missing SHM behavior;
sequence_idvalues are reported as corruption;and empty-conversation cases;
system prompt, tool use/result, thinking, redacted thinking, web search, image,
and unknown content cases;
including a carried prior distillation summary;
suppression;
other_usage_datapreservation without fake aggregation;base64/encrypted/signature/
Content.Displayexclusion;Role0/1 andType2 through 9;user_initiateddisagreement cases;CASS
tests/connector_shelley.rsusing a sanitized current-schemaSQLite fixture with migration records;
timestamp/role/content are not replay-deduplicated;
rejected;
including removed/renamed sidecars;
workspace updates that do not rely on
updated_atchanging;CASS_SKIP_SUBAGENTSusing non-null Shelley parent metadata in batch,streaming, watch, and targeted-retry paths;
--agent shelleyfinds user, assistant, tool-result, thinking, anddistillation-summary content;
If a fixture is committed, add provenance, sanitization notes, and checksums to
tests/fixtures/connectors/MANIFEST.json.Acceptance criteria
cass capabilities --jsonincludesshelley.agent_slug="shelley", including a database whose filename is notshelley.db.~/.config/shelley/shelley.dbis detectedautomatically when enabled; Shelley's actual CLI default remains
./shelley.dbrelative to its process working directory.distillation-summary content in sequence order.
generation metadata.
hook URLs, image/base64 bytes, signatures, and encrypted continuation data do
not enter normalized searchable storage.
double-count it; heterogeneous indirect usage is preserved without being
misreported.
database, WAL, or SHM files.
than creating a duplicate.
produces an actionable ingest error rather than silent zero-result success;
automatic schema non-matches remain ordinary non-detections.
Non-goals for the first change
export exists;
other_usage_dataanalytics in CASS's current one-row-per-message token ledger;
hard-deleted; a fresh archive will omit it, while cleanup of an existing CASS
archive remains explicit;
cass resumesupport for Shelley.Optional follow-up: versioned Shelley export
A later Shelley-owned
connector-export-v1command/API could provide a stableinstance ID, schema version, sanitized conversation projection, consistent
snapshot revision, per-conversation content hashes, and deletion tombstones.
That would enable safe remote sync and improve deletion-aware incremental
indexing without coupling CASS to Shelley's private schema. It is not required
for the local direct read-only SQLite connector above.