search/fetch: silent & misleading error/semantics signaling (DSL forms, multi-app filters, result cap, exit codes)

[fostiropoulos]

Summary

Hands-on evaluation of the ctxd CLI (v0.1.11) surfaced a cluster of error-handling and semantic-transparency bugs in ctxd search / ctxd fetch. The common thread: failures and non-obvious query semantics resolve silently — the caller gets a plausible-looking result with error: null and rc=0, and cannot tell a wrong/incomplete answer from a correct one without running controlled experiments.

📄 Full report (commands tried, controlled probes, PM analysis): https://gist.github.com/fostiropoulos/abc0d2d4661a8404c28a4ed61ffe6630

Environment: ctxd 0.1.11, authenticated, integrations: github / google_calendar / google_drive / slack.

---

Bugs

1. Parse-error suggestion recommends a fix that changes recall ~5×

Unquoted multi-word text correctly errors (rc=1) with an otherwise excellent message, but it suggests quotes or parentheses as interchangeable fixes:

text:incident postmortem  ->  ERROR
Suggested fix: Wrap multi-word text value in quotes: text:"incident postmortem"
               or parentheses: text:(incident postmortem)

They are not equivalent (same corpus, google_drive):

Form	Results
text:(incident response)	79
text:"incident response"	15
text:incident AND text:response	10

Expected: the error should not present two fixes that differ 5× in recall as equivalent. Name the semantics of each.

2. text:(a b) is implicit OR, undocumented

Probe: text:zzqphishing→0, text:incident→14, text:(zzqphishing incident)→14 (= 0 OR 14). Parens is OR, but reads as grouping/AND. Not documented.

3. Quoted value is semantic, not a literal phrase, undocumented

text:incident→14 but text:"incident response"→15. A literal phrase should be a subset of incident (≤14); returning more means "..." triggers semantic/vector search. This is the opposite of the universal search convention that quotes = exact phrase, and nothing signals it.

4. Multi-app filters: three syntaxes, two silently wrong, zero errors

Combining google_drive (18 hits) + slack (1 hit) for onboarding:

Syntax	Results
application:google_drive OR application:slack	25	only working union
application:(google_drive OR slack)	0	silently returns nothing
application:google_drive application:slack	18	2nd filter silently dropped

Expected: grouped form works, or repeated/grouped app filters error instead of silently returning 0/wrong subset.

5. Silent hard cap of 100 results

text:the and text:document both return exactly 100, with no total, truncated flag, or pagination. A capped set is indistinguishable from a complete one.

6. Unknown / empty application: silently returns []

application:nonexistent_app text:foo → {"results": [], "error": null} (rc=0). Same for application: (empty). A typo is indistinguishable from a genuine no-match.

7. Inconsistent exit codes

Failure	Exit code
DSL parse error	1	✅
fetch <nonexistent-uid>	0	❌ prints "could not be resolved" but exits 0
Unknown application filter	0	⚠️ (see #6)

A failed fetch exiting 0 breaks set -e and if ctxd fetch … guards.

---

Suggested fixes (priority order)

1. Echo interpretation in every search response — interpreted_as, applied_filters, total/returned/truncated, unknown_filters. This single change makes dsl parsing errors #2, improve readme (content and branding) and contributing.md #4, add pre-commit hooks, black, lint, flake8 #5, Fix CLI text quoting and main-only PyPI publishing #6 self-evident without docs or experiments.
2. Fix the parse-error suggestion (only publish to pypi when merging to main from a PR #1) so it doesn't equate quotes and parens.
3. Fix or reject grouped/repeated application: filters (improve readme (content and branding) and contributing.md #4).
4. Fail loud on unknown application (Fix CLI text quoting and main-only PyPI publishing #6) and non-zero exit on failed fetch (add repo policies, ie approvers, push to main, etc #7).
5. Document parens=OR, quotes=semantic in search --help (dsl parsing errors #2, update pypi package publishing policy to accept releases from this repo #3).

Related: prior #2 "dsl parsing errors".