Skip to content

feat: opt-in observability — Sentry-protocol errors + Langfuse eval traces - #175

Merged
fazpu merged 2 commits into
mainfrom
feat/observability-wiring
Jul 29, 2026
Merged

feat: opt-in observability — Sentry-protocol errors + Langfuse eval traces#175
fazpu merged 2 commits into
mainfrom
feat/observability-wiring

Conversation

@fazpu

@fazpu fazpu commented Jul 29, 2026

Copy link
Copy Markdown
Member

Owner-directed observability wiring, strictly opt-in via env (absent config = byte-identical behavior, nothing imported — proven by an import-purity test):

  • Error tracking: REMEMBERSTACK_SENTRY_DSN (Sentry-protocol — works with Sentry/GlitchTip/Bugsink) initializes in the selfhost api/worker/setup entrypoints; the worker exception path captures with stage/lane/processing_id tags. Metadata only — never prompt/completion text (allowlisted before_send).
  • Langfuse eval tracing: standard LANGFUSE_* vars; the LoCoMo answer/judge stages emit one trace per question with per-call spans (model, usage, cost, latency, verdicts). Observer-only: the protocol fingerprint (prompts/schemas/catalog/manifests) has zero diff, asserted against baseline.
  • Observability never alters production control flow (review round): all sink/tracer failures are best-effort-isolated — only the authoritative local sink may raise; regression tests include a raising Sentry sink through run_one and a raising tracer through answer/judge with identical outputs.
  • Optional dependency extra observability; compose/.env.example/docs wired. Instance provisioning (Langfuse self-host vs cloud; GlitchTip vs Betterstack) is an infra decision tracked separately.

Review: Grok REQUEST-CHANGES (2 MAJOR on control-flow isolation, 3 MINOR, 2 NIT) → all applied → verified. Final review by Claude. Fast gate local (130 passed); full matrix in CI.

@fazpu
fazpu enabled auto-merge (squash) July 29, 2026 14:30
@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/rememberstack/adapters
  __init__.py
  sentry.py 119-125, 157, 166, 169, 172
  src/rememberstack/adapters/selfhost
  __init__.py
  telemetry.py 58, 63-68
  src/rememberstack/profiles
  selfhost.py 115, 159-166, 171-172, 314-317, 327-336, 540-542, 572-579, 587, 595, 599, 610
Project Total  

This report was generated by python-coverage-comment-action

@fazpu
fazpu merged commit 5a20584 into main Jul 29, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant