feat(#126): introduce Langfuse observability for the memory pipeline

[alexosugo]

Summary

Closes #126
Closes #103

Introduces end-to-end LLM observability for the memory distillation pipeline using Langfuse.

Note on #103 scope: #103 asked for a self-hosted Langfuse deployment via Docker Compose. This PR ships Langfuse Cloud instead — self-hosting is deferred until the agent platform has matured enough to justify the operational overhead of running our own instance. Trace capture, span structure, and the LangChain integration itself fully satisfy the eval-foundation goal of #103; only the hosting model differs from what was originally scoped.

• src/observability/index.ts — shared module: lazy singleton Langfuse client (getLangfuse()) plus a single startTrace() primitive that returns { trace, handler } — a trace and the LangChain callback handler rooted on it. Replaces the earlier separate makeLangfuseHandler/createTrace/flushLangfuse helpers. All calls are no-ops when LANGFUSE_ENABLED=false.
• src/types/pipeline.ts — FilterOptions and DistillOptions gain an optional langfuseHandler?: BaseCallbackHandler field.
• src/scripts/filter.ts — threads the handler to chain.invoke({ callbacks }) inside llmTriage.
• src/scripts/distiller.ts — threads the handler to chain.invoke({ callbacks }) inside llmDistill.
• src/scripts/run-pipeline.ts — processSinglePR creates one Langfuse trace per PR via startTrace() (with a scrape span and a distill-outcome score), sets trace.update({ output }), and flushes via getLangfuse().flushAsync(). All traces from a single runPipeline call share a session UUID.
• .github/workflows/unit_tests.yml — sets LANGFUSE_ENABLED=false so CI tests never contact Langfuse.
• .github/workflows/run-pipeline.yml — adds LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_BASE_URL from repository secrets/vars.
• docs/observability.md — setup guide, trace structure, naming conventions, and a step-by-step recipe for instrumenting new agents/workflows, plus a "Future Work" section (prompt versioning, dashboards, alerting, eval datasets, agent instrumentation).
• .env.example — documents the four Langfuse variables (LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL, LANGFUSE_ENABLED).

Trace shape (per PR)

memory-pipeline-pr   (input: { prNum, repo, url }; tags: [memory-pipeline, <repo>]; session: <run UUID>)
├── span: scrape          input: { prNum, repo }  output: { fileCount }
├── generation: filter    ← LangChain callback auto-captures model + tokens + latency
└── generation: distill   ← LangChain callback auto-captures model + tokens + latency
    score: distill-outcome           1 = written, 0 = flag-for-human

The trace id is generated by Langfuse per run (not derived from the PR number), so reprocessing the same PR produces a distinct trace rather than mutating an earlier run's session. PR identity lives in input/tags/metadata so it stays filterable.

Design notes

• Lazy singleton: the Langfuse client is created on first call (not at module load time) so dotenv.config() runs before the constructor reads process.env.
• LANGFUSE_ENABLED=false: startTrace()'s handler is still constructed but no-ops; callers use handler ? [handler] : undefined to skip callbacks entirely.
• Env var naming: LANGFUSE_HOST was renamed to LANGFUSE_BASE_URL across source, docs, workflow, and .env.example for clarity (matches the Langfuse client constructor option name).
• Injection point preserved: opts.triageFn / opts.distillFn in tests still bypass the real LLM and the handler, keeping test isolation intact.
• Scoring today: distill-outcome is the only score currently emitted, set directly in code (trace.score(...) in run-pipeline.ts) — not an LLM-judge or human annotation. It only fires for PRs that reach the distill stage; filtered-out PRs get an unscored trace with decision: 'skip' in the output.

Test plan

• npm run build — tsc clean
• npm run lint — eslint clean
• LANGFUSE_ENABLED=false npm run test:coverage — 416 tests pass, all coverage thresholds met (exit 0)
• Ran npm run run-pipeline -- --pr <cht-core-pr> locally with real Langfuse keys and confirmed a trace appears in the Langfuse Cloud dashboard with scrape span, LLM generations, and the distill-outcome score.
• Add LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY as repository secrets, and LANGFUSE_BASE_URL as a repository variable before the next scheduled pipeline run.