Goal
Let an agentic ir-backed connector run its LLM (query reformulation / LLM-selection / synopsis) on the client's Claude subscription instead of a metered API key — so a connector costs nothing beyond the corpus search (which is already offline).
Background
- ir's core search is offline/free; the LLM is opt-in and injectable at every seam (
make_llm_formulator(rewriter=), make_llm_selector(chooser=), with_synopsis(synthesize=), eval_gen generators). ir needs zero changes. See ir misc/docs/ir_10.
- The adapter is a host-side concern (depends on
fastmcp.Context / claude_agent_sdk, not on ir), so it belongs here, not in ir.
To build
sampling_llm(ctx) adapter — a prompt -> str callable that does await ctx.sample(prompt) (FastMCP Context.sample), bridged sync↔async (the ir seams are sync). The connector tool declares ctx: fastmcp.Context (FastMCP auto-injects it; no py2mcp change needed).- Wire-up helper — given a
ConnectorSpec flagged agentic, inject sampling_llm(ctx) into the ir search call's seams (e.g. expose a raglab agent / make_llm_* with the adapter) per request.
- Graceful fallback — if the client doesn't advertise
sampling (ctx.sample raises), fall back to offline search (or a configured key). Never hard-fail the connector.
- Claude Code path — for local/dev use, an analogous
prompt -> str over claude_agent_sdk / coact.resolve_llm (exists today; runs on host auth).
Blocker / first step
Verify whether Claude.ai remote connectors honour sampling/createMessage. Historically sampling has been a Claude Desktop / local-client feature. If remote connectors don't support it, ship the offline-by-default connector and document subscription-routing as Desktop/Claude-Code-only for now.
Related: i2mint/ir ir_10, and the ir issue tracking client-support verification.
https://claude.ai/code/session_0154TnwYASjN1u1Hw97JQ2vd
Goal
Let an agentic ir-backed connector run its LLM (query reformulation / LLM-selection / synopsis) on the client's Claude subscription instead of a metered API key — so a connector costs nothing beyond the corpus search (which is already offline).
Background
make_llm_formulator(rewriter=),make_llm_selector(chooser=),with_synopsis(synthesize=),eval_gengenerators). ir needs zero changes. Seeirmisc/docs/ir_10.fastmcp.Context/claude_agent_sdk, not on ir), so it belongs here, not in ir.To build
sampling_llm(ctx)adapter — aprompt -> strcallable that doesawait ctx.sample(prompt)(FastMCPContext.sample), bridged sync↔async (the ir seams are sync). The connector tool declaresctx: fastmcp.Context(FastMCP auto-injects it; no py2mcp change needed).ConnectorSpecflagged agentic, injectsampling_llm(ctx)into the ir search call's seams (e.g. expose araglabagent /make_llm_*with the adapter) per request.sampling(ctx.sampleraises), fall back to offline search (or a configured key). Never hard-fail the connector.prompt -> stroverclaude_agent_sdk/coact.resolve_llm(exists today; runs on host auth).Blocker / first step
Verify whether Claude.ai remote connectors honour
sampling/createMessage. Historically sampling has been a Claude Desktop / local-client feature. If remote connectors don't support it, ship the offline-by-default connector and document subscription-routing as Desktop/Claude-Code-only for now.Related: i2mint/ir
ir_10, and the ir issue tracking client-support verification.https://claude.ai/code/session_0154TnwYASjN1u1Hw97JQ2vd