[OpenSpec] persistent-llm-process

[github-actions]

⚠️ OpenSpec-managed issue — this content is automatically synced
from the openspec/ directory. Manual edits will be overwritten on next sync.

Artifacts

• ✅ Metadata
• ✅ Proposal
• ✅ Specs
• ✅ Design
• ✅ Tasks
• 🔲 Plan
• 🔲 Review

Specs

• 📄 llm-runtime-latency

Tasks

• Land tool-scope-security-default first
• Decide the token strategy (design D0.1) before writing any pool code
• Switch the anthropic provider from -p one-shot to --input-format stream-json + --output-format stream-json
• Add a process pool in exapp/llm-runner keyed by (agent, user)
• Dispatch a turn as a newline-delimited {"type":"user","message":{…}} frame to a live process
• Bound the pool and reap idle processes
• Prove no conversation state crosses turns, using a distinctive token
• Invalidate an agent's pooled processes when its grants change, immediately — not at idle reap
• Reap a process on repeated api_retry with error_status: 401
• Fall back to spawning on any pool miss, unhealthy process, or dispatch failure
• Measure request-start → first MCP packet, before and after, same instance, same prompt
• Expose the pool hit rate
• Report the result even if it disappoints
• Stop the CLI's GET /apps/hermiq/api/mcp/run → 401 round trip

Design

See design.md for technical design details.

---

Synced from openspec/changes/persistent-llm-process by OpenSpec workflow
App: hermiq