Skip to content

runtime: qualify omitted native Claude profiles with fresh exact cases #162

Description

@sumitake

Summary

Native Claude qualification for the PR #3041 implementation admitted only context.documents.intent. The exact architecture.conceptual, context.documents.extract, and context.documents.reason cases were consumed once and omitted from the compiled graph without replay.

Evidence and disposition

  • context.documents.reason: setup failed before provider start because the shared supervisor rejected the required empty non-executable value in --tools "". The supervisor now accepts empty later argv values while retaining a nonempty executable requirement, but this consumed case remains unqualified.
  • architecture.conceptual: one provider process completed with a structured terminal presentation that the pre-amendment adapter rejected as execution_contract_violation. The case was consumed and remains unqualified.
  • context.documents.extract: one provider process returned a closed successful stream. Offline diagnosis showed the current CLI uses a reserved StructuredOutput pseudo-tool, reports inert installed-plugin/built-in-agent catalogs, and may list the initialized primary plus an internal helper under terminal modelUsage. The adapter now accepts only that exact closed schema cycle and additive identity form while rejecting every external or mismatched tool. The consumed case remains unqualified.
  • context.documents.intent: a distinct exact source-head call succeeded with one provider process, two complete document reads, two reported turns, matched primary-model identity, confirmed cleanup, and no external tool/MCP activity. A reviewed matched rubric passed against the successful Grok incumbent; Gemini produced only advisory/unknown evidence.

No failed or unavailable request was replayed. Private evidence retains create-exclusive consumed markers, result digests, and bounded raw-stream digests; source contents are not attached here.

Follow-up acceptance

Define a fresh, distinct per-profile corpus and request identity before any future qualification. Admit a profile only after its own one-shot source-head canary and matched offline quality rubric pass on the exact adapter/runtime bytes. Do not reactivate these profiles as shadow routes, infer qualification from the shared transport, or reuse the consumed requests.

This issue tracks capability follow-up only; it does not block the intent-only release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions