PUBLISH ingress P2: NL-description -> IntegrationSpec (opt-in LLM, aix-backed) - #37
Merged
Conversation
… aix-backed) The second PUBLISH ingress arm (landscape doc 9.2): refine a natural-language description into a *draft* IntegrationSpec via oa's prompt-as-function machinery, routed through aix (provider-agnostic) -- NOT oa's OpenAI default -- to keep the multi-target promise honest. Kept strictly opt-in (DECISIONS D18): - coact/nl_ingress.py: integration_spec_from_description(description, *, llm=None, model=None, name=None, infer_tool_schemas=True). Backend injectable (callable / model-name / None -> aix.chat); oa/aix imported LAZILY so importing coact stays provider-free (D10, regression-tested in a subprocess). - IntegrationSpec grows tool_specs: list[ToolSpec] (name/description/input_schema/ handler) alongside the unchanged tools refs (non-breaking; landscape 9.1). A ToolSpec with a handler is bound (runnable); without, it is a proposed draft. Added runnable_refs() + render(); is_empty() now counts tool_specs. - publish_mcpb builds the server config from runnable_refs(); a pure draft (no runnable ref) RAISES with guidance instead of writing a dead bundle; proposed tools are listed in the manifest with a warning. - per-tool input-schema inference via oa.infer_schema_from_verbal_description, made backend-injectable upstream (oa PR; first customer = coact). - authoring prompt SSOT in coact (DFLT_AUTHORING_PROMPTS), injectable via prompt_template= (pyrompt = the iteration home, not a hard dep). - CLI verb 'coact describe'; exports ToolSpec + integration_spec_from_description; extra coact[nl] (oa, aix); skill + README updated. - 17 offline tests (fake backend, no provider call). Full suite: 380 passed.
An adversarial multi-agent review of the P2 diff surfaced 9 confirmed issues,
all fixed (offline regression tests added for each):
- [high] integration_spec_from dropped a spec's tool_specs/resources/prompts when
the spec was passed inside a list -> bound handlers silently lost. Now carried
through; the empty-guard counts them too.
- [med] non-string LLM field values (number/list/object where a string was asked)
crashed _spec_from_extracted with a raw AttributeError/TypeError. Now coerced.
- [med] _parse_json_object seized the FIRST balanced {...}; brace-bearing prose
before the real JSON defeated it. Now tries whole-text, fenced body, then each
top-level balanced span.
- [med] NL ingress hard-required aix even when a callable backend was injected.
check_requirements is now path-aware (aix only on the default/model-name path).
- [low] a bare-string resources/prompts was exploded per character. Now wrapped.
- [low] .mcpb draft-guard said "0 proposed tool(s)" for a resources/prompts-only
spec. Now a distinct, accurate message.
- [low] manifest could advertise a curated name the server would not serve. Keep
the bound function's own name/docstring (runtime truth); a curated description
only fills an empty one.
- [low] str-llm (model-name) branch was untested; README NL snippet implied it was
offline. Added tests + an honest README caveat.
Full suite: 393 passed, 4 skipped; ruff clean.
…essions
CI does not install the optional `coact[nl]` extra, so the NL ingress path
(which genuinely needs `oa` as its prompt orchestrator) cannot run there. Match
the project's established posture (litellm/langgraph/crewai backend tests skip in
bare CI, run in dev):
- test_nl_ingress.py guards the whole module with pytest.importorskip("oa")
(aix is never imported — backends are injected/monkeypatched).
- the oa-FREE mechanical regressions moved to test_publish.py so they still run
in CI: the high-severity list-branch handler-preservation fix (#5), manifest
fidelity (#4), the resources-only guard message (#7), proposed-tool warning,
and the D10 import-isolation check.
- the `coact describe` CLI test guards with importorskip("oa").
Dev: 393 passed, 4 skipped. Bare-CI simulation (oa/aix blocked): the affected
files run with 0 failures (oa-dependent tests skip, oa-free ones pass).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds the second PUBLISH ingress arm (landscape doc §9.2): refine a
natural-language description into a draft
IntegrationSpec— proposed toolswith inferred input schemas — routing generation through aix (provider-agnostic)
via oa's prompt-as-function machinery. Kept strictly opt-in (DECISIONS D18).
coact describe "a connector that looks up the weather for a city"Design
coact/nl_ingress.py—integration_spec_from_description(description, *, llm=None, model=None, name=None, infer_tool_schemas=True). Backend is injectable (callable / model-name /None→aix.chat);oa/aiximported lazily soimport coactstays provider-free (D10, regression-tested in a subprocess).IntegrationSpecgrowstool_specs: list[ToolSpec](name / description / input_schema / handler) alongside the unchangedtoolsrefs — non-breaking; aligns with landscape §9.1. AToolSpecwith a handler is bound (runnable); without, a proposed draft. Newrunnable_refs()+render().publish_mcpbbuilds the server config fromrunnable_refs(); a pure draft (no runnable ref) raises with guidance instead of writing a dead bundle; proposed tools are listed in the manifest with a warning.oa.infer_schema_from_verbal_description, made backend-injectable upstream (oa#14, merged; first customer = coact).DFLT_AUTHORING_PROMPTS), injectable viaprompt_template=(pyrompt = the iteration home, not a hard dep).coact describe; exportsToolSpec+integration_spec_from_description; extracoact[nl](oa,aix); skill + README updated.Why aix, not oa's default
Per the route-through-aix policy, NL generation defaults to aix (multi-provider), not oa's OpenAI default — keeping the multi-target promise honest.
oa.infer_schema_from_verbal_descriptionwas hardwired to oa'schat; it was made backend-injectable upstream so this path can use any provider.Review
An adversarial multi-agent review of the diff (3 finder dimensions → per-finding verification) surfaced 9 confirmed findings, all fixed with regression tests (see the
harden:commit): silent handler loss through the list branch (high), non-string/brace-prose robustness, path-aware optional-dep check, clearer guards, and manifest/runtime name fidelity.Tests
393 passed, 4 skipped; ruff clean. All NL-ingress tests are offline (injected fake backend — no provider call).