Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ All notable changes to this project will be documented here. The project follows
- A stateless-by-default OpenAI Responses API adapter with text, refusal, function-call, and tool-result normalization.
- Content-free sync/async attempt observation and inspectable provider health snapshots.
- Cross-request transient-failure cooldown that deprioritizes unhealthy routes without removing last-resort fallback.
- A canonical support-ticket triage reference consumer and deterministic public-API contract fixture.

### Changed

Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,8 @@ The built-in HTTP providers own API translation. `UnifiedLLM` owns policy and ha

For new OpenAI integrations, use `OpenAIResponsesProvider`. It targets `/responses`, keeps server-side storage disabled by default, and participates in the same retry, fallback, request-size, and response-size boundaries. See [examples/responses_api.py](examples/responses_api.py).

For a production-shaped consumer, see [examples/support_triage.py](examples/support_triage.py) and its [consumer contract](docs/CONSUMER_CONTRACT.md). The example routes support tickets through a strict function schema and validates every model-supplied field before application use.

## Security, privacy, reliability, and cost

- Treat endpoint URLs, API keys, models, and custom provider adapters as trusted operator configuration.
Expand Down
5 changes: 3 additions & 2 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This roadmap separates four gates: merge, release, publication, and flagship ado

Portfolio role: **reusable library or sdk**. Keep this as a small, independently versioned package. Samsarix Unified should consume it only through a public API adapter; private monorepo imports and copied implementations are out of scope.

Current disposition: Already integrated; roadmap changes should use an ordinary review branch.
Current disposition: The library is productized on its own default branch, but no external canonical consumer has adopted this reference contract yet. Roadmap changes should use an ordinary review branch.

## Stabilize the productized default

Expand All @@ -17,7 +17,8 @@ Current disposition: Already integrated; roadmap changes should use an ordinary
- Completed in the next hardening increment: account for the exact outbound body, cap inbound response bytes and normalized structures, and validate custom adapters.
- Completed in the following API increment: add a stateless-by-default OpenAI Responses adapter without weakening the shared bounds or fallback contract.
- Completed in the health increment: expose content-free attempt observation and deprioritize repeatedly failing routes during a bounded cooldown.
- Next: prove one canonical consumer and a capped live endpoint before publication.
- Completed as repository evidence: a canonical support-triage reference consumer with a public-API contract fixture, explicit limits, ownership, compatibility, and rollback guidance.
- Next: adopt the reference contract in one external canonical consumer and prove a capped live endpoint before publication.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- Review priority: prove one consumer, green wheel CI, one capped live endpoint, and trusted publication.

## Release candidate
Expand Down
64 changes: 64 additions & 0 deletions docs/CONSUMER_CONTRACT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Support-triage consumer contract

## Purpose and status

`examples/support_triage.py` is the repository's canonical reference consumer. It demonstrates a production-shaped use case: classify an untrusted support ticket into a queue and urgency through one strict function call, then validate the model-supplied arguments before application use.

This is deterministic contract evidence, not a claim of live deployment or Samsarix Unified adoption. A live endpoint remains an external release gate because it requires operator-owned credentials and can incur provider charges.

## Supported public boundary

The consumer imports only names exported by `unified_llm`:

- `OpenAIResponsesProvider`
- `Route`
- `UnifiedLLM`

It does not import implementation modules, private symbols, legacy services, persistence, or account logic. The fixture targets the current `0.1.0` public contract. Until a `0.2.0` release establishes a wider compatibility promise, consumers should pin the exact prerelease version or artifact digest they verify.

## Contract

| Boundary | Consumer requirement | Evidence |
| --- | --- | --- |
| Authentication | The API key is supplied only to the provider adapter and becomes an authorization header, never JSON payload data. | `test_support_triage_public_api_contract` |
| Privacy | Responses requests set `store=false`; the router has no persistence or content logging; the observer receives only sanitized attempt metadata. Model-written summary text is validated and discarded, then replaced with a local queue/urgency template. | Exact outbound payload assertion, adversarial summary fixture, and core architecture tests |
| Request safety | Ticket and strict tool schema are included in the exact request-byte calculation. The reference client caps requests at 64,000 bytes and output at 1,000 tokens. | Router request-bound tests and reference client configuration |
| Response safety | Raw responses are capped at 256,000 bytes, normalized content at 8,000 characters, and tool calls at four. | Reference client configuration and provider-bound tests |
| Decision integrity | Exactly one named function call is required. JSON must contain only `queue`, `urgency`, and `summary`; enum values and summary length are checked after parsing. The returned summary is generated locally from validated enums. | Happy-path, adversarial-summary, and malformed-decision tests |
| Failure behavior | Configuration, validation, provider, and fallback failures remain typed SDK exceptions. Invalid business output becomes a consumer-owned `ValueError`; no empty success value is returned. | Consumer negative tests and SDK error contract |
| Metadata | The decision records normalized provider, served model, and reported token total for audit/cost attribution. | Happy-path fixture |
| Lifecycle | The reference uses `async with UnifiedLLM(...)`, closing SDK-owned HTTP resources deterministically. | `main()` and lifecycle tests |

## Verification

Run from a clean checkout:

```bash
python -m pytest tests/test_consumer_contract.py -q
python -m mypy unified_llm tests examples
python -m ruff check .
python -m ruff format --check .
```

The HTTP fixture uses `httpx.MockTransport`; it requires no credentials, sends no network traffic, and asserts the provider boundary directly.

## Live conformance gate

Before describing this consumer as live-compatible, a maintainer must run one capped request against the intended endpoint using a revocable, least-privilege key and record:

1. exact package artifact digest and Python version;
2. provider, endpoint class, and exact model identifier without recording credentials or ticket content;
3. request/response limits and whether provider-side storage was disabled;
4. returned tool-call shape, usage metadata, and typed failure behavior;
5. timestamp, maintainer, cost, and credential revocation or rotation result.

Live evidence must not contain prompts, responses, authorization headers, customer data, or secrets.

## Ownership, compatibility, and rollback

- Owner: Samsarix LLC
- Support: `support@samsarix.com`
- Commercial contact: `contact@samsarix.com`
- Compatibility: exact verified prerelease artifact until a published compatibility window exists
- Rollback: stop invoking `triage_ticket`, remove the reference-consumer integration, and pin the previously verified SDK artifact; the core router and provider adapters remain independent of this example
- Adoption signal: count successful consumer-owned contract runs and, only after live adoption, valid routed tickets versus rejected malformed decisions
1 change: 1 addition & 0 deletions examples/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Runnable public-API examples and reference consumers."""
126 changes: 126 additions & 0 deletions examples/support_triage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
"""Reference consumer: classify a support ticket through a bounded function call."""

from __future__ import annotations

import asyncio
import json
import os
from dataclasses import dataclass
from typing import Any

from unified_llm import ConfigurationError, OpenAIResponsesProvider, Route, UnifiedLLM

TRIAGE_TOOL: dict[str, Any] = {
"type": "function",
"function": {
"name": "route_support_ticket",
"description": "Return the queue, urgency, and safe one-line summary for a support ticket.",
"parameters": {
"type": "object",
"properties": {
"queue": {"type": "string", "enum": ["billing", "bug", "security", "general"]},
"urgency": {"type": "string", "enum": ["low", "normal", "high", "critical"]},
"summary": {"type": "string", "minLength": 1, "maxLength": 240},
},
"required": ["queue", "urgency", "summary"],
"additionalProperties": False,
},
"strict": True,
},
}

_QUEUES = frozenset({"billing", "bug", "security", "general"})
_URGENCIES = frozenset({"low", "normal", "high", "critical"})


@dataclass(frozen=True, slots=True)
class TriageDecision:
queue: str
urgency: str
summary: str
provider: str
model: str
total_tokens: int


def build_client(api_key: str) -> UnifiedLLM:
"""Build the reference consumer's privacy-first production client."""

provider = OpenAIResponsesProvider(name="openai", api_key=api_key, store=False)
return UnifiedLLM(
[Route(provider, "gpt-5-mini")],
request_timeout=20,
max_attempts_per_route=2,
max_total_attempts=2,
max_request_bytes=64_000,
max_response_bytes=256_000,
max_response_chars=8_000,
max_tool_calls=4,
max_output_tokens=1_000,
)


async def triage_ticket(client: UnifiedLLM, ticket: str) -> TriageDecision:
"""Classify one ticket and reject malformed model tool arguments."""

if not isinstance(ticket, str) or not ticket.strip():
raise ValueError("ticket must be a non-empty string")
response = await client.chat_with_tools(
[
{
"role": "developer",
"content": (
"Classify support tickets. Never include credentials, tokens, payment details, "
"or other secrets in the summary. Always call route_support_ticket exactly once."
),
Comment thread
coderabbitai[bot] marked this conversation as resolved.
},
{"role": "user", "content": ticket},
],
tools=[TRIAGE_TOOL],
max_tokens=300,
temperature=0,
)
if len(response.tool_calls) != 1:
raise ValueError("provider must return exactly one support-routing tool call")
function = response.tool_calls[0].get("function")
if not isinstance(function, dict) or function.get("name") != "route_support_ticket":
raise ValueError("provider returned the wrong support-routing tool")
arguments = function.get("arguments")
if not isinstance(arguments, str):
raise ValueError("provider returned invalid support-routing arguments")
try:
decision = json.loads(arguments)
except (TypeError, ValueError) as exc:
raise ValueError("provider returned malformed support-routing JSON") from exc
if not isinstance(decision, dict) or set(decision) != {"queue", "urgency", "summary"}:
raise ValueError("provider returned an invalid support-routing object")
queue = decision["queue"]
urgency = decision["urgency"]
summary = decision["summary"]
if not isinstance(queue, str) or queue not in _QUEUES:
raise ValueError("provider returned an unsupported support-routing value")
if not isinstance(urgency, str) or urgency not in _URGENCIES:
raise ValueError("provider returned an unsupported support-routing value")
if not isinstance(summary, str) or not summary.strip() or len(summary) > 240:
raise ValueError("provider returned an invalid support-routing summary")
return TriageDecision(
queue=queue,
urgency=urgency,
summary=f"{urgency.capitalize()}-urgency ticket routed to {queue} support.",
provider=response.provider,
model=response.model,
total_tokens=response.total_tokens,
)


async def main() -> None:
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
raise ConfigurationError("OPENAI_API_KEY is required.")
async with build_client(api_key) as client:
decision = await triage_ticket(client, input("Support ticket: "))
print(decision)


if __name__ == "__main__":
asyncio.run(main())
149 changes: 149 additions & 0 deletions tests/test_consumer_contract.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
from __future__ import annotations

import json
from pathlib import Path

import httpx
import pytest

from examples.support_triage import TRIAGE_TOOL, TriageDecision, main, triage_ticket
from unified_llm import ConfigurationError, OpenAIResponsesProvider, Route, UnifiedLLM, UnifiedLLMResponse

ROOT = Path(__file__).parents[1]


async def test_support_triage_public_api_contract() -> None:
seen: dict[str, object] = {}

def handler(request: httpx.Request) -> httpx.Response:
seen["method"] = request.method
seen["url"] = str(request.url)
seen["authorization"] = request.headers.get("Authorization")
seen["payload"] = json.loads(request.content)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
return httpx.Response(
200,
json={
"model": "gpt-5-mini-2026-06-01",
"status": "completed",
"output": [
{
"type": "function_call",
"call_id": "call-triage",
"name": "route_support_ticket",
"arguments": json.dumps(
{"queue": "security", "urgency": "critical", "summary": "Possible account takeover"}
),
}
],
"usage": {"input_tokens": 30, "output_tokens": 12, "total_tokens": 42},
},
)

http_client = httpx.AsyncClient(transport=httpx.MockTransport(handler))
provider = OpenAIResponsesProvider(name="openai", api_key="test-secret", client=http_client, store=False)
client = UnifiedLLM(
[Route(provider, "gpt-5-mini")],
max_request_bytes=64_000,
max_response_bytes=256_000,
max_response_chars=8_000,
max_tool_calls=4,
)
result = await triage_ticket(client, "Unknown login changed my recovery email and locked me out.")

assert result == TriageDecision(
queue="security",
urgency="critical",
summary="Critical-urgency ticket routed to security support.",
provider="openai",
model="gpt-5-mini-2026-06-01",
total_tokens=42,
)
assert seen["method"] == "POST"
assert seen["url"] == "https://api.openai.com/v1/responses"
assert seen["authorization"] == "Bearer test-secret"
payload = seen["payload"]
assert isinstance(payload, dict)
assert payload["store"] is False
assert payload["max_output_tokens"] == 300
assert payload["temperature"] == 0
assert payload["tools"] == [
{
"type": "function",
"name": TRIAGE_TOOL["function"]["name"],
"description": TRIAGE_TOOL["function"]["description"],
"parameters": TRIAGE_TOOL["function"]["parameters"],
"strict": True,
}
]
assert "test-secret" not in json.dumps(payload)
await client.aclose()
await http_client.aclose()


class DecisionProvider:
name = "decision"

def __init__(self, *, name: str = "route_support_ticket", arguments: str, count: int = 1) -> None:
self.function_name = name
self.arguments = arguments
self.count = count

async def complete(self, **_kwargs: object) -> UnifiedLLMResponse:
call = {
"id": "call-1",
"type": "function",
"function": {"name": self.function_name, "arguments": self.arguments},
}
return UnifiedLLMResponse(
content="",
model="contract-model",
provider=self.name,
tool_calls=tuple(dict(call) for _ in range(self.count)),
)


@pytest.mark.parametrize(
"provider",
[
DecisionProvider(arguments="not-json"),
DecisionProvider(arguments='{"queue":"unknown","urgency":"normal","summary":"x"}'),
DecisionProvider(arguments='{"queue":[],"urgency":"normal","summary":"x"}'),
DecisionProvider(arguments='{"queue":"bug","urgency":{},"summary":"x"}'),
DecisionProvider(arguments='{"queue":"bug","urgency":"normal","summary":"x","extra":true}'),
DecisionProvider(name="wrong_tool", arguments='{"queue":"bug","urgency":"normal","summary":"x"}'),
DecisionProvider(arguments='{"queue":"bug","urgency":"normal","summary":"x"}', count=2),
],
)
async def test_support_triage_rejects_malformed_model_decisions(provider: DecisionProvider) -> None:
client = UnifiedLLM([Route(provider, "contract-model")])
with pytest.raises(ValueError, match="provider"):
await triage_ticket(client, "The app crashed")


async def test_support_triage_rejects_empty_ticket() -> None:
client = UnifiedLLM([Route(DecisionProvider(arguments="{}"), "contract-model")])
with pytest.raises(ValueError, match="ticket"):
await triage_ticket(client, " ")


async def test_support_triage_never_returns_model_supplied_summary() -> None:
secret = "sk-secret-that-must-not-cross-the-boundary"
provider = DecisionProvider(arguments=json.dumps({"queue": "billing", "urgency": "high", "summary": secret}))
client = UnifiedLLM([Route(provider, "contract-model")])

decision = await triage_ticket(client, "My payment failed")

assert decision.summary == "High-urgency ticket routed to billing support."
assert secret not in decision.summary


async def test_support_triage_main_requires_api_key(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("OPENAI_API_KEY", raising=False)
with pytest.raises(ConfigurationError, match="OPENAI_API_KEY"):
await main()


def test_reference_consumer_uses_only_public_package_imports() -> None:
source = (ROOT / "examples" / "support_triage.py").read_text(encoding="utf-8")
assert "unified_llm.unified_llm" not in source
assert "from unified_llm import" in source
Loading