From 799683c9c72cc01f09d8518040edd073452dae71 Mon Sep 17 00:00:00 2001 From: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> Date: Thu, 15 Oct 2026 10:20:09 +0200 Subject: [PATCH 1/6] feat(server): add explicit LiteLLM Responses dialect and recorded evidence Keep this as a separate follow-up PR to the typed OpenAI Responses implementation. Its review base is efa77f8525; the preceding buffered, streaming and composition commits are not part of this change. Recognize six reviewed response metadata fields and the SSE model annotation through explicitly bound typed declarations. Reuse request policy, event roles, text selection and lifecycle hooks. Do not change the strict OpenAI endpoint, Chat dialect, shared kernel, transport or application factory. Capture buffered and streaming evidence for one OpenAI model route through the gateway advertising LiteLLM 1.105.1. Preserve item-ID relationships while redacting credentials and encrypted state. Add policy, export and passthrough regressions plus the generated inspection contract. Bridged backends and nested LiteLLM SSE errors remain outside the supported boundary. Validation: 298 affected unit/integration tests and 16 offline recorded/helper tests passed. Pre-commit, export drift and cassette privacy checks passed. This commit is kept locally for separate-PR bookkeeping and review; no remote PR or push is included. Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> --- .../_generated/litellm-responses.guard.yaml | 3476 +++++++++++++++++ .../openai/dialects/litellm/README.md | 4 + .../openai/dialects/litellm/responses.md | 119 + .../openai/dialects/litellm/responses.py | 221 ++ tests/recorded/README.md | 9 + ...es_preserve_policy_and_wire[buffered].yaml | 127 + ...s_preserve_policy_and_wire[streaming].yaml | 300 ++ .../experimental/openai/responses_support.py | 79 + .../openai/test_litellm_responses.py | 231 ++ tests/recorded/test_responses_sanitization.py | 68 + .../test_litellm_responses_dialect.py | 162 + 11 files changed, 4796 insertions(+) create mode 100644 nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml create mode 100644 nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md create mode 100644 nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.py create mode 100644 tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[buffered].yaml create mode 100644 tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[streaming].yaml create mode 100644 tests/recorded/server/experimental/openai/responses_support.py create mode 100644 tests/recorded/server/experimental/openai/test_litellm_responses.py create mode 100644 tests/recorded/test_responses_sanitization.py create mode 100644 tests/server/experimental/test_litellm_responses_dialect.py diff --git a/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml b/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml new file mode 100644 index 0000000000..7a6fa00ca3 --- /dev/null +++ b/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml @@ -0,0 +1,3476 @@ +version: 1.0.0-alpha.1 +operationId: createResponse +profile: single_text.v1 +request: + additionalProperties: false + description: Bind the declared input subject and strict response-mode selector. + properties: + background: + const: false + default: false + type: boolean + x-nemo-guardrails: + classification: constrained + reason: core_capability.background_execution + context_management: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.conversation_state + conversation: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.conversation_state + include: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.additional_content + input: + minLength: 1 + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: user + instructions: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.additional_input + max_output_tokens: + x-nemo-guardrails: + classification: opaque + max_tool_calls: + x-nemo-guardrails: + classification: opaque + metadata: + x-nemo-guardrails: + classification: opaque + model: + x-nemo-guardrails: + classification: opaque + moderation: + x-nemo-guardrails: + classification: opaque + parallel_tool_calls: + x-nemo-guardrails: + classification: opaque + previous_response_id: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.conversation_state + prompt: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.additional_input + prompt_cache_key: + x-nemo-guardrails: + classification: opaque + prompt_cache_options: + x-nemo-guardrails: + classification: opaque + prompt_cache_retention: + x-nemo-guardrails: + classification: opaque + reasoning: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - additionalProperties: false + description: Allow reviewed reasoning controls, but no separate reasoning + text input. + properties: + context: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.reasoning_content + effort: + x-nemo-guardrails: + classification: opaque + generate_summary: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.reasoning_content + mode: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.reasoning_content + summary: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.reasoning_content + title: ResponsesReasoningConfigProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/Reasoning' + unknown_fields: configurable + reject_case_aliases: true + - type: 'null' + safety_identifier: + x-nemo-guardrails: + classification: opaque + service_tier: + x-nemo-guardrails: + classification: opaque + store: + x-nemo-guardrails: + classification: opaque + stream: + default: false + type: boolean + x-nemo-guardrails: + classification: constrained + stream_options: + x-nemo-guardrails: + classification: opaque + temperature: + x-nemo-guardrails: + classification: opaque + text: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - additionalProperties: false + description: Retain verbosity while restricting any supplied format to plain + text. + properties: + format: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - additionalProperties: false + description: Disallow structured-output instructions outside the guarded + input. + properties: + type: + const: text + type: string + x-nemo-guardrails: + classification: constrained + reason: projection_policy.plain_text_output + required: + - type + title: ResponsesTextFormatProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseFormatText' + unknown_fields: configurable + reject_case_aliases: true + - type: 'null' + verbosity: + x-nemo-guardrails: + classification: opaque + title: ResponsesTextConfigProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseTextParam' + unknown_fields: configurable + reject_case_aliases: true + - type: 'null' + tool_choice: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.tool_content + tools: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: core_capability.tool_content + top_logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - maximum: 0 + minimum: 0 + type: integer + - type: 'null' + top_p: + x-nemo-guardrails: + classification: opaque + truncation: + x-nemo-guardrails: + classification: opaque + user: + x-nemo-guardrails: + classification: opaque + required: + - input + title: openai.responses.request.text.v1 + type: object + x-nemo-guardrails: + source: '#/components/schemas/CreateResponse' + unknown_fields: forbid + reject_case_aliases: true + model: ResponsesGuardedRequest + stream_selector_field: stream +response: + additionalProperties: false + description: Derive coverage and extraction from the complete dialect declaration. + properties: + access_programs: + default: null + x-nemo-guardrails: + classification: opaque + background: + x-nemo-guardrails: + classification: opaque + billing: + default: null + x-nemo-guardrails: + classification: opaque + completed_at: + x-nemo-guardrails: + classification: opaque + conversation: + x-nemo-guardrails: + classification: opaque + created_at: + x-nemo-guardrails: + classification: opaque + error: + x-nemo-guardrails: + classification: opaque + frequency_penalty: + default: null + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + incomplete_details: + x-nemo-guardrails: + classification: opaque + instructions: + x-nemo-guardrails: + classification: opaque + max_output_tokens: + x-nemo-guardrails: + classification: opaque + max_tool_calls: + x-nemo-guardrails: + classification: opaque + metadata: + x-nemo-guardrails: + classification: opaque + model: + x-nemo-guardrails: + classification: opaque + moderation: + x-nemo-guardrails: + classification: opaque + object: + x-nemo-guardrails: + classification: opaque + output: + items: + oneOf: + - additionalProperties: false + description: Preserve opaque reasoning metadata, not separate plaintext + reasoning. + properties: + content: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + encrypted_content: + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + summary: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + type: + const: reasoning + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + title: ResponsesReasoningItemProjection + type: object + x-nemo-guardrails: + variant: + cardinality: any + match: + type: reasoning + source: '#/components/schemas/ReasoningItem' + unknown_fields: configurable + reject_case_aliases: true + - additionalProperties: false + description: Select exactly one assistant message without assuming its output + index. + properties: + content: + items: + additionalProperties: false + description: Guard output text and reject uninspected citation and + token text. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + minLength: 1 + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesOutputTextProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + maxItems: 1 + minItems: 1 + type: array + x-nemo-guardrails: + classification: guarded + id: + x-nemo-guardrails: + classification: opaque + phase: + x-nemo-guardrails: + classification: opaque + role: + const: assistant + type: string + x-nemo-guardrails: + classification: constrained + status: + x-nemo-guardrails: + classification: opaque + type: + const: message + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - role + - content + title: ResponsesAssistantMessageProjection + type: object + x-nemo-guardrails: + variant: + cardinality: exactly_one + match: + role: assistant + type: message + source: '#/components/schemas/OutputMessage' + unknown_fields: configurable + reject_case_aliases: true + type: array + x-nemo-guardrails: + classification: guarded + output_text: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: provider_integrity.duplicate_text + parallel_tool_calls: + x-nemo-guardrails: + classification: opaque + presence_penalty: + default: null + x-nemo-guardrails: + classification: opaque + previous_response_id: + x-nemo-guardrails: + classification: opaque + prompt: + x-nemo-guardrails: + classification: opaque + prompt_cache_diagnostics: + x-nemo-guardrails: + classification: opaque + prompt_cache_key: + x-nemo-guardrails: + classification: opaque + prompt_cache_options: + x-nemo-guardrails: + classification: opaque + prompt_cache_retention: + x-nemo-guardrails: + classification: opaque + reasoning: + x-nemo-guardrails: + classification: opaque + safety_identifier: + x-nemo-guardrails: + classification: opaque + service_tier: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + store: + default: null + x-nemo-guardrails: + classification: opaque + temperature: + x-nemo-guardrails: + classification: opaque + text: + x-nemo-guardrails: + classification: opaque + tool_choice: + x-nemo-guardrails: + classification: opaque + tool_usage: + default: null + x-nemo-guardrails: + classification: opaque + tools: + x-nemo-guardrails: + classification: opaque + top_logprobs: + x-nemo-guardrails: + classification: opaque + top_p: + x-nemo-guardrails: + classification: opaque + truncation: + x-nemo-guardrails: + classification: opaque + usage: + x-nemo-guardrails: + classification: opaque + user: + x-nemo-guardrails: + classification: opaque + required: + - output + title: openai.litellm.responses.response.text.v1 + type: object + x-nemo-guardrails: + source: '#/components/schemas/Response' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMGuardedResponses +integration: + name: litellm_responses + endpoint: + route_path: /v1/responses + operation_label: LiteLLM Responses + unsupported_request_code: unsupported_responses_shape + unsupported_response_code: unsupported_responses_response_shape +stream: + title: openai.litellm.responses.stream.text.v1 + oneOf: + - additionalProperties: false + description: Keep the delta as the only newly generated text subject. + properties: + content_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - maximum: 0 + minimum: 0 + type: integer + - type: 'null' + delta: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + item_id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + model: + default: null + x-nemo-guardrails: + classification: opaque + obfuscation: + x-nemo-guardrails: + classification: opaque + output_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.output_text.delta + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - delta + title: LiteLLMTextDeltaEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseTextDeltaEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMTextDeltaEvent + event: + classification: guarded_delta + variants: + - source_schema: ResponseTextDeltaEvent + shape: response.output_text.delta + match: + type: response.output_text.delta + missing_text: reject + - additionalProperties: false + description: Keep completed text subject to snapshot consistency checks. + properties: + content_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - maximum: 0 + minimum: 0 + type: integer + - type: 'null' + item_id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + model: + default: null + x-nemo-guardrails: + classification: opaque + output_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: response.output_text.done + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: LiteLLMTextDoneEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseTextDoneEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMTextDoneEvent + event: + classification: snapshot + variants: + - source_schema: ResponseTextDoneEvent + shape: response.output_text.done + match: + type: response.output_text.done + missing_text: reject + - additionalProperties: false + description: Retain the initial text part and its inherited content restrictions. + properties: + content_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - maximum: 0 + minimum: 0 + type: integer + - type: 'null' + item_id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + model: + default: null + x-nemo-guardrails: + classification: opaque + output_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + part: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations and logprobs + stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + classification: guarded + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.content_part.added + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - part + title: LiteLLMPartAddedEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseContentPartAddedEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMPartAddedEvent + event: + classification: snapshot + variants: + - source_schema: ResponseContentPartAddedEvent + shape: response.content_part.added + match: + type: response.content_part.added + missing_text: reject + - additionalProperties: false + description: Retain the completed text part as a checked snapshot. + properties: + content_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - maximum: 0 + minimum: 0 + type: integer + - type: 'null' + item_id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + model: + default: null + x-nemo-guardrails: + classification: opaque + output_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + part: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations and logprobs + stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + classification: guarded + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.content_part.done + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - part + title: LiteLLMPartDoneEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseContentPartDoneEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMPartDoneEvent + event: + classification: snapshot + variants: + - source_schema: ResponseContentPartDoneEvent + shape: response.content_part.done + match: + type: response.content_part.done + missing_text: reject + - additionalProperties: false + description: Preserve message identity and single-part cardinality. + properties: + item: + additionalProperties: false + description: Allow an initially empty message, but never a second guarded + text part. + properties: + content: + items: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations and + logprobs stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + maxItems: 1 + type: array + x-nemo-guardrails: + classification: guarded + id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + phase: + x-nemo-guardrails: + classification: opaque + role: + const: assistant + type: string + x-nemo-guardrails: + classification: constrained + status: + x-nemo-guardrails: + classification: opaque + type: + const: message + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - role + - content + title: ResponsesSnapshotMessageProjection + type: object + x-nemo-guardrails: + classification: guarded + variant: + cardinality: at_most_one + match: + role: assistant + type: message + source: '#/components/schemas/OutputMessage' + unknown_fields: configurable + reject_case_aliases: true + model: + default: null + x-nemo-guardrails: + classification: opaque + output_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.output_item.added + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - item + title: LiteLLMMessageAddedEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseOutputItemAddedEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMMessageAddedEvent + event: + classification: snapshot + variants: + - source_schema: ResponseOutputItemAddedEvent + shape: response.output_item.added:message + match: + type: response.output_item.added + item.type: message + missing_text: empty + - additionalProperties: false + description: Preserve message completion checks and the original item identity. + properties: + item: + additionalProperties: false + description: Allow an initially empty message, but never a second guarded + text part. + properties: + content: + items: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations and + logprobs stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + maxItems: 1 + type: array + x-nemo-guardrails: + classification: guarded + id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + phase: + x-nemo-guardrails: + classification: opaque + role: + const: assistant + type: string + x-nemo-guardrails: + classification: constrained + status: + x-nemo-guardrails: + classification: opaque + type: + const: message + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - role + - content + title: ResponsesSnapshotMessageProjection + type: object + x-nemo-guardrails: + classification: guarded + variant: + cardinality: at_most_one + match: + role: assistant + type: message + source: '#/components/schemas/OutputMessage' + unknown_fields: configurable + reject_case_aliases: true + model: + default: null + x-nemo-guardrails: + classification: opaque + output_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.output_item.done + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - item + title: LiteLLMMessageDoneEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseOutputItemDoneEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMMessageDoneEvent + event: + classification: snapshot + variants: + - source_schema: ResponseOutputItemDoneEvent + shape: response.output_item.done:message + match: + type: response.output_item.done + item.type: message + missing_text: empty + - additionalProperties: false + description: Check the creation snapshot with the reviewed response metadata. + properties: + model: + default: null + x-nemo-guardrails: + classification: opaque + response: + additionalProperties: false + description: Apply the same reviewed metadata to initial and terminal response + snapshots. + properties: + access_programs: + default: null + x-nemo-guardrails: + classification: opaque + background: + x-nemo-guardrails: + classification: opaque + billing: + default: null + x-nemo-guardrails: + classification: opaque + completed_at: + x-nemo-guardrails: + classification: opaque + conversation: + x-nemo-guardrails: + classification: opaque + created_at: + x-nemo-guardrails: + classification: opaque + error: + x-nemo-guardrails: + classification: opaque + frequency_penalty: + default: null + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + incomplete_details: + x-nemo-guardrails: + classification: opaque + instructions: + x-nemo-guardrails: + classification: opaque + max_output_tokens: + x-nemo-guardrails: + classification: opaque + max_tool_calls: + x-nemo-guardrails: + classification: opaque + metadata: + x-nemo-guardrails: + classification: opaque + model: + x-nemo-guardrails: + classification: opaque + moderation: + x-nemo-guardrails: + classification: opaque + object: + x-nemo-guardrails: + classification: opaque + output: + items: + oneOf: + - additionalProperties: false + description: Preserve opaque reasoning metadata, not separate plaintext + reasoning. + properties: + content: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + encrypted_content: + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + summary: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + type: + const: reasoning + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + title: ResponsesReasoningItemProjection + type: object + x-nemo-guardrails: + variant: + cardinality: any + match: + type: reasoning + source: '#/components/schemas/ReasoningItem' + unknown_fields: configurable + reject_case_aliases: true + - additionalProperties: false + description: Allow an initially empty message, but never a second + guarded text part. + properties: + content: + items: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations + and logprobs stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + maxItems: 1 + type: array + x-nemo-guardrails: + classification: guarded + id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + phase: + x-nemo-guardrails: + classification: opaque + role: + const: assistant + type: string + x-nemo-guardrails: + classification: constrained + status: + x-nemo-guardrails: + classification: opaque + type: + const: message + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - role + - content + title: ResponsesSnapshotMessageProjection + type: object + x-nemo-guardrails: + variant: + cardinality: at_most_one + match: + role: assistant + type: message + source: '#/components/schemas/OutputMessage' + unknown_fields: configurable + reject_case_aliases: true + type: array + x-nemo-guardrails: + classification: guarded + output_text: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: provider_integrity.duplicate_text + parallel_tool_calls: + x-nemo-guardrails: + classification: opaque + presence_penalty: + default: null + x-nemo-guardrails: + classification: opaque + previous_response_id: + x-nemo-guardrails: + classification: opaque + prompt: + x-nemo-guardrails: + classification: opaque + prompt_cache_diagnostics: + x-nemo-guardrails: + classification: opaque + prompt_cache_key: + x-nemo-guardrails: + classification: opaque + prompt_cache_options: + x-nemo-guardrails: + classification: opaque + prompt_cache_retention: + x-nemo-guardrails: + classification: opaque + reasoning: + x-nemo-guardrails: + classification: opaque + safety_identifier: + x-nemo-guardrails: + classification: opaque + service_tier: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + store: + default: null + x-nemo-guardrails: + classification: opaque + temperature: + x-nemo-guardrails: + classification: opaque + text: + x-nemo-guardrails: + classification: opaque + tool_choice: + x-nemo-guardrails: + classification: opaque + tool_usage: + default: null + x-nemo-guardrails: + classification: opaque + tools: + x-nemo-guardrails: + classification: opaque + top_logprobs: + x-nemo-guardrails: + classification: opaque + top_p: + x-nemo-guardrails: + classification: opaque + truncation: + x-nemo-guardrails: + classification: opaque + usage: + x-nemo-guardrails: + classification: opaque + user: + x-nemo-guardrails: + classification: opaque + required: + - output + title: LiteLLMLifecycleResponse + type: object + x-nemo-guardrails: + classification: guarded + source: '#/components/schemas/Response' + unknown_fields: forbid + reject_case_aliases: true + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.created + type: string + x-nemo-guardrails: + classification: constrained + required: + - response + - type + title: LiteLLMCreatedEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseCreatedEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMCreatedEvent + event: + classification: snapshot + variants: + - source_schema: ResponseCreatedEvent + shape: response.created + match: + type: response.created + missing_text: empty + - additionalProperties: false + description: Check the progress snapshot without introducing a separate text target. + properties: + model: + default: null + x-nemo-guardrails: + classification: opaque + response: + additionalProperties: false + description: Apply the same reviewed metadata to initial and terminal response + snapshots. + properties: + access_programs: + default: null + x-nemo-guardrails: + classification: opaque + background: + x-nemo-guardrails: + classification: opaque + billing: + default: null + x-nemo-guardrails: + classification: opaque + completed_at: + x-nemo-guardrails: + classification: opaque + conversation: + x-nemo-guardrails: + classification: opaque + created_at: + x-nemo-guardrails: + classification: opaque + error: + x-nemo-guardrails: + classification: opaque + frequency_penalty: + default: null + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + incomplete_details: + x-nemo-guardrails: + classification: opaque + instructions: + x-nemo-guardrails: + classification: opaque + max_output_tokens: + x-nemo-guardrails: + classification: opaque + max_tool_calls: + x-nemo-guardrails: + classification: opaque + metadata: + x-nemo-guardrails: + classification: opaque + model: + x-nemo-guardrails: + classification: opaque + moderation: + x-nemo-guardrails: + classification: opaque + object: + x-nemo-guardrails: + classification: opaque + output: + items: + oneOf: + - additionalProperties: false + description: Preserve opaque reasoning metadata, not separate plaintext + reasoning. + properties: + content: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + encrypted_content: + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + summary: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + type: + const: reasoning + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + title: ResponsesReasoningItemProjection + type: object + x-nemo-guardrails: + variant: + cardinality: any + match: + type: reasoning + source: '#/components/schemas/ReasoningItem' + unknown_fields: configurable + reject_case_aliases: true + - additionalProperties: false + description: Allow an initially empty message, but never a second + guarded text part. + properties: + content: + items: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations + and logprobs stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + maxItems: 1 + type: array + x-nemo-guardrails: + classification: guarded + id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + phase: + x-nemo-guardrails: + classification: opaque + role: + const: assistant + type: string + x-nemo-guardrails: + classification: constrained + status: + x-nemo-guardrails: + classification: opaque + type: + const: message + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - role + - content + title: ResponsesSnapshotMessageProjection + type: object + x-nemo-guardrails: + variant: + cardinality: at_most_one + match: + role: assistant + type: message + source: '#/components/schemas/OutputMessage' + unknown_fields: configurable + reject_case_aliases: true + type: array + x-nemo-guardrails: + classification: guarded + output_text: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: provider_integrity.duplicate_text + parallel_tool_calls: + x-nemo-guardrails: + classification: opaque + presence_penalty: + default: null + x-nemo-guardrails: + classification: opaque + previous_response_id: + x-nemo-guardrails: + classification: opaque + prompt: + x-nemo-guardrails: + classification: opaque + prompt_cache_diagnostics: + x-nemo-guardrails: + classification: opaque + prompt_cache_key: + x-nemo-guardrails: + classification: opaque + prompt_cache_options: + x-nemo-guardrails: + classification: opaque + prompt_cache_retention: + x-nemo-guardrails: + classification: opaque + reasoning: + x-nemo-guardrails: + classification: opaque + safety_identifier: + x-nemo-guardrails: + classification: opaque + service_tier: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + store: + default: null + x-nemo-guardrails: + classification: opaque + temperature: + x-nemo-guardrails: + classification: opaque + text: + x-nemo-guardrails: + classification: opaque + tool_choice: + x-nemo-guardrails: + classification: opaque + tool_usage: + default: null + x-nemo-guardrails: + classification: opaque + tools: + x-nemo-guardrails: + classification: opaque + top_logprobs: + x-nemo-guardrails: + classification: opaque + top_p: + x-nemo-guardrails: + classification: opaque + truncation: + x-nemo-guardrails: + classification: opaque + usage: + x-nemo-guardrails: + classification: opaque + user: + x-nemo-guardrails: + classification: opaque + required: + - output + title: LiteLLMLifecycleResponse + type: object + x-nemo-guardrails: + classification: guarded + source: '#/components/schemas/Response' + unknown_fields: forbid + reject_case_aliases: true + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.in_progress + type: string + x-nemo-guardrails: + classification: constrained + required: + - response + - type + title: LiteLLMInProgressEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseInProgressEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMInProgressEvent + event: + classification: snapshot + variants: + - source_schema: ResponseInProgressEvent + shape: response.in_progress + match: + type: response.in_progress + missing_text: empty + - additionalProperties: false + description: Recognize queued metadata without enabling background requests. + properties: + model: + default: null + x-nemo-guardrails: + classification: opaque + response: + additionalProperties: false + description: Apply the same reviewed metadata to initial and terminal response + snapshots. + properties: + access_programs: + default: null + x-nemo-guardrails: + classification: opaque + background: + x-nemo-guardrails: + classification: opaque + billing: + default: null + x-nemo-guardrails: + classification: opaque + completed_at: + x-nemo-guardrails: + classification: opaque + conversation: + x-nemo-guardrails: + classification: opaque + created_at: + x-nemo-guardrails: + classification: opaque + error: + x-nemo-guardrails: + classification: opaque + frequency_penalty: + default: null + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + incomplete_details: + x-nemo-guardrails: + classification: opaque + instructions: + x-nemo-guardrails: + classification: opaque + max_output_tokens: + x-nemo-guardrails: + classification: opaque + max_tool_calls: + x-nemo-guardrails: + classification: opaque + metadata: + x-nemo-guardrails: + classification: opaque + model: + x-nemo-guardrails: + classification: opaque + moderation: + x-nemo-guardrails: + classification: opaque + object: + x-nemo-guardrails: + classification: opaque + output: + items: + oneOf: + - additionalProperties: false + description: Preserve opaque reasoning metadata, not separate plaintext + reasoning. + properties: + content: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + encrypted_content: + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + summary: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + type: + const: reasoning + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + title: ResponsesReasoningItemProjection + type: object + x-nemo-guardrails: + variant: + cardinality: any + match: + type: reasoning + source: '#/components/schemas/ReasoningItem' + unknown_fields: configurable + reject_case_aliases: true + - additionalProperties: false + description: Allow an initially empty message, but never a second + guarded text part. + properties: + content: + items: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations + and logprobs stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + maxItems: 1 + type: array + x-nemo-guardrails: + classification: guarded + id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + phase: + x-nemo-guardrails: + classification: opaque + role: + const: assistant + type: string + x-nemo-guardrails: + classification: constrained + status: + x-nemo-guardrails: + classification: opaque + type: + const: message + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - role + - content + title: ResponsesSnapshotMessageProjection + type: object + x-nemo-guardrails: + variant: + cardinality: at_most_one + match: + role: assistant + type: message + source: '#/components/schemas/OutputMessage' + unknown_fields: configurable + reject_case_aliases: true + type: array + x-nemo-guardrails: + classification: guarded + output_text: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: provider_integrity.duplicate_text + parallel_tool_calls: + x-nemo-guardrails: + classification: opaque + presence_penalty: + default: null + x-nemo-guardrails: + classification: opaque + previous_response_id: + x-nemo-guardrails: + classification: opaque + prompt: + x-nemo-guardrails: + classification: opaque + prompt_cache_diagnostics: + x-nemo-guardrails: + classification: opaque + prompt_cache_key: + x-nemo-guardrails: + classification: opaque + prompt_cache_options: + x-nemo-guardrails: + classification: opaque + prompt_cache_retention: + x-nemo-guardrails: + classification: opaque + reasoning: + x-nemo-guardrails: + classification: opaque + safety_identifier: + x-nemo-guardrails: + classification: opaque + service_tier: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + store: + default: null + x-nemo-guardrails: + classification: opaque + temperature: + x-nemo-guardrails: + classification: opaque + text: + x-nemo-guardrails: + classification: opaque + tool_choice: + x-nemo-guardrails: + classification: opaque + tool_usage: + default: null + x-nemo-guardrails: + classification: opaque + tools: + x-nemo-guardrails: + classification: opaque + top_logprobs: + x-nemo-guardrails: + classification: opaque + top_p: + x-nemo-guardrails: + classification: opaque + truncation: + x-nemo-guardrails: + classification: opaque + usage: + x-nemo-guardrails: + classification: opaque + user: + x-nemo-guardrails: + classification: opaque + required: + - output + title: LiteLLMLifecycleResponse + type: object + x-nemo-guardrails: + classification: guarded + source: '#/components/schemas/Response' + unknown_fields: forbid + reject_case_aliases: true + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.queued + type: string + x-nemo-guardrails: + classification: constrained + required: + - response + - type + title: LiteLLMQueuedEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseQueuedEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMQueuedEvent + event: + classification: snapshot + variants: + - source_schema: ResponseQueuedEvent + shape: response.queued + match: + type: response.queued + missing_text: empty + - additionalProperties: false + description: Check the native terminal snapshot against accumulated deltas. + properties: + model: + default: null + x-nemo-guardrails: + classification: opaque + response: + additionalProperties: false + description: Apply the same reviewed metadata to initial and terminal response + snapshots. + properties: + access_programs: + default: null + x-nemo-guardrails: + classification: opaque + background: + x-nemo-guardrails: + classification: opaque + billing: + default: null + x-nemo-guardrails: + classification: opaque + completed_at: + x-nemo-guardrails: + classification: opaque + conversation: + x-nemo-guardrails: + classification: opaque + created_at: + x-nemo-guardrails: + classification: opaque + error: + x-nemo-guardrails: + classification: opaque + frequency_penalty: + default: null + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + incomplete_details: + x-nemo-guardrails: + classification: opaque + instructions: + x-nemo-guardrails: + classification: opaque + max_output_tokens: + x-nemo-guardrails: + classification: opaque + max_tool_calls: + x-nemo-guardrails: + classification: opaque + metadata: + x-nemo-guardrails: + classification: opaque + model: + x-nemo-guardrails: + classification: opaque + moderation: + x-nemo-guardrails: + classification: opaque + object: + x-nemo-guardrails: + classification: opaque + output: + items: + oneOf: + - additionalProperties: false + description: Preserve opaque reasoning metadata, not separate plaintext + reasoning. + properties: + content: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + encrypted_content: + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + summary: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + type: + const: reasoning + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + title: ResponsesReasoningItemProjection + type: object + x-nemo-guardrails: + variant: + cardinality: any + match: + type: reasoning + source: '#/components/schemas/ReasoningItem' + unknown_fields: configurable + reject_case_aliases: true + - additionalProperties: false + description: Allow an initially empty message, but never a second + guarded text part. + properties: + content: + items: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations + and logprobs stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + maxItems: 1 + type: array + x-nemo-guardrails: + classification: guarded + id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + phase: + x-nemo-guardrails: + classification: opaque + role: + const: assistant + type: string + x-nemo-guardrails: + classification: constrained + status: + x-nemo-guardrails: + classification: opaque + type: + const: message + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - role + - content + title: ResponsesSnapshotMessageProjection + type: object + x-nemo-guardrails: + variant: + cardinality: at_most_one + match: + role: assistant + type: message + source: '#/components/schemas/OutputMessage' + unknown_fields: configurable + reject_case_aliases: true + type: array + x-nemo-guardrails: + classification: guarded + output_text: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: provider_integrity.duplicate_text + parallel_tool_calls: + x-nemo-guardrails: + classification: opaque + presence_penalty: + default: null + x-nemo-guardrails: + classification: opaque + previous_response_id: + x-nemo-guardrails: + classification: opaque + prompt: + x-nemo-guardrails: + classification: opaque + prompt_cache_diagnostics: + x-nemo-guardrails: + classification: opaque + prompt_cache_key: + x-nemo-guardrails: + classification: opaque + prompt_cache_options: + x-nemo-guardrails: + classification: opaque + prompt_cache_retention: + x-nemo-guardrails: + classification: opaque + reasoning: + x-nemo-guardrails: + classification: opaque + safety_identifier: + x-nemo-guardrails: + classification: opaque + service_tier: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + store: + default: null + x-nemo-guardrails: + classification: opaque + temperature: + x-nemo-guardrails: + classification: opaque + text: + x-nemo-guardrails: + classification: opaque + tool_choice: + x-nemo-guardrails: + classification: opaque + tool_usage: + default: null + x-nemo-guardrails: + classification: opaque + tools: + x-nemo-guardrails: + classification: opaque + top_logprobs: + x-nemo-guardrails: + classification: opaque + top_p: + x-nemo-guardrails: + classification: opaque + truncation: + x-nemo-guardrails: + classification: opaque + usage: + x-nemo-guardrails: + classification: opaque + user: + x-nemo-guardrails: + classification: opaque + required: + - output + title: LiteLLMLifecycleResponse + type: object + x-nemo-guardrails: + classification: guarded + source: '#/components/schemas/Response' + unknown_fields: forbid + reject_case_aliases: true + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.completed + type: string + x-nemo-guardrails: + classification: constrained + required: + - response + - type + title: LiteLLMCompletedEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseCompletedEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMCompletedEvent + event: + classification: snapshot + variants: + - source_schema: ResponseCompletedEvent + shape: response.completed + match: + type: response.completed + missing_text: empty + - additionalProperties: false + description: Keep any output in failed responses subject to snapshot checks. + properties: + model: + default: null + x-nemo-guardrails: + classification: opaque + response: + additionalProperties: false + description: Apply the same reviewed metadata to initial and terminal response + snapshots. + properties: + access_programs: + default: null + x-nemo-guardrails: + classification: opaque + background: + x-nemo-guardrails: + classification: opaque + billing: + default: null + x-nemo-guardrails: + classification: opaque + completed_at: + x-nemo-guardrails: + classification: opaque + conversation: + x-nemo-guardrails: + classification: opaque + created_at: + x-nemo-guardrails: + classification: opaque + error: + x-nemo-guardrails: + classification: opaque + frequency_penalty: + default: null + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + incomplete_details: + x-nemo-guardrails: + classification: opaque + instructions: + x-nemo-guardrails: + classification: opaque + max_output_tokens: + x-nemo-guardrails: + classification: opaque + max_tool_calls: + x-nemo-guardrails: + classification: opaque + metadata: + x-nemo-guardrails: + classification: opaque + model: + x-nemo-guardrails: + classification: opaque + moderation: + x-nemo-guardrails: + classification: opaque + object: + x-nemo-guardrails: + classification: opaque + output: + items: + oneOf: + - additionalProperties: false + description: Preserve opaque reasoning metadata, not separate plaintext + reasoning. + properties: + content: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + encrypted_content: + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + summary: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + type: + const: reasoning + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + title: ResponsesReasoningItemProjection + type: object + x-nemo-guardrails: + variant: + cardinality: any + match: + type: reasoning + source: '#/components/schemas/ReasoningItem' + unknown_fields: configurable + reject_case_aliases: true + - additionalProperties: false + description: Allow an initially empty message, but never a second + guarded text part. + properties: + content: + items: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations + and logprobs stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + maxItems: 1 + type: array + x-nemo-guardrails: + classification: guarded + id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + phase: + x-nemo-guardrails: + classification: opaque + role: + const: assistant + type: string + x-nemo-guardrails: + classification: constrained + status: + x-nemo-guardrails: + classification: opaque + type: + const: message + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - role + - content + title: ResponsesSnapshotMessageProjection + type: object + x-nemo-guardrails: + variant: + cardinality: at_most_one + match: + role: assistant + type: message + source: '#/components/schemas/OutputMessage' + unknown_fields: configurable + reject_case_aliases: true + type: array + x-nemo-guardrails: + classification: guarded + output_text: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: provider_integrity.duplicate_text + parallel_tool_calls: + x-nemo-guardrails: + classification: opaque + presence_penalty: + default: null + x-nemo-guardrails: + classification: opaque + previous_response_id: + x-nemo-guardrails: + classification: opaque + prompt: + x-nemo-guardrails: + classification: opaque + prompt_cache_diagnostics: + x-nemo-guardrails: + classification: opaque + prompt_cache_key: + x-nemo-guardrails: + classification: opaque + prompt_cache_options: + x-nemo-guardrails: + classification: opaque + prompt_cache_retention: + x-nemo-guardrails: + classification: opaque + reasoning: + x-nemo-guardrails: + classification: opaque + safety_identifier: + x-nemo-guardrails: + classification: opaque + service_tier: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + store: + default: null + x-nemo-guardrails: + classification: opaque + temperature: + x-nemo-guardrails: + classification: opaque + text: + x-nemo-guardrails: + classification: opaque + tool_choice: + x-nemo-guardrails: + classification: opaque + tool_usage: + default: null + x-nemo-guardrails: + classification: opaque + tools: + x-nemo-guardrails: + classification: opaque + top_logprobs: + x-nemo-guardrails: + classification: opaque + top_p: + x-nemo-guardrails: + classification: opaque + truncation: + x-nemo-guardrails: + classification: opaque + usage: + x-nemo-guardrails: + classification: opaque + user: + x-nemo-guardrails: + classification: opaque + required: + - output + title: LiteLLMLifecycleResponse + type: object + x-nemo-guardrails: + classification: guarded + source: '#/components/schemas/Response' + unknown_fields: forbid + reject_case_aliases: true + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.failed + type: string + x-nemo-guardrails: + classification: constrained + required: + - response + - type + title: LiteLLMFailedEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseFailedEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMFailedEvent + event: + classification: snapshot + variants: + - source_schema: ResponseFailedEvent + shape: response.failed + match: + type: response.failed + missing_text: empty + - additionalProperties: false + description: Keep partial output guarded even when generation is incomplete. + properties: + model: + default: null + x-nemo-guardrails: + classification: opaque + response: + additionalProperties: false + description: Apply the same reviewed metadata to initial and terminal response + snapshots. + properties: + access_programs: + default: null + x-nemo-guardrails: + classification: opaque + background: + x-nemo-guardrails: + classification: opaque + billing: + default: null + x-nemo-guardrails: + classification: opaque + completed_at: + x-nemo-guardrails: + classification: opaque + conversation: + x-nemo-guardrails: + classification: opaque + created_at: + x-nemo-guardrails: + classification: opaque + error: + x-nemo-guardrails: + classification: opaque + frequency_penalty: + default: null + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + incomplete_details: + x-nemo-guardrails: + classification: opaque + instructions: + x-nemo-guardrails: + classification: opaque + max_output_tokens: + x-nemo-guardrails: + classification: opaque + max_tool_calls: + x-nemo-guardrails: + classification: opaque + metadata: + x-nemo-guardrails: + classification: opaque + model: + x-nemo-guardrails: + classification: opaque + moderation: + x-nemo-guardrails: + classification: opaque + object: + x-nemo-guardrails: + classification: opaque + output: + items: + oneOf: + - additionalProperties: false + description: Preserve opaque reasoning metadata, not separate plaintext + reasoning. + properties: + content: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + encrypted_content: + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + summary: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + type: + const: reasoning + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + title: ResponsesReasoningItemProjection + type: object + x-nemo-guardrails: + variant: + cardinality: any + match: + type: reasoning + source: '#/components/schemas/ReasoningItem' + unknown_fields: configurable + reject_case_aliases: true + - additionalProperties: false + description: Allow an initially empty message, but never a second + guarded text part. + properties: + content: + items: + additionalProperties: false + description: An empty initial text snapshot is valid; annotations + and logprobs stay restricted. + properties: + annotations: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.citation_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + logprobs: + default: null + x-nemo-guardrails: + classification: constrained + reason: provider_integrity.token_logprobs + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + text: + type: string + x-nemo-guardrails: + classification: guarded + subject: + kind: text + replaceable: false + role: assistant + type: + const: output_text + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - text + title: ResponsesSnapshotTextProjection + type: object + x-nemo-guardrails: + source: '#/components/schemas/OutputTextContent' + unknown_fields: configurable + reject_case_aliases: true + maxItems: 1 + type: array + x-nemo-guardrails: + classification: guarded + id: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - type: string + - type: 'null' + phase: + x-nemo-guardrails: + classification: opaque + role: + const: assistant + type: string + x-nemo-guardrails: + classification: constrained + status: + x-nemo-guardrails: + classification: opaque + type: + const: message + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - role + - content + title: ResponsesSnapshotMessageProjection + type: object + x-nemo-guardrails: + variant: + cardinality: at_most_one + match: + role: assistant + type: message + source: '#/components/schemas/OutputMessage' + unknown_fields: configurable + reject_case_aliases: true + type: array + x-nemo-guardrails: + classification: guarded + output_text: + default: null + type: 'null' + x-nemo-guardrails: + classification: constrained + gate: disabled + reason: provider_integrity.duplicate_text + parallel_tool_calls: + x-nemo-guardrails: + classification: opaque + presence_penalty: + default: null + x-nemo-guardrails: + classification: opaque + previous_response_id: + x-nemo-guardrails: + classification: opaque + prompt: + x-nemo-guardrails: + classification: opaque + prompt_cache_diagnostics: + x-nemo-guardrails: + classification: opaque + prompt_cache_key: + x-nemo-guardrails: + classification: opaque + prompt_cache_options: + x-nemo-guardrails: + classification: opaque + prompt_cache_retention: + x-nemo-guardrails: + classification: opaque + reasoning: + x-nemo-guardrails: + classification: opaque + safety_identifier: + x-nemo-guardrails: + classification: opaque + service_tier: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + store: + default: null + x-nemo-guardrails: + classification: opaque + temperature: + x-nemo-guardrails: + classification: opaque + text: + x-nemo-guardrails: + classification: opaque + tool_choice: + x-nemo-guardrails: + classification: opaque + tool_usage: + default: null + x-nemo-guardrails: + classification: opaque + tools: + x-nemo-guardrails: + classification: opaque + top_logprobs: + x-nemo-guardrails: + classification: opaque + top_p: + x-nemo-guardrails: + classification: opaque + truncation: + x-nemo-guardrails: + classification: opaque + usage: + x-nemo-guardrails: + classification: opaque + user: + x-nemo-guardrails: + classification: opaque + required: + - output + title: LiteLLMLifecycleResponse + type: object + x-nemo-guardrails: + classification: guarded + source: '#/components/schemas/Response' + unknown_fields: forbid + reject_case_aliases: true + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.incomplete + type: string + x-nemo-guardrails: + classification: constrained + required: + - response + - type + title: LiteLLMIncompleteEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseIncompleteEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMIncompleteEvent + event: + classification: snapshot + variants: + - source_schema: ResponseIncompleteEvent + shape: response.incomplete + match: + type: response.incomplete + missing_text: empty + - additionalProperties: false + description: Permit reviewed reasoning metadata, never new plaintext reasoning. + properties: + item: + additionalProperties: false + description: Preserve opaque reasoning metadata, not separate plaintext reasoning. + properties: + content: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + encrypted_content: + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + summary: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + type: + const: reasoning + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + title: ResponsesReasoningItemProjection + type: object + x-nemo-guardrails: + classification: constrained + variant: + cardinality: any + match: + type: reasoning + source: '#/components/schemas/ReasoningItem' + unknown_fields: configurable + reject_case_aliases: true + model: + default: null + x-nemo-guardrails: + classification: opaque + output_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.output_item.added + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - item + title: LiteLLMReasoningAddedEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseOutputItemAddedEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMReasoningAddedEvent + event: + classification: opaque + variants: + - source_schema: ResponseOutputItemAddedEvent + shape: response.output_item.added:reasoning + match: + type: response.output_item.added + item.type: reasoning + - additionalProperties: false + description: Keep reasoning summaries and content constrained to empty values. + properties: + item: + additionalProperties: false + description: Preserve opaque reasoning metadata, not separate plaintext reasoning. + properties: + content: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + encrypted_content: + x-nemo-guardrails: + classification: opaque + id: + x-nemo-guardrails: + classification: opaque + status: + x-nemo-guardrails: + classification: opaque + summary: + default: null + x-nemo-guardrails: + classification: constrained + reason: core_capability.reasoning_content + oneOf: + - items: {} + maxItems: 0 + type: array + - type: 'null' + type: + const: reasoning + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + title: ResponsesReasoningItemProjection + type: object + x-nemo-guardrails: + classification: constrained + variant: + cardinality: any + match: + type: reasoning + source: '#/components/schemas/ReasoningItem' + unknown_fields: configurable + reject_case_aliases: true + model: + default: null + x-nemo-guardrails: + classification: opaque + output_index: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: response.output_item.done + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - item + title: LiteLLMReasoningDoneEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseOutputItemDoneEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMReasoningDoneEvent + event: + classification: opaque + variants: + - source_schema: ResponseOutputItemDoneEvent + shape: response.output_item.done:reasoning + match: + type: response.output_item.done + item.type: reasoning + - additionalProperties: false + description: Accept model metadata on the inherited flat diagnostic error shape. + properties: + code: + x-nemo-guardrails: + classification: opaque + message: + type: string + x-nemo-guardrails: + classification: constrained + model: + default: null + x-nemo-guardrails: + classification: opaque + param: + x-nemo-guardrails: + classification: opaque + sequence_number: + default: null + x-nemo-guardrails: + classification: constrained + oneOf: + - minimum: 0 + type: integer + - type: 'null' + type: + const: error + type: string + x-nemo-guardrails: + classification: constrained + required: + - type + - message + title: LiteLLMErrorEvent + type: object + x-nemo-guardrails: + source: '#/components/schemas/ResponseErrorEvent' + unknown_fields: forbid + reject_case_aliases: true + model: LiteLLMErrorEvent + event: + classification: provider_error + variants: + - source_schema: ResponseErrorEvent + shape: error + match: + type: error + x-nemo-guardrails: + transport: + require_sse_event: false + data_discriminator: type + non_data_shape: keepalive + sentinels: + '[DONE]': done diff --git a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md index b7aec74d25..fe3a793dc6 100644 --- a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md +++ b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md @@ -1,5 +1,9 @@ # LiteLLM Chat dialect +For the separately reviewed Responses operation, see the +[LiteLLM Responses dialect](responses.md). Chat extension bags and backend +evidence do not automatically apply to Responses. + This experimental dialect applies the `single_text.v1` guardrail profile to reviewed LiteLLM Chat Completions wire shapes. Typed Python declarations in [chat_completions.py](chat_completions.py) and [stream.py](stream.py) are diff --git a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md new file mode 100644 index 0000000000..5f4417d00a --- /dev/null +++ b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md @@ -0,0 +1,119 @@ +# LiteLLM Responses dialect + +The experimental [typed declaration](responses.py) binds the existing Responses +single-text policy to a reviewed LiteLLM gateway surface. It is selected explicitly; +the strict OpenAI endpoint and the LiteLLM Chat dialect are unchanged. No LiteLLM +package, new transport, new kernel, or alternate lifecycle hooks are required. + +## Policy difference + +| Location | Decision | +| --- | --- | +| Request | Reuse the strict Responses binding unchanged: one string input, foreground generation, no tools, conversation history, or additional plaintext instructions. | +| Buffered response and lifecycle `response` snapshots | Recognize optional `access_programs`, `billing`, `frequency_penalty`, `presence_penalty`, `store`, and `tool_usage` as opaque metadata. | +| Root of each reviewed SSE event | Recognize optional `model` as opaque metadata, never as an event selector or text source. | +| Output items, content parts, delta text, annotations, logprobs, reasoning summaries | Inherit the existing constrained/guarded policy unchanged. No Chat-specific `provider_specific_fields` bags are added. | +| Event selection and lifecycle | Retain all 15 existing event rules, derived paths/coverage, native terminal requirements, identity checks, snapshot consistency, and error framing. | + +Opaque metadata is passed through **without inspecting its values or validating +its nested structure**. For example, `tool_usage` is accounting metadata, not +permission to emit tool calls. The actual output items still reject tool content. +Allowing `store` in the response does not introduce a new request control: it was +already reviewed in the base request declaration. + +These names describe the reviewed **gateway surface**, not fields all invented +by LiteLLM. For example, `access_programs` appears in the +[native OpenAI response reference](https://developers.openai.com/api/reference/resources/responses/methods/create). +This change does not broaden the strict endpoint's pinned field inventory. +Unknown root members, new events and unsupported content still fail closed. +The base profile's trusted nested unknown-field override remains unchanged. + +## Explicit composition + +Use the existing application assembly with trusted endpoint declarations: + +```python +from dataclasses import replace + +from nemoguardrails.server.experimental.providers.openai.integration import create_openai_integration +from nemoguardrails.server.experimental.providers.openai.dialects.litellm.chat_completions import ( + LITELLM_CHAT_COMPLETIONS_ENDPOINT, +) +from nemoguardrails.server.experimental.providers.openai.dialects.litellm.responses import ( + LITELLM_RESPONSES_ENDPOINT, +) + +integration = replace( + create_openai_integration(base_url="https://your-gateway.example/v1"), + endpoints=(LITELLM_CHAT_COMPLETIONS_ENDPOINT, LITELLM_RESPONSES_ENDPOINT), +) +``` + +Supply this integration to the existing proxy application alongside its Rails +configuration. Credentials use the application's existing HTTP authorization +path; neither YAML exports, model strings nor gateway hostnames select a dialect. +This is the transparent HTTP path, not a LangChain or LiteLLM client integration. + +## Source and recording evidence + +On 2026-10-15 the authenticated deployment OpenAPI advertised LiteLLM `1.105.1`. +Both `/responses` and `/v1/responses` declared `ResponsesAPIResponse` for buffered +success and a string SSE body for streaming. The latter is not an event schema. +The private schema is not vendored, and its advertised version does not establish +the absence of deployment patches. + +The matching public tag resolves to +[`d665dc1a6315847fcad6cc2c1264ceb47c06abce`](https://github.com/BerriAI/litellm/tree/d665dc1a6315847fcad6cc2c1264ceb47c06abce): + +- [`litellm/types/llms/openai.py`](https://github.com/BerriAI/litellm/blob/d665dc1a6315847fcad6cc2c1264ceb47c06abce/litellm/types/llms/openai.py) + declares `ResponsesAPIResponse`, lifecycle events and a separate nested error + shape. SHA-256: `99ce22ba2bb83dc37d9afb423ef2a6cf919a9cafca834eaad833c4a795088fdd`. +- [`litellm/llms/openai/responses/transformation.py`](https://github.com/BerriAI/litellm/blob/d665dc1a6315847fcad6cc2c1264ceb47c06abce/litellm/llms/openai/responses/transformation.py) + constructs a Responses object from native OpenAI JSON, with a construction + fallback on validation failure. SHA-256: `7c6f3bea7678b76b6e99bc5b4c499eef9aead8947a6a81e534dc5e42c84bfc70`. + +Two genuine cassettes live in +`tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/`. +They capture `openai/openai/gpt-5-nano` through NVIDIA's inference gateway, with a +harmless greeting, `store: false`, and a 256-token output limit. Buffered evidence +includes all six metadata additions. Streaming evidence includes the model +annotation and initial/final snapshots, reasoning-item metadata, assistant-item +snapshots, content-part snapshots and text deltas/done. No new event type or Chat +extension bag was observed. They are evidence for this route, not raw vLLM or +every LiteLLM backend. + +The [LiteLLM Responses documentation](https://docs.litellm.ai/docs/response_api) +also describes Chat-to-Responses bridging. These cassettes do not establish +compatibility with bridged routes. Nested LiteLLM SSE errors, additional content +channels, and other backend-specific fields remain unsupported and fail closed. +Flat diagnostic errors and other inherited events have synthetic unit coverage; +they are not claimed as recorded observations. + +The recording sanitizer removes credentials, routing/item identifiers, timestamps +and encrypted provider state. It preserves distinct item IDs and their references, +text, event order, sequence numbers and field presence. Privacy redaction is not +a policy repair: unsupported fields must not be deleted to make replay pass. +Exact passthrough comparisons are against the sanitized replay bytes, not a claim +that the cassette preserves original HTTP chunk boundaries or all original bytes. + +## Validation and export + +```bash +NVIDIA_INFERENCE_KEY=... make record-cassettes \ + RECORDED_TESTS=tests/recorded/server/experimental/openai/test_litellm_responses.py \ + RECORDED_REQUIRED_KEYS=NVIDIA_INFERENCE_KEY + +make replay-cassettes \ + RECORDED_TESTS=tests/recorded/server/experimental/openai/test_litellm_responses.py + +uv run --locked python -m nemoguardrails.server.experimental.provider.contract_export \ + nemoguardrails.server.experimental.providers.openai.dialects.litellm.responses:LITELLM_RESPONSES_ENDPOINT \ + --output nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml +``` + +Capture consumes the entire upstream response before policy assertions. Mandatory +offline replay then checks the actual endpoint, export, text extraction, terminal +behavior and passthrough bytes. A successful capture alone is not compatibility +validation. Dialect rejection cases use synthetic unit fixtures; generic blocking, +cancellation and malformed-stream tests remain with the existing Responses stack. +The exported YAML describes the effective declarations and is not a runtime input. diff --git a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.py b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.py new file mode 100644 index 0000000000..4020ddad88 --- /dev/null +++ b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.py @@ -0,0 +1,221 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Bind reviewed LiteLLM Responses metadata without relaxing text boundaries. + +The added fields describe the observed gateway surface, not a claim that every +field originates in LiteLLM. Requests and output-item policy remain OpenAI's. +See responses.md for source, recording provenance and explicit selection. +""" + +from dataclasses import replace +from typing import Annotated, Any + +from pydantic import BaseModel + +from nemoguardrails.server.experimental.provider.projection_policy import ( + PolicyModel, + guarded, + opaque, + payload_contract, + text_location, +) +from nemoguardrails.server.experimental.provider.stream import StreamBinding +from nemoguardrails.server.experimental.provider.stream_classifier import StreamClassifierDefinition +from nemoguardrails.server.experimental.provider.stream_policy import ( + build_policy_stream_classifier, + stream_shape_coverage, + stream_text_path, +) +from nemoguardrails.server.experimental.providers.openai.responses import stream_projection as models +from nemoguardrails.server.experimental.providers.openai.responses.endpoint import RESPONSES_ENDPOINT +from nemoguardrails.server.experimental.providers.openai.responses.response_projection import ( + ResponsesGuardedResponseProjection, +) +from nemoguardrails.server.experimental.providers.openai.responses.stream_classifier import ( + STREAM_CONTRACT as OPENAI_STREAM_CONTRACT, +) +from nemoguardrails.server.experimental.providers.openai.responses.stream_classifier import ( + STREAM_NON_DATA_SHAPE, + STREAM_SENTINELS, +) +from nemoguardrails.server.experimental.providers.openai.responses.stream_classifier import ( + STREAM_RULES as OPENAI_STREAM_RULES, +) +from nemoguardrails.server.experimental.providers.openai.responses.stream_hooks import ResponsesStreamHooks + + +class LiteLLMResponsesMetadata(PolicyModel): + """Preserve reviewed accounting and response-control metadata uninspected. + + These are optional response fields, never request controls or event selectors. + Containers are intentionally opaque like the inherited usage metadata; this + does not authorize new output items, plaintext reasoning, or extension bags. + """ + + access_programs: Annotated[Any, opaque()] = None + billing: Annotated[Any, opaque()] = None + frequency_penalty: Annotated[Any, opaque()] = None + presence_penalty: Annotated[Any, opaque()] = None + store: Annotated[Any, opaque()] = None + tool_usage: Annotated[Any, opaque()] = None + + +class LiteLLMResponsesProjection(ResponsesGuardedResponseProjection, LiteLLMResponsesMetadata): + """Retain one assistant text output and the inherited reasoning restrictions.""" + + +class LiteLLMGuardedResponses(LiteLLMResponsesProjection): + """Derive coverage and extraction from the complete dialect declaration.""" + + guarded_text_location = text_location(LiteLLMResponsesProjection) + projection_contract = payload_contract( + LiteLLMResponsesProjection, + projection_id="openai.litellm.responses.response.text.v1", + direction="response", + ) + + +class LiteLLMResponsesEventMetadata(PolicyModel): + """Accept the gateway's model annotation without using it to select events.""" + + model: Annotated[Any, opaque()] = None + + +class LiteLLMLifecycleResponse(models.ResponsesLifecycleResponseProjection, LiteLLMResponsesMetadata): + """Apply the same reviewed metadata to initial and terminal response snapshots.""" + + +class LiteLLMTextDeltaEvent(models.ResponsesTextDeltaEventProjection, LiteLLMResponsesEventMetadata): + """Keep the delta as the only newly generated text subject.""" + + +class LiteLLMTextDoneEvent(models.ResponsesTextDoneEventProjection, LiteLLMResponsesEventMetadata): + """Keep completed text subject to snapshot consistency checks.""" + + +class LiteLLMPartAddedEvent(models.ResponsesContentPartAddedEventProjection, LiteLLMResponsesEventMetadata): + """Retain the initial text part and its inherited content restrictions.""" + + +class LiteLLMPartDoneEvent(models.ResponsesContentPartDoneEventProjection, LiteLLMResponsesEventMetadata): + """Retain the completed text part as a checked snapshot.""" + + +class LiteLLMMessageAddedEvent(models.ResponsesMessageAddedEventProjection, LiteLLMResponsesEventMetadata): + """Preserve message identity and single-part cardinality.""" + + +class LiteLLMMessageDoneEvent(models.ResponsesMessageDoneEventProjection, LiteLLMResponsesEventMetadata): + """Preserve message completion checks and the original item identity.""" + + +class LiteLLMReasoningAddedEvent(models.ResponsesReasoningAddedEventProjection, LiteLLMResponsesEventMetadata): + """Permit reviewed reasoning metadata, never new plaintext reasoning.""" + + +class LiteLLMReasoningDoneEvent(models.ResponsesReasoningDoneEventProjection, LiteLLMResponsesEventMetadata): + """Keep reasoning summaries and content constrained to empty values.""" + + +class LiteLLMCreatedEvent(models.ResponsesCreatedEventProjection, LiteLLMResponsesEventMetadata): + """Check the creation snapshot with the reviewed response metadata.""" + + response: Annotated[LiteLLMLifecycleResponse, guarded()] + + +class LiteLLMInProgressEvent(models.ResponsesInProgressEventProjection, LiteLLMResponsesEventMetadata): + """Check the progress snapshot without introducing a separate text target.""" + + response: Annotated[LiteLLMLifecycleResponse, guarded()] + + +class LiteLLMQueuedEvent(models.ResponsesQueuedEventProjection, LiteLLMResponsesEventMetadata): + """Recognize queued metadata without enabling background requests.""" + + response: Annotated[LiteLLMLifecycleResponse, guarded()] + + +class LiteLLMCompletedEvent(models.ResponsesCompletedEventProjection, LiteLLMResponsesEventMetadata): + """Check the native terminal snapshot against accumulated deltas.""" + + response: Annotated[LiteLLMLifecycleResponse, guarded()] + + +class LiteLLMFailedEvent(models.ResponsesFailedEventProjection, LiteLLMResponsesEventMetadata): + """Keep any output in failed responses subject to snapshot checks.""" + + response: Annotated[LiteLLMLifecycleResponse, guarded()] + + +class LiteLLMIncompleteEvent(models.ResponsesIncompleteEventProjection, LiteLLMResponsesEventMetadata): + """Keep partial output guarded even when generation is incomplete.""" + + response: Annotated[LiteLLMLifecycleResponse, guarded()] + + +class LiteLLMErrorEvent(models.ResponsesErrorEventProjection, LiteLLMResponsesEventMetadata): + """Accept model metadata on the inherited flat diagnostic error shape.""" + + +_EVENT_MODELS: dict[type[BaseModel], type[PolicyModel]] = { + models.ResponsesTextDeltaEventProjection: LiteLLMTextDeltaEvent, + models.ResponsesTextDoneEventProjection: LiteLLMTextDoneEvent, + models.ResponsesContentPartAddedEventProjection: LiteLLMPartAddedEvent, + models.ResponsesContentPartDoneEventProjection: LiteLLMPartDoneEvent, + models.ResponsesMessageAddedEventProjection: LiteLLMMessageAddedEvent, + models.ResponsesMessageDoneEventProjection: LiteLLMMessageDoneEvent, + models.ResponsesReasoningAddedEventProjection: LiteLLMReasoningAddedEvent, + models.ResponsesReasoningDoneEventProjection: LiteLLMReasoningDoneEvent, + models.ResponsesCreatedEventProjection: LiteLLMCreatedEvent, + models.ResponsesInProgressEventProjection: LiteLLMInProgressEvent, + models.ResponsesQueuedEventProjection: LiteLLMQueuedEvent, + models.ResponsesCompletedEventProjection: LiteLLMCompletedEvent, + models.ResponsesFailedEventProjection: LiteLLMFailedEvent, + models.ResponsesIncompleteEventProjection: LiteLLMIncompleteEvent, + models.ResponsesErrorEventProjection: LiteLLMErrorEvent, +} +STREAM_RULES = tuple( + replace( + rule, + model=_EVENT_MODELS[rule.model], + text_path=stream_text_path(_EVENT_MODELS[rule.model]) if rule.text_path is not None else None, + ) + for rule in OPENAI_STREAM_RULES +) +STREAM_CONTRACT = replace( + OPENAI_STREAM_CONTRACT, + projection_id="openai.litellm.responses.stream.text.v1", + shapes=stream_shape_coverage(STREAM_RULES, sentinels=STREAM_SENTINELS, non_data_shape=STREAM_NON_DATA_SHAPE), +) +STREAM_CLASSIFIER = build_policy_stream_classifier( + StreamClassifierDefinition( + subject=STREAM_CONTRACT.projection_id, + contract=STREAM_CONTRACT, + rules=STREAM_RULES, + event_type_field="type", + sentinels=STREAM_SENTINELS, + non_data_shape=STREAM_NON_DATA_SHAPE, + ) +) + + +LITELLM_RESPONSES_ENDPOINT = replace( + RESPONSES_ENDPOINT, + contract_name="litellm_responses", + operation="LiteLLM Responses", + guarded_response_model=LiteLLMGuardedResponses, + stream=StreamBinding(STREAM_CLASSIFIER, ResponsesStreamHooks), +) diff --git a/tests/recorded/README.md b/tests/recorded/README.md index e1ce02892f..fee44c3a68 100644 --- a/tests/recorded/README.md +++ b/tests/recorded/README.md @@ -128,6 +128,15 @@ exchanges with synthetic checker decisions, not real Rails configuration. Replayed-body equality is not a claim of original provider-wire byte fidelity or live streaming delivery. Real-config and socket coverage is separate. +The separately reviewed Responses recordings explicitly select the +[LiteLLM Responses dialect](../../nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md). +Refresh only `tests/recorded/server/experimental/openai/test_litellm_responses.py` +with `RECORDED_REQUIRED_KEYS=NVIDIA_INFERENCE_KEY`. Capture stores the complete +sanitized response before checking policy; the subsequent offline replay is the +acceptance gate. The local sanitizer preserves distinct item-ID relationships +and removes encrypted provider state. These two cassettes establish neither Chat +bridge nor arbitrary-backend compatibility. + These six cassettes predate the dialect: their provider payloads are unchanged. Two buffered successes contain message `provider_specific_fields: {refusal: null}` and empty choice extensions. Two streaming successes contain no extension bags; diff --git a/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[buffered].yaml b/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[buffered].yaml new file mode 100644 index 0000000000..ab2ceca76e --- /dev/null +++ b/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[buffered].yaml @@ -0,0 +1,127 @@ +version: 1 +interactions: +- request: + method: POST + uri: https://inference-api.nvidia.com/v1/responses + headers: + host: + - inference-api.nvidia.com + accept: + - '*/*' + connection: + - keep-alive + user-agent: + - python-httpx/0.28.1 + accept-encoding: + - identity + content-type: + - application/json + parsed_body: + model: openai/openai/gpt-5-nano + input: 'Reply with exactly: Hello.' + stream: false + store: false + max_output_tokens: 256 + reasoning: + effort: minimal + response: + status: + code: 200 + message: OK + headers: + content-type: + - application/json + content-security-policy: + - frame-ancestors 'none' + vary: + - Accept-Encoding + body: + parsed_body: + id: '[RECORDED_RESPONSE_ID]' + created_at: 0 + error: null + incomplete_details: null + instructions: null + metadata: {} + model: openai/openai/gpt-5-nano + object: response + output: + - id: recorded_id_1 + summary: [] + type: reasoning + content: [] + encrypted_content: '[REDACTED_ENCRYPTED_CONTENT]' + - id: recorded_id_2 + content: + - annotations: [] + text: Hello. + type: output_text + logprobs: [] + role: assistant + status: completed + type: message + phase: null + parallel_tool_calls: true + temperature: 1.0 + tool_choice: auto + tools: [] + top_p: 1.0 + max_output_tokens: 256 + previous_response_id: null + reasoning: + context: current_turn + effort: minimal + mode: standard + summary: null + status: completed + text: + format: + type: text + verbosity: medium + truncation: disabled + usage: + input_tokens: 12 + input_tokens_details: + audio_tokens: null + cached_tokens: 0 + cached_tokens_details: null + image_tokens: null + text_tokens: null + video_tokens: null + cache_write_tokens: 0 + output_tokens: 20 + output_tokens_details: + audio_tokens: null + reasoning_tokens: 0 + text_tokens: null + total_tokens: 32 + cost: null + user: null + store: false + access_programs: null + background: false + billing: + payer: developer + completed_at: 0 + frequency_penalty: 0.0 + max_tool_calls: null + moderation: null + presence_penalty: 0.0 + prompt_cache_key: null + prompt_cache_retention: in_memory + safety_identifier: null + service_tier: default + tool_usage: + image_gen: + input_tokens: 0 + input_tokens_details: + image_tokens: 0 + text_tokens: 0 + output_tokens: 0 + output_tokens_details: + image_tokens: 0 + text_tokens: 0 + total_tokens: 0 + web_search: + num_requests: 0 + top_logprobs: 0 diff --git a/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[streaming].yaml b/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[streaming].yaml new file mode 100644 index 0000000000..b005b8af66 --- /dev/null +++ b/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[streaming].yaml @@ -0,0 +1,300 @@ +version: 1 +interactions: +- request: + method: POST + uri: https://inference-api.nvidia.com/v1/responses + headers: + host: + - inference-api.nvidia.com + accept: + - '*/*' + connection: + - keep-alive + user-agent: + - python-httpx/0.28.1 + accept-encoding: + - identity + content-type: + - application/json + parsed_body: + model: openai/openai/gpt-5-nano + input: 'Reply with exactly: Hello.' + stream: true + store: false + max_output_tokens: 256 + reasoning: + effort: minimal + response: + status: + code: 200 + message: OK + headers: + cache-control: + - no-cache + content-type: + - text/event-stream; charset=utf-8 + content-security-policy: + - frame-ancestors 'none' + transfer-encoding: + - chunked + body: + parsed_body: + - type: response.created + response: + id: recorded_id_0 + created_at: 0 + metadata: {} + model: gpt-5-nano-2025-08-07 + object: response + output: [] + parallel_tool_calls: true + temperature: 1.0 + tool_choice: auto + tools: [] + top_p: 1.0 + max_output_tokens: 256 + reasoning: + context: current_turn + effort: minimal + mode: standard + summary: null + status: in_progress + text: + format: + type: text + verbosity: medium + truncation: disabled + store: false + background: false + frequency_penalty: 0.0 + presence_penalty: 0.0 + prompt_cache_retention: in_memory + service_tier: auto + tool_usage: + image_gen: + input_tokens: 0 + input_tokens_details: + image_tokens: 0 + text_tokens: 0 + output_tokens: 0 + output_tokens_details: + image_tokens: 0 + text_tokens: 0 + total_tokens: 0 + web_search: + num_requests: 0 + top_logprobs: 0 + sequence_number: 0 + model: openai/openai/gpt-5-nano + - type: response.in_progress + response: + id: recorded_id_0 + created_at: 0 + metadata: {} + model: gpt-5-nano-2025-08-07 + object: response + output: [] + parallel_tool_calls: true + temperature: 1.0 + tool_choice: auto + tools: [] + top_p: 1.0 + max_output_tokens: 256 + reasoning: + context: current_turn + effort: minimal + mode: standard + summary: null + status: in_progress + text: + format: + type: text + verbosity: medium + truncation: disabled + store: false + background: false + frequency_penalty: 0.0 + presence_penalty: 0.0 + prompt_cache_retention: in_memory + service_tier: auto + tool_usage: + image_gen: + input_tokens: 0 + input_tokens_details: + image_tokens: 0 + text_tokens: 0 + output_tokens: 0 + output_tokens_details: + image_tokens: 0 + text_tokens: 0 + total_tokens: 0 + web_search: + num_requests: 0 + top_logprobs: 0 + sequence_number: 1 + model: openai/openai/gpt-5-nano + - type: response.output_item.added + output_index: 0 + item: + id: recorded_id_1 + type: reasoning + content: [] + encrypted_content: '[REDACTED_ENCRYPTED_CONTENT]' + summary: [] + sequence_number: 2 + model: openai/openai/gpt-5-nano + - type: response.output_item.done + output_index: 0 + sequence_number: 3 + item: + id: recorded_id_1 + type: reasoning + content: [] + encrypted_content: '[REDACTED_ENCRYPTED_CONTENT]' + summary: [] + model: openai/openai/gpt-5-nano + - type: response.output_item.added + output_index: 1 + item: + id: recorded_id_2 + type: message + status: in_progress + content: [] + role: assistant + sequence_number: 4 + model: openai/openai/gpt-5-nano + - type: response.content_part.added + item_id: recorded_id_2 + output_index: 1 + content_index: 0 + part: + type: output_text + annotations: [] + logprobs: [] + text: '' + sequence_number: 5 + model: openai/openai/gpt-5-nano + - type: response.output_text.delta + item_id: recorded_id_2 + output_index: 1 + content_index: 0 + delta: Hello + logprobs: [] + obfuscation: k5gxEnU5U0A + sequence_number: 6 + model: openai/openai/gpt-5-nano + - type: response.output_text.delta + item_id: recorded_id_2 + output_index: 1 + content_index: 0 + delta: . + logprobs: [] + obfuscation: 9p6RkErDA9Rqkiy + sequence_number: 7 + model: openai/openai/gpt-5-nano + - type: response.output_text.done + item_id: recorded_id_2 + output_index: 1 + content_index: 0 + text: Hello. + logprobs: [] + sequence_number: 8 + model: openai/openai/gpt-5-nano + - type: response.content_part.done + item_id: recorded_id_2 + output_index: 1 + content_index: 0 + part: + type: output_text + text: Hello. + annotations: [] + logprobs: [] + sequence_number: 9 + model: openai/openai/gpt-5-nano + - type: response.output_item.done + output_index: 1 + sequence_number: 10 + item: + id: recorded_id_2 + type: message + status: completed + content: + - type: output_text + annotations: [] + logprobs: [] + text: Hello. + role: assistant + model: openai/openai/gpt-5-nano + - type: response.completed + response: + id: recorded_id_0 + created_at: 0 + metadata: {} + model: gpt-5-nano-2025-08-07 + object: response + output: + - id: recorded_id_3 + summary: [] + type: reasoning + content: [] + encrypted_content: '[REDACTED_ENCRYPTED_CONTENT]' + - id: recorded_id_2 + content: + - annotations: [] + text: Hello. + type: output_text + logprobs: [] + role: assistant + status: completed + type: message + parallel_tool_calls: true + temperature: 1.0 + tool_choice: auto + tools: [] + top_p: 1.0 + max_output_tokens: 256 + reasoning: + context: current_turn + effort: minimal + mode: standard + summary: null + status: completed + text: + format: + type: text + verbosity: medium + truncation: disabled + usage: + input_tokens: 12 + input_tokens_details: + cached_tokens: 0 + cache_write_tokens: 0 + output_tokens: 20 + output_tokens_details: + reasoning_tokens: 0 + total_tokens: 32 + cost: 6.312399999999999e-06 + store: false + background: false + completed_at: 0 + frequency_penalty: 0.0 + presence_penalty: 0.0 + prompt_cache_retention: in_memory + service_tier: default + tool_usage: + image_gen: + input_tokens: 0 + input_tokens_details: + image_tokens: 0 + text_tokens: 0 + output_tokens: 0 + output_tokens_details: + image_tokens: 0 + text_tokens: 0 + total_tokens: 0 + web_search: + num_requests: 0 + top_logprobs: 0 + sequence_number: 11 + model: openai/openai/gpt-5-nano + - '[DONE]' diff --git a/tests/recorded/server/experimental/openai/responses_support.py b/tests/recorded/server/experimental/openai/responses_support.py new file mode 100644 index 0000000000..707d7e64d3 --- /dev/null +++ b/tests/recorded/server/experimental/openai/responses_support.py @@ -0,0 +1,79 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Sanitize Responses evidence without merging distinct stream item identities.""" + +import json +from copy import deepcopy + +from tests.recorded.conftest import before_record_response + + +def sanitize_responses_recording(response): + """Remove routing identifiers while retaining identity equality and event order. + + One mapping spans the complete HTTP body, including all SSE events. Unlike + replacing every id with one sentinel, this preserves mismatches and distinct + items. Text, sequence numbers, output indices and extension fields survive. + The shared sanitizer still owns credential and HTTP header redaction. + """ + response = deepcopy(response) + identities = {} + + def scrub(value): + if isinstance(value, list): + return [scrub(item) for item in value] + if not isinstance(value, dict): + return value + result = {} + for key, nested in value.items(): + if key in {"id", "item_id", "response_id"} and isinstance(nested, str): + result[key] = identities.setdefault(nested, f"recorded_id_{len(identities)}") + elif key in {"created_at", "completed_at"} and nested is not None: + result[key] = 0 + elif key == "encrypted_content" and isinstance(nested, str): + result[key] = "[REDACTED_ENCRYPTED_CONTENT]" + else: + result[key] = scrub(nested) + return result + + body = response.get("body", {}).get("string") + if isinstance(body, (str, bytes)): + text = body.decode() if isinstance(body, bytes) else body + content_type = next( + (value for name, value in response.get("headers", {}).items() if name.lower() == "content-type"), [] + ) + if isinstance(content_type, str): + content_type = [content_type] + if any("text/event-stream" in value.lower() for value in content_type): + lines = [] + for line in text.splitlines(keepends=True): + if line.startswith("data: ") and line.strip() != "data: [DONE]": + try: + payload = json.loads(line[6:]) + except json.JSONDecodeError: + pass + else: + ending = line[len(line.rstrip("\r\n")) :] + line = "data: " + json.dumps(scrub(payload), separators=(",", ":")) + ending + lines.append(line) + text = "".join(lines) + else: + try: + text = json.dumps(scrub(json.loads(text))) + except json.JSONDecodeError: + pass + response["body"]["string"] = text.encode() if isinstance(body, bytes) else text + return before_record_response(response) diff --git a/tests/recorded/server/experimental/openai/test_litellm_responses.py b/tests/recorded/server/experimental/openai/test_litellm_responses.py new file mode 100644 index 0000000000..3309f64c7c --- /dev/null +++ b/tests/recorded/server/experimental/openai/test_litellm_responses.py @@ -0,0 +1,231 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Replay gateway Responses evidence through the actual bound policy and HTTP path. + +Capture completes before policy validation, so a new unsupported shape is not +silently truncated or changed to fit the contract. Recording is followed by the +mandatory offline replay, where bound-policy and passthrough assertions run. +""" + +import json +from contextlib import asynccontextmanager + +import httpx +import pytest +from fastapi import FastAPI +from jsonschema import Draft202012Validator + +from nemoguardrails.server.experimental._content_checker import ( + ContentAllowed, + ContentInspectionPolicy, + StreamBufferingPolicy, +) +from nemoguardrails.server.experimental._guarded_proxy import create_guarded_http_operation +from nemoguardrails.server.experimental._http_kernel import create_http_proxy_router +from nemoguardrails.server.experimental._provider_transport import ProviderTransport +from nemoguardrails.server.experimental.provider.contract_export import export_guard_contract +from nemoguardrails.server.experimental.provider.integration import UpstreamSettings +from nemoguardrails.server.experimental.provider.sse import iter_sse_events +from nemoguardrails.server.experimental.provider.stream import StreamEventRole +from nemoguardrails.server.experimental.providers.openai.dialects.litellm.responses import LITELLM_RESPONSES_ENDPOINT +from nemoguardrails.server.experimental.providers.openai.errors import OPENAI_ERROR_MAPPING +from nemoguardrails.server.experimental.providers.openai.responses.endpoint import RESPONSES_ENDPOINT +from tests.recorded.cassette import cassette_request_jsons +from tests.recorded.server.experimental.openai.responses_support import sanitize_responses_recording + +pytestmark = [ + pytest.mark.recorded, + pytest.mark.vcr( + ignore_hosts=["recorded-proxy.test"], + allow_playback_repeats=True, + before_record_response=sanitize_responses_recording, + ), + pytest.mark.asyncio, +] + +BASE_URL = "https://inference-api.nvidia.com/v1" +MODEL = "openai/openai/gpt-5-nano" +INPUT = "Reply with exactly: Hello." +ENDPOINT = LITELLM_RESPONSES_ENDPOINT + + +def responses_request(stream, *, model=MODEL): + return { + "model": model, + "input": INPUT, + "stream": stream, + "store": False, + "max_output_tokens": 256, + "reasoning": {"effort": "minimal"}, + } + + +class Checker: + def __init__(self): + self.calls = [] + + def inspection_policy(self): + return ContentInspectionPolicy(True, True, StreamBufferingPolicy(1, 0)) + + async def check_input(self, check): + self.calls.append(("input", check.message.content)) + return ContentAllowed() + + async def check_output(self, check): + self.calls.append(("output", check.output_content)) + return ContentAllowed() + + +@asynccontextmanager +async def proxy(api_key, checker, dispatches): + async with httpx.AsyncClient(timeout=60) as upstream: + transport = ProviderTransport(UpstreamSettings(base_url=BASE_URL, mount_path="/v1"), upstream) + + async def buffered(request): + dispatches.append(request) + return await transport.dispatch(request, require_identity_response=True) + + async def streaming(request): + dispatches.append(request) + return await transport.stream_dispatch(request, require_identity_response=True) + + app = FastAPI() + app.include_router( + create_http_proxy_router( + operations=( + create_guarded_http_operation( + ENDPOINT, + OPENAI_ERROR_MAPPING, + stream_dispatch=streaming, + max_stream_event_bytes=1024 * 1024, + max_pending_stream_bytes=1024 * 1024, + ), + ), + checker=checker, + dispatch=buffered, + render_outcome=OPENAI_ERROR_MAPPING.renderer, + ) + ) + async with httpx.AsyncClient( + transport=httpx.ASGITransport(app=app), + base_url="http://recorded-proxy.test", + headers={"authorization": f"Bearer {api_key}"}, + ) as client: + yield client + + +async def capture(api_key, request): + async with httpx.AsyncClient(timeout=60) as client: + return await client.post( + BASE_URL + "/responses", + json=request, + headers={"authorization": f"Bearer {api_key}", "accept-encoding": "identity"}, + ) + + +async def events(wire): + async def chunks(): + yield wire + + return [event async for event in iter_sse_events(chunks(), max_event_bytes=1024 * 1024)] + + +async def validate_inventory(upstream, request): + document = export_guard_contract(ENDPOINT) + base_document = export_guard_contract(RESPONSES_ENDPOINT) + ENDPOINT.guarded_request_model.validate_payload(request) + Draft202012Validator(document["request"]).validate(request) + if not request["stream"]: + payload = upstream.json() + assert payload.keys() - base_document["response"]["properties"].keys() == { + "access_programs", + "billing", + "frequency_penalty", + "presence_penalty", + "store", + "tool_usage", + } + assert payload["access_programs"] is None + assert payload["billing"] == {"payer": "developer"} + assert payload["store"] is False + assert payload["frequency_penalty"] == payload["presence_penalty"] == 0 + assert set(payload["tool_usage"]) == {"image_gen", "web_search"} + parsed = ENDPOINT.guarded_response_model.validate_payload(payload) + Draft202012Validator(document["response"]).validate(payload) + return parsed.locate_guarded_message(payload).message.content + + assert ENDPOINT.stream is not None + adapter = ENDPOINT.stream.create_adapter() + validator = Draft202012Validator(document["stream"]) + text = [] + kinds = [] + for event in await events(upstream.content): + if event.data not in (None, b"[DONE]"): + payload = json.loads(event.data) + validator.validate(payload) + assert payload["model"] == MODEL + assert "provider_specific_fields" not in payload + if "response" in payload: + assert payload["response"].keys() - base_document["response"]["properties"].keys() == { + "frequency_penalty", + "presence_penalty", + "store", + "tool_usage", + } + kinds.append(payload["type"]) + classified = adapter.classify_event(event) + if classified.role is StreamEventRole.GUARDED_TEXT: + assert classified.text is not None + text.append(classified.text) + adapter.validate_end_of_stream() + assert set(kinds) == { + "response.created", + "response.in_progress", + "response.output_item.added", + "response.output_item.done", + "response.content_part.added", + "response.content_part.done", + "response.output_text.delta", + "response.output_text.done", + "response.completed", + } + assert kinds[-1] == "response.completed" + return "".join(text) + + +@pytest.mark.parametrize("stream", [False, True], ids=["buffered", "streaming"]) +async def test_gateway_responses_preserve_policy_and_wire( + nvidia_inference_key, record_mode, recorded_cassette_path, stream +): + request = responses_request(stream) + upstream = await capture(nvidia_inference_key, request) + assert upstream.status_code == 200 + if record_mode != "none": + return + + assert cassette_request_jsons(recorded_cassette_path) == [request] + text = await validate_inventory(upstream, request) + assert text + checker = Checker() + dispatches = [] + async with proxy(nvidia_inference_key, checker, dispatches) as client: + response = await client.post("/v1/responses", json=request) + assert response.status_code == upstream.status_code + assert response.content == upstream.content + assert len(dispatches) == 1 + assert json.loads(dispatches[0].body) == request + assert checker.calls[0] == ("input", INPUT) + assert "".join(value for kind, value in checker.calls if kind == "output") == text diff --git a/tests/recorded/test_responses_sanitization.py b/tests/recorded/test_responses_sanitization.py new file mode 100644 index 0000000000..06229e8403 --- /dev/null +++ b/tests/recorded/test_responses_sanitization.py @@ -0,0 +1,68 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Protect relationships in sanitized Responses recordings, not real provider evidence.""" + +import json + +from tests.recorded.server.experimental.openai.responses_support import sanitize_responses_recording + + +def test_responses_sanitization_preserves_distinct_ids_and_references(): + payloads = [ + {"type": "response.created", "response": {"id": "private-routing-id", "created_at": 123}}, + {"type": "response.output_item.added", "item": {"id": "first-item"}}, + {"type": "response.output_text.delta", "item_id": "first-item", "delta": "original text", "sequence_number": 4}, + {"type": "response.output_item.done", "item": {"id": "other-item"}}, + {"type": "response.completed", "response": {"id": "private-routing-id", "completed_at": 456}}, + ] + response = { + "headers": {"content-type": ["text/event-stream"], "authorization": ["Bearer test-secret"]}, + "body": {"string": "".join(f"event: {p['type']}\ndata: {json.dumps(p)}\n\n" for p in payloads)}, + } + sanitized = sanitize_responses_recording(response) + text = sanitized["body"]["string"] + events = [json.loads(line[6:]) for line in text.splitlines() if line.startswith("data: ")] + assert events[0]["response"]["id"] == events[4]["response"]["id"] + assert events[1]["item"]["id"] == events[2]["item_id"] + assert events[1]["item"]["id"] != events[3]["item"]["id"] + assert events[2]["delta"] == "original text" + assert events[2]["sequence_number"] == 4 + assert events[0]["response"]["created_at"] == events[4]["response"]["completed_at"] == 0 + assert [line for line in text.splitlines() if line.startswith("event:")] == [ + f"event: {p['type']}" for p in payloads + ] + assert not any(value in text for value in ("private-routing-id", "first-item", "other-item")) + assert "authorization" not in sanitized["headers"] + assert "private-routing-id" in response["body"]["string"] + + +def test_responses_sanitization_preserves_unknown_fields_and_disabled_content(): + body = { + "id": "routing-id", + "created_at": 123, + "output": [{"id": "item", "type": "message", "encrypted_content": "opaque-provider-state"}], + "new_field": {"text": "do not hide"}, + "api_key": "secret", + } + sanitized = sanitize_responses_recording( + {"headers": {"content-type": ["application/json"]}, "body": {"string": json.dumps(body).encode()}} + ) + payload = json.loads(sanitized["body"]["string"]) + assert payload["new_field"] == body["new_field"] + assert payload["output"][0]["type"] == "message" + assert payload["api_key"] == "[REDACTED]" + assert payload["output"][0]["id"] != "item" + assert payload["output"][0]["encrypted_content"] == "[REDACTED_ENCRYPTED_CONTENT]" diff --git a/tests/server/experimental/test_litellm_responses_dialect.py b/tests/server/experimental/test_litellm_responses_dialect.py new file mode 100644 index 0000000000..f29aa16db5 --- /dev/null +++ b/tests/server/experimental/test_litellm_responses_dialect.py @@ -0,0 +1,162 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Verify explicit dialect policy with synthetic cases, not provider recordings.""" + +import json +from copy import deepcopy +from dataclasses import replace +from pathlib import Path + +import pytest +import yaml +from jsonschema import Draft202012Validator +from pydantic import ValidationError + +from nemoguardrails.server.experimental.provider.contract_export import export_guard_contract +from nemoguardrails.server.experimental.provider.projection_policy import EXTENSION +from nemoguardrails.server.experimental.provider.stream import UnsupportedProviderStream +from nemoguardrails.server.experimental.providers.openai.dialects.litellm.responses import ( + LITELLM_RESPONSES_ENDPOINT, + STREAM_RULES, + LiteLLMLifecycleResponse, +) +from nemoguardrails.server.experimental.providers.openai.responses.endpoint import RESPONSES_ENDPOINT +from nemoguardrails.server.experimental.providers.openai.responses.stream_classifier import STREAM_RULES as BASE_RULES +from tests.server.experimental.test_openai_responses_projections import response_payload +from tests.server.experimental.test_openai_responses_streaming_projections import event + +METADATA = { + "access_programs": None, + "billing": {"payer": "developer"}, + "frequency_penalty": 0.0, + "presence_penalty": 0.0, + "store": False, + "tool_usage": {"web_search": {"num_requests": 0}}, +} + + +def test_dialect_is_explicit_and_reuses_request_policy_and_lifecycle_hooks(): + endpoint = LITELLM_RESPONSES_ENDPOINT + assert endpoint.guarded_request_model is RESPONSES_ENDPOINT.guarded_request_model + assert endpoint.stream.hooks_factory is RESPONSES_ENDPOINT.stream.hooks_factory + assert endpoint.operation_paths == RESPONSES_ENDPOINT.operation_paths + assert len(STREAM_RULES) == len(BASE_RULES) == 15 + for base, dialect in zip(BASE_RULES, STREAM_RULES): + assert replace(dialect, model=base.model) == base + assert issubclass(dialect.model, base.model) + + +@pytest.mark.parametrize("name,value", METADATA.items()) +def test_reviewed_metadata_is_response_only_and_preserves_the_original_payload(name, value): + payload = {**response_payload(), name: value} + original = deepcopy(payload) + with pytest.raises(ValidationError): + RESPONSES_ENDPOINT.guarded_response_model.validate_payload(payload) + projection = LITELLM_RESPONSES_ENDPOINT.guarded_response_model.validate_payload(payload) + assert projection.locate_guarded_message(payload).message.content == "answer" + assert payload == original + LiteLLMLifecycleResponse.model_validate(payload) + exported = export_guard_contract(LITELLM_RESPONSES_ENDPOINT) + assert exported["response"]["properties"][name][EXTENSION]["classification"] == "opaque" + Draft202012Validator(exported["response"]).validate(payload) + + +@pytest.mark.parametrize( + "updates", + [{"guardrails": []}, {"fallbacks": []}, {"billing": {}}, {"input": []}, {"tools": []}, {"instructions": "hidden"}], +) +def test_dialect_does_not_enable_gateway_request_controls_or_extra_content(updates): + with pytest.raises(ValidationError): + LITELLM_RESPONSES_ENDPOINT.guarded_request_model.validate_payload({"input": "hello", **updates}) + + +@pytest.mark.parametrize( + "updates", + [{"provider_specific_fields": {}}, {"future_text": "hidden"}, {"output_text": "hidden"}, {"Model": "alias"}], +) +def test_dialect_rejects_unreviewed_response_fields_and_duplicate_text(updates): + with pytest.raises(ValidationError): + LITELLM_RESPONSES_ENDPOINT.guarded_response_model.validate_payload( + {**response_payload(), **METADATA, **updates} + ) + + +@pytest.mark.parametrize("shape", [rule.shape for rule in STREAM_RULES]) +def test_every_rebound_event_retains_role_text_and_export_consistency(shape): + base_rule = next(rule for rule in BASE_RULES if rule.shape == shape) + native = shape.split(":", 1)[0] + payload = response_payload() + if shape.endswith(":reasoning"): + fields = {"item": payload["output"][0]} + elif shape.endswith(":message"): + fields = {"item": payload["output"][1]} + elif native.startswith("response.content_part."): + fields = {"part": payload["output"][1]["content"][0]} + elif native == "response.output_text.delta": + fields = {"delta": "answer"} + elif native == "response.output_text.done": + fields = {"text": "answer"} + elif native == "error": + fields = {"message": "provider failure"} + else: + fields = {"response": payload} + base = RESPONSES_ENDPOINT.stream.classifier.classify_event(event(native, **fields)) + if "response" in fields: + fields["response"].update(METADATA) + original = deepcopy(fields) + wire = event(native, **fields, model="gateway-model") + with pytest.raises(UnsupportedProviderStream): + RESPONSES_ENDPOINT.stream.classifier.classify_event(wire) + classified = LITELLM_RESPONSES_ENDPOINT.stream.classifier.classify_event(wire) + assert classified.role == base.role == base_rule.role + assert classified.text == base.text + assert fields == original + Draft202012Validator(export_guard_contract(LITELLM_RESPONSES_ENDPOINT)["stream"]).validate(json.loads(wire.data)) + + +@pytest.mark.parametrize( + "native,fields", + [ + ("response.output_text.delta", {"delta": "answer", "provider_specific_fields": {}}), + ("response.output_text.delta", {"delta": "answer", "future_text": "hidden"}), + ("response.output_text.delta", {"delta": "answer", "Model": "alias"}), + ("response.output_text.delta", {"delta": "answer", "logprobs": [{"token": "hidden"}]}), + ("response.reasoning_summary_text.delta", {"delta": "hidden"}), + ("response.output_item.done", {"item": {"type": "reasoning", "summary": [{"text": "hidden"}]}}), + ("error", {"error": {"message": "nested diagnostic", "code": "provider_error"}}), + ], +) +def test_model_annotation_does_not_relax_content_or_authorize_new_event_shapes(native, fields): + with pytest.raises(UnsupportedProviderStream): + LITELLM_RESPONSES_ENDPOINT.stream.classifier.classify_event(event(native, **fields, model="gateway-model")) + + +def test_bound_dialect_retains_identity_and_native_completion_checks(): + adapter = LITELLM_RESPONSES_ENDPOINT.stream.create_adapter() + adapter.classify_event(event("response.output_text.delta", delta="answer", item_id="first", model="gateway-model")) + with pytest.raises(UnsupportedProviderStream): + adapter.classify_event( + event("response.output_text.done", text="answer", item_id="different", model="gateway-model") + ) + with pytest.raises(UnsupportedProviderStream): + LITELLM_RESPONSES_ENDPOINT.stream.create_adapter().validate_end_of_stream() + + +def test_dialect_export_matches_generated_artifact_and_format(): + root = Path(__file__).parents[3] / "nemoguardrails/server/experimental/contracts" + document = export_guard_contract(LITELLM_RESPONSES_ENDPOINT) + assert document == yaml.safe_load((root / "openai/_generated/litellm-responses.guard.yaml").read_text()) + Draft202012Validator(json.loads((root / "guard-contract.schema.json").read_text())).validate(document) From c567f7a0dba568131a2b1c45ffddc3267328ab0e Mon Sep 17 00:00:00 2001 From: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> Date: Thu, 15 Oct 2026 10:31:37 +0200 Subject: [PATCH 2/6] test(server): preserve Responses content with shared recording sanitizer Align the LiteLLM Responses recordings with phase-d-final-20261013-d2-recorded. Remove the extra Responses sanitizer rather than redacting encrypted content or remapping nested identifiers. Keep shared credential filtering and metadata normalization unchanged. Refresh both genuine gateway cassettes with the existing greeting request, preserving response content. Add buffered/SSE regressions for content preservation by the shared sanitizer and update the recording documentation. The removed helper and its tests remain recoverable from e971db1ff9. Keep the committed mixin-based dialect, runtime bindings and generated contract unchanged. This is a recording-only correction within the separate LiteLLM Responses follow-up PR scope. Validation: 16 offline recorded/helper tests and 41 dialect tests passed; pre-commit and credential scans passed. Nothing pushed. Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> --- .../openai/dialects/litellm/responses.md | 10 ++- tests/recorded/README.md | 6 +- ...es_preserve_policy_and_wire[buffered].yaml | 10 +-- ...s_preserve_policy_and_wire[streaming].yaml | 46 +++++------ .../experimental/openai/responses_support.py | 79 ------------------- .../openai/test_litellm_responses.py | 7 +- tests/recorded/test_recorded_helpers.py | 35 ++++++++ tests/recorded/test_responses_sanitization.py | 68 ---------------- 8 files changed, 73 insertions(+), 188 deletions(-) delete mode 100644 tests/recorded/server/experimental/openai/responses_support.py delete mode 100644 tests/recorded/test_responses_sanitization.py diff --git a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md index 5f4417d00a..ec99578af6 100644 --- a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md +++ b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md @@ -89,10 +89,12 @@ channels, and other backend-specific fields remain unsupported and fail closed. Flat diagnostic errors and other inherited events have synthetic unit coverage; they are not claimed as recorded observations. -The recording sanitizer removes credentials, routing/item identifiers, timestamps -and encrypted provider state. It preserves distinct item IDs and their references, -text, event order, sequence numbers and field presence. Privacy redaction is not -a policy repair: unsupported fields must not be deleted to make replay pass. +Recordings use the same shared sanitizer as the Chat recordings: credentials are +removed and selected response metadata is normalized. There is no Responses-only +content redaction. Assistant text, encrypted content, nested item identifiers, +event order, sequence numbers and field presence are retained, subject to the +shared credential filters. Sanitization is not a policy repair: unsupported +fields must not be deleted to make replay pass. Exact passthrough comparisons are against the sanitized replay bytes, not a claim that the cassette preserves original HTTP chunk boundaries or all original bytes. diff --git a/tests/recorded/README.md b/tests/recorded/README.md index fee44c3a68..dece7a19c6 100644 --- a/tests/recorded/README.md +++ b/tests/recorded/README.md @@ -133,9 +133,9 @@ The separately reviewed Responses recordings explicitly select the Refresh only `tests/recorded/server/experimental/openai/test_litellm_responses.py` with `RECORDED_REQUIRED_KEYS=NVIDIA_INFERENCE_KEY`. Capture stores the complete sanitized response before checking policy; the subsequent offline replay is the -acceptance gate. The local sanitizer preserves distinct item-ID relationships -and removes encrypted provider state. These two cassettes establish neither Chat -bridge nor arbitrary-backend compatibility. +acceptance gate. Responses uses the same shared sanitizer as Chat, with no extra +content redaction; encrypted content and nested item-ID relationships are retained. +These two cassettes establish neither Chat bridge nor arbitrary-backend compatibility. These six cassettes predate the dialect: their provider payloads are unchanged. Two buffered successes contain message `provider_specific_fields: {refusal: null}` diff --git a/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[buffered].yaml b/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[buffered].yaml index ab2ceca76e..e50b637643 100644 --- a/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[buffered].yaml +++ b/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[buffered].yaml @@ -38,7 +38,7 @@ interactions: body: parsed_body: id: '[RECORDED_RESPONSE_ID]' - created_at: 0 + created_at: 1792052980 error: null incomplete_details: null instructions: null @@ -46,12 +46,12 @@ interactions: model: openai/openai/gpt-5-nano object: response output: - - id: recorded_id_1 + - id: encitem_bGl0ZWxsbTptb2RlbF9pZDpvcGVuYWkvb3BlbmFpL2dwdC01LW5hbm87aXRlbV9pZDpyc18wMGI5Nzk5MWYzYzY4N2U1MDE2YWQwOGVmNGUwYTg4N2QxYWJiYmYxYzM2Yjc3ZjExNA== summary: [] type: reasoning content: [] - encrypted_content: '[REDACTED_ENCRYPTED_CONTENT]' - - id: recorded_id_2 + encrypted_content: litellm_enc:bW9kZWxfaWQ6b3BlbmFpL29wZW5haS9ncHQtNS1uYW5v;gAAAAABq0I71yg-QEK67UCTTRSxv5rUFezcpi9Hc2o1rrbfdk5ui5HtuFV8NqyuPLiHR_3lP_WxBrSFTr5GYQTCHdf4uRic8mxcdl5GgNiPNMpZSgEoMQlTKXS8Fh-nseqGECfBc6fN-AtCzptqlCwWaiSRPG7W1StVBkbuyvBztPVc6o3caW6ZtV2JH5Gy1IbsB1KQwRelL9r8R7qzzaDAn9MmbiXEJD0TeTlrxtCkYoYbTXNYsfUHxdOOeGFNDmoshW4q0MzZczZmH4SjV6cFHWpTNAsb7gOZx1tHVgYUgZvDu6CT_PzEY_ZNlhnfrmSJYO8KhlX1FdnEJvRKMbhu1t5fkr0c1MGhhWm5LaakHHGoIVlU5gUIxg5--bcgXk6QZzpr2sKl0oJ0S0yV1io4ztIl64kqSzjyASeQ5EB584dOGP7jKJD1Bcru26CrriX8c7WlFCEBfP5Q5OQUv0YPap5vEZzyF37xjPijA8lj36NsEsOpXM1oeZUsbd3IpdU6R2jh_YvrU2Rzp79DaDRXEiuitNoJxWYG5qvzDt2hqcsbAzV2JWxPp7rO-vX-0RWMHe8bc2mXs4cjTfbVFOGpUyvVeOFGetu4TDK5k8c2ANyA8WQBjSYunu7DNrv4z9aZcdB3cVTqc5Y3GrAV3-gTxNblTZrbJmkdj9RB17jDnLadA6l2uVHjbLfFSsOJdtU3j3pYyhJA-78N91vjBQdygNNoGzRhJcELEItq08-zaKlLq1Rizen1J7FfX8KFLH7rD8DOZz7wkoE4VdxSGkcRdAOs_AX_drY6gi0WFKJAQCf1PFOcaQQS1M7lGwaG0aGsEQ1OrY2BjIT5MdWPIAv_RnEgRvwmZ6A3Agc_e_uB7_Lv0uNAyInAavNeAJhdTxzQRCKFEjqp-sN-o24fu7ZJynyiPQtyL8HrsCcCu-2jbIKqTRmt8nf06upmV79U2ZHltYG0tunvYOtLzkT_yZD0tazFHPEW7t9EgN2XURg8ppEPf1U-9x0RtklhKa7bOZ98XyI1HGteGNPvNzb_SZXA0F4jFixRtJaJ4K6XMy0AqqwvOXE4jJ4BxIAmqcPokBtKfVSt3NBtu4q4EX8R-e1sukYrXdbAiUcd-gLtOxu8rZDxEuzy6jDP8YUkMPV8ojum2mQo9ZJPyIiruh2Lxuq_JH2ByA_iHgimeFNpaz7Y6xX-TH09sNH5asK5VT_Bh00HAuuJO3F8OpFaq5kdPGoTcUBhF6Yxyiw79OUM39KCM_rAW7gGEMj7QzXXWlSWnPTC09ubdz4S6-3fUkYGLIkyzNZ0odoW1SBiMZM_1MrOw2BgpjwdE_H6iI_xuq1Kg1O1cD4fAUFIKxq7_vK0zPfLiTWKNEzSksxtW5tYKcQF7YcHMo5ZLSWOAEWEbWnxB8QX8tC-SoBgex-9hydJ4zh7gWSsrWuRogbHGTYSj2j4E9HakP6FGLAA4D6OzmT7iDEMu0vC51uXY5EHvjMVvNJejbnNWONdobQBA58b2IUVCpgCEctycR3ahYXXM4WpnxIY2z4DotWgKx3H16NIy7tB0D7GVEXyODQhMm1DRvmfaUPHgeU3e40anij5fDAmcyx36EqGK0ahBVMdyqI5OiqwgEZOrCoN-h2e3hU-lSnzDBducMUsljkU= + - id: msg_00b97991f3c687e5016ad08ef4fcbc87d1802a5fb912d3f67d content: - annotations: [] text: Hello. @@ -102,7 +102,7 @@ interactions: background: false billing: payer: developer - completed_at: 0 + completed_at: 1792052981 frequency_penalty: 0.0 max_tool_calls: null moderation: null diff --git a/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[streaming].yaml b/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[streaming].yaml index b005b8af66..b64ba17278 100644 --- a/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[streaming].yaml +++ b/tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/test_gateway_responses_preserve_policy_and_wire[streaming].yaml @@ -41,8 +41,8 @@ interactions: parsed_body: - type: response.created response: - id: recorded_id_0 - created_at: 0 + id: resp_SoSS-EM2MI313Kq8bM5fhzlbbZNcn20PTpRDoLj_9CgNJDxCJ6keDMnIMzuGw_4KZkXZFS4zSQ1sLYXnY_uPwBPOv6ajUwwyu2ReRCEK1rKfxgP9E_kuGEqvcU46k4mo6yA9y7SpZiaaHyX-ZLsUMyoOlVx2dq0r6uRBo6mFt3esLh3mDe9u0MXNVVKfnGL2NAzAnS7A5aKj3HY2VHitw-StlkJP8oit_dJyz3N_84fShvN6GN9ytMMqvYN-v7fDnEuu1s9wek77BC3m5lJsH0y3Np5fwbyrDKal6NgzTITeQYvVjZ_99mrmZMqcaPhuV26Vv-CMbrHgxI8fxKAApKEZ6PWiNlSbEQj7OzTqDbnbgo_qOEobta7EsudQjN5wXtzvOFRbG7f09_XI7SxWRfiHuYvBrPahC98jbLXCAwq6ofY= + created_at: 1792052982 metadata: {} model: gpt-5-nano-2025-08-07 object: response @@ -88,8 +88,8 @@ interactions: model: openai/openai/gpt-5-nano - type: response.in_progress response: - id: recorded_id_0 - created_at: 0 + id: resp_SoSS-EM2MI313Kq8bM5fhzlbbZNcn20PTpRDoLj_9CgNJDxCJ6keDMnIMzuGw_4KZkXZFS4zSQ1sLYXnY_uPwBPOv6ajUwwyu2ReRCEK1rKfxgP9E_kuGEqvcU46k4mo6yA9y7SpZiaaHyX-ZLsUMyoOlVx2dq0r6uRBo6mFt3esLh3mDe9u0MXNVVKfnGL2NAzAnS7A5aKj3HY2VHitw-StlkJP8oit_dJyz3N_84fShvN6GN9ytMMqvYN-v7fDnEuu1s9wek77BC3m5lJsH0y3Np5fwbyrDKal6NgzTITeQYvVjZ_99mrmZMqcaPhuV26Vv-CMbrHgxI8fxKAApKEZ6PWiNlSbEQj7OzTqDbnbgo_qOEobta7EsudQjN5wXtzvOFRbG7f09_XI7SxWRfiHuYvBrPahC98jbLXCAwq6ofY= + created_at: 1792052982 metadata: {} model: gpt-5-nano-2025-08-07 object: response @@ -136,10 +136,10 @@ interactions: - type: response.output_item.added output_index: 0 item: - id: recorded_id_1 + id: rs_07a564c4e2337089016ad08ef64ac887d1aa52acdf7c08e73b type: reasoning content: [] - encrypted_content: '[REDACTED_ENCRYPTED_CONTENT]' + encrypted_content: litellm_enc:bW9kZWxfaWQ6b3BlbmFpL29wZW5haS9ncHQtNS1uYW5v;gAAAAABq0I7215IoHaKySxQ_3LvcowiS6wczpRKov6OBWAmaYJXrS0l1fR1juXyzE0ta13Lv4gBdmfB1neTYTLm4tEcfCV2xlCSLLep6RVy9ZDjLAyIoXZLnuT9A7W7aloUxeWk9K-3FdWRjhvkK3iqSjDFWwZef5K_z6-nGR8MA8lrcYaGVU3HFEpydh6Xx8oQfqIaHErCpYYEeow1cSI61cREz5ibKbGUXo6Gxi5OZ2ITm4zoTMCjQv5jPzo4CSoJYhOMgdMunMnWlxFsIWAClwWtUN1GjZJANYlHU_reB9lYuMBZnEvxRTwdJImij-SpiP6obvrT6ovpovfmxBxtOtlPMUqyIZUaqXwtYJbbTH4sCdD73LJLbHSeQ9eoPEJeksFiFR3KnhUZcDTbQzCGhb-J1E-4nxgq2hQ1e3Y7bMkx0dRCK99h1VtcNDLbCac9aF_tBKujQjXIQWvPomsRGHvgIZqvd5POq539XMWnW3PAOVKDb1Tj1nUVZTosAomjdspzu8j2KFs9DSZIyCn09fuqox6BGdBVe3asWzexOpk4ra2Cp-K8nq3nsjziAeWlXnIdnx-8SpVLXtBYIOh3wx95S7XbvtXsc9PrPRXk21zyAYkiNlmZuKip_h8MifdOEy-YaL0PsX2BOM2JEHI0rqehYzP67wMrGsX9_xf6QK9kLkFpbMic-h2I-63hMuNCwqGRQllkGSnVL7kxWkGMkZjGB8RdDqCN3uS5vwORU5IdywzRVCpAjF_F6Wf1jxWbeXuK-Pr62w3mivYceXA5hsd9IYgUWuIDbMvE_q7Tc_3TQ3vSuP-5PVP2dp3JVh3ho7VyH_nv8KarD2_aqeGsKh9kDR5Zn5iiAHUh-8X9PbBILOM34dWJlD5REN5baxhrH3AOK9q5arWPkAly023jmP5ITEkcwnA93fj135S-XfMsKs568kMbz_KRPfnoYlsaq4zEYlFPbcLTIJAZkTY_h3xHhA1wBt2sRqCSySzK96Smcmje9TGoMK1WhfEjwK4IaOCYBJBCYMMIwxB89-bY-Wsv_0a79o_gx0SdnbieEEjRMwMz_-yRcNydPAPfOyNDGb-ChFkRhlgJxpLfNIw-gTM5W8UtPbmyImv0-JcG2InV8oaHtrlNcdHe4RQCQWLgjNyZvSKpycAuEHJ4OKhK0tcaM2qZ1vfw4O_6kmtdVVTDxf-iu4lnAyTnX_mk0bkGOsq7Xztwj summary: [] sequence_number: 2 model: openai/openai/gpt-5-nano @@ -147,16 +147,16 @@ interactions: output_index: 0 sequence_number: 3 item: - id: recorded_id_1 + id: rs_07a564c4e2337089016ad08ef64ac887d1aa52acdf7c08e73b type: reasoning content: [] - encrypted_content: '[REDACTED_ENCRYPTED_CONTENT]' + encrypted_content: litellm_enc:bW9kZWxfaWQ6b3BlbmFpL29wZW5haS9ncHQtNS1uYW5v;gAAAAABq0I72nJtVAvxlORNN-HM1TMXwCezYge_75H4fhD_lnvN8X2pYAGQlAF4mMjxMSmGm2BUA_qevAG0tleBGeCVLD498CbtLCibMSEm2Qx16XS5tSyw6x16OUMorCrves6-pLkTon_4DvQGoMvm51gLCMxdAhGuWYY7KCTEfuiTx3IVRGQvX0oALzT-Y8Ha1WYu-oGJAcX3v4fKwhcn7AZ3fTJZgzajfuYaEh78-CuwpccNEv_C2MZDH2-QKy8k3LA8VbOTnuEF9frtm9BnxtixHRN0pEDaZbjoFRVccGmXm5x4nMkeeLZky5LkYQoBI07LWIOd207Mm-0dAW5zeO-DbYlcMWgQaTPw1wc0JrsGK7Xc69RVQcN3RiXBXDuE4oyERd_m_3ksf1L9F0hoqBu-Gx1F6hWYWsKQt8RHk3i1Zlm8zrfrJ1BguJJB4BuzpSnnMaeQf3Rb7K8R5-Ab1JqRwYc6jg7oHwk04RmHQefL3lCXsnW_Z6nxh8PFWPodo0L1m5byFQN-yddIgzUY5hghFxZtmQWaTamq5ExomgidJ5mH-c2AKYJYdG6uzTydm7peoXeLRHC8GeTixD1w1ykKDAV4_BsXa4otLPv0rnULrf965JYM-45pXmItPnHTGG6IXBytPr3sh6diq59lIv5jV9S3j3gU9rShU1DETRslDhhzDOz8yIc7v3gF1vLYdzoZE7OPoNBxuwmAxRkv8v_QDT_xapT9cnif81h5XyBj4ieoPDbYQBoq2O0zhzn_bLUSjd-d_sp3axOGt82Cv681j3J8DwCC8Lt-_Tw8BP7CDIQ0YV6KZLNNOQAyE7jMgeiwDtMOswZbNOo07tAe7NQvLlHTn_dBBRr8MjI8lj2XTVBwgMIJJRhSbMilHVnDKkeCPwPrRU9aTlWxGNBVR_bkDAan_AsOG5ACEIshaPImnHPXaO5y21-jSCG9s6QjiMj2jr5E_kh1u4hfzbwCV3tEC2dvhuoYys9pNf5WCkVMVOqOd8tpdkH6PnVtbUINnK-CZmLsNg7p4TPMbPwOqwF0de4tT_aLsbTUGwi6dvk2T2bGRW15bOK9QZYLeGpTKuz1yPdNFG3-ZjLf-HtHRtHtsQKZ_oy-tW800NNbn5zCsYEz8pzDD-khU8Q5NNOAop6rdVzpLyBlkPoweIc7OVrBQa9vu11dSLra-c3oZr0_F0yVqRbyfSkQLaTJC_GUHMUTGFbTnBu93qJVkSxG3iPEZkSaLrizGElxkvByVqjT7wgiUMUHRneN50WfEOs3PyKrurBVbvLx6nh5dw9QGpd5oiA1xRinN7b71pup-TJAeckLcvNdix8oUqNzgo9ZrH9AEmB10Kom8Cw-ynTcEfWTvQZEQxvoTBrcvzg20OG6rQcej-L_VS8xXw7mrJbdEk6jfWHwwoZrPLBvXboplSuwamMsDkZeAbYH5ONQJsFmFf9yhuQ-pPdqKQ78sWa1xOKbALgjLY39OOHzFBM2sabGqMk7TkpkW73HeATIanHBfS9W07dJKgjBnt1Y1d8oooF6dbQ9s25Hlm0Zrv9C9Cu4f-PhEB7aKCtg1UoPiNfoeV5mbjOTxOo6D00tup1RVt43OxlURPVE_hR2rt_vOJjARjxOKOdqE4JXbNs7sLnF0IHdKQ8E= summary: [] model: openai/openai/gpt-5-nano - type: response.output_item.added output_index: 1 item: - id: recorded_id_2 + id: msg_07a564c4e2337089016ad08ef65f3c87d18c862c13f7ea43c2 type: message status: in_progress content: [] @@ -164,7 +164,7 @@ interactions: sequence_number: 4 model: openai/openai/gpt-5-nano - type: response.content_part.added - item_id: recorded_id_2 + item_id: msg_07a564c4e2337089016ad08ef65f3c87d18c862c13f7ea43c2 output_index: 1 content_index: 0 part: @@ -175,25 +175,25 @@ interactions: sequence_number: 5 model: openai/openai/gpt-5-nano - type: response.output_text.delta - item_id: recorded_id_2 + item_id: msg_07a564c4e2337089016ad08ef65f3c87d18c862c13f7ea43c2 output_index: 1 content_index: 0 delta: Hello logprobs: [] - obfuscation: k5gxEnU5U0A + obfuscation: bj28IlODqhb sequence_number: 6 model: openai/openai/gpt-5-nano - type: response.output_text.delta - item_id: recorded_id_2 + item_id: msg_07a564c4e2337089016ad08ef65f3c87d18c862c13f7ea43c2 output_index: 1 content_index: 0 delta: . logprobs: [] - obfuscation: 9p6RkErDA9Rqkiy + obfuscation: 21EZpykDzlXEYs2 sequence_number: 7 model: openai/openai/gpt-5-nano - type: response.output_text.done - item_id: recorded_id_2 + item_id: msg_07a564c4e2337089016ad08ef65f3c87d18c862c13f7ea43c2 output_index: 1 content_index: 0 text: Hello. @@ -201,7 +201,7 @@ interactions: sequence_number: 8 model: openai/openai/gpt-5-nano - type: response.content_part.done - item_id: recorded_id_2 + item_id: msg_07a564c4e2337089016ad08ef65f3c87d18c862c13f7ea43c2 output_index: 1 content_index: 0 part: @@ -215,7 +215,7 @@ interactions: output_index: 1 sequence_number: 10 item: - id: recorded_id_2 + id: msg_07a564c4e2337089016ad08ef65f3c87d18c862c13f7ea43c2 type: message status: completed content: @@ -227,18 +227,18 @@ interactions: model: openai/openai/gpt-5-nano - type: response.completed response: - id: recorded_id_0 - created_at: 0 + id: resp_SoSS-EM2MI313Kq8bM5fhzlbbZNcn20PTpRDoLj_9CgNJDxCJ6keDMnIMzuGw_4KZkXZFS4zSQ1sLYXnY_uPwBPOv6ajUwwyu2ReRCEK1rKfxgP9E_kuGEqvcU46k4mo6yA9y7SpZiaaHyX-ZLsUMyoOlVx2dq0r6uRBo6mFt3esLh3mDe9u0MXNVVKfnGL2NAzAnS7A5aKj3HY2VHitw-StlkJP8oit_dJyz3N_84fShvN6GN9ytMMqvYN-v7fDnEuu1s9wek77BC3m5lJsH0y3Np5fwbyrDKal6NgzTITeQYvVjZ_99mrmZMqcaPhuV26Vv-CMbrHgxI8fxKAApKEZ6PWiNlSbEQj7OzTqDbnbgo_qOEobta7EsudQjN5wXtzvOFRbG7f09_XI7SxWRfiHuYvBrPahC98jbLXCAwq6ofY= + created_at: 1792052982 metadata: {} model: gpt-5-nano-2025-08-07 object: response output: - - id: recorded_id_3 + - id: encitem_bGl0ZWxsbTptb2RlbF9pZDpvcGVuYWkvb3BlbmFpL2dwdC01LW5hbm87aXRlbV9pZDpyc18wN2E1NjRjNGUyMzM3MDg5MDE2YWQwOGVmNjRhYzg4N2QxYWE1MmFjZGY3YzA4ZTczYg== summary: [] type: reasoning content: [] - encrypted_content: '[REDACTED_ENCRYPTED_CONTENT]' - - id: recorded_id_2 + encrypted_content: litellm_enc:bW9kZWxfaWQ6b3BlbmFpL29wZW5haS9ncHQtNS1uYW5v;gAAAAABq0I72zysi-6j_rjlrjCxbklwEzLP2ILzeQAZaN2OtWXa84risdrn9RYyzhHTXbdyRRyWxFELgsEujouhw6FnL-juidX6fdPXmeYKCcUQ20EGKo1H8c2TjAQuZXTwG8b7zQKjYxlf58yjrLDaaa0WvmYJmkV1FQvzW8pRqAmtDW-FVFQPz5j9JItvzqqdRrJAAv_mPYLeNVuFsMZCDlmduP9brgD8q2zfnHQv_S-McWig_UMLvfsG1ubBrZYNa3U8Zg66wrzKuZDSgc7UdzorXSRhLmkyMtiwYDx0NCHU8J-rZ8R0Scifos9chtv0qMzZB5BnWli6S2I_3trqLY1GkWzDIkrQisF-0kpynmouMArqR3qPgkj35jxgnCAeXxfHrASQKE4fpGnTwccH9bJA2YxMvAr_L2aqHHKoPbcfIAGHUC7zxF0IC7HuuirBgmY7S9I_SVOObQpNAJOUsaXBIeyd80GoPJEKwfKcpx3Rtjw8JZ2dv39-9OhTLgi75UhypTPIPXjiiNXubBNCTYfvihw9vueLOtGEIKrcY8UH6Uy3xkEcKQiYv1jdr_DX0PofM6rkef5-F_X2_36luL6FMlmykSa0Qvk-r8-DpAbJqGH4KabeH8bMBqIesPCFjQx_QHHCYkXfz-FwM9agJegIQ19Gz6yl307H0nfIdfXtObncIX5lhcwtToGILhM0ilSQLC4eEshAXyEdCOJjgzuwx562Lj-G8MyGsb38ZwcUNFSWn2FzF2ea7CIlJ0EeJulJELt80cbDbOww-Jyn6Uu9BiTWr-znJf4_KJM7bAjlLC1TCefPdhcgXEzl0CMUtVXBaO8bqoqDktTRZjjCDwpU6-QELSaXd1pghKzjk1LmcUWO-ICrHW5vD3n3-PD_ri8JQeoB0yUwNy5RpLoYjC7WpIlUbkXguCFzRRNPsgHwfe-s0E2SJHeLwhDHKNOnWPZr7z-sICb9piaexl05gtTC0LSna5wrij8jpIvFs71mOx78Aj_nWZ1gUNWgm3HBLcFKD_22_PmElbYVTo16B7tpERUIy50D2qpvF5A9x8G-GqeveFfQef3-MBOZy1H7Mw4Tv6A2voL8V3XeYKQjcW-CdMvLJDoODbo7mpyokg_jLkTAvCch7xwrojTw2b06y3QiZX8ynjGoTlRgp_MG4UQTb7BAzmLEk_J85xIP7GyISrbB5dFdGbpVh0qUMxp9uZiaSJVelvE1NsTv30SMNYCYxk-Ges7Uca16OfaQV36CXcNAjJaO6rZzkUOthO9ui7rzOfNC3dEXQiwf9-2PPwAVbZjSekJk1Q61Swt69AUmOvwErBu5ESeCmZP7ZkRcvnSHqB_o3q_Z3dFtOu7i5aTQowyEeLimQOuQp44jqOvn8ftEeab5X0d_EKudPaSw_beEZ6s7n60nKsFKXjSOaNkcQZDIO-26hHwyT_T7CfilpGzn6kbY67Tk8OpWSwCkqA-SZDxLvoWc4hIiJFkYRLBXi1UKmEHv1sfiCHuX_0hLdf1WLbYGvyTeDhbvvaDlz_TP8Wv9hcnYF030k5OBYIV1RkpdWr3IQvXDn8N76GHR2_XjaHVVJ6S2RZeTBOMGkeO7qBuV3IlJCYoVloMCSSYPBPKOAvHbdpXDArzKCb8JF76_4PPo= + - id: msg_07a564c4e2337089016ad08ef65f3c87d18c862c13f7ea43c2 content: - annotations: [] text: Hello. @@ -276,7 +276,7 @@ interactions: cost: 6.312399999999999e-06 store: false background: false - completed_at: 0 + completed_at: 1792052982 frequency_penalty: 0.0 presence_penalty: 0.0 prompt_cache_retention: in_memory diff --git a/tests/recorded/server/experimental/openai/responses_support.py b/tests/recorded/server/experimental/openai/responses_support.py deleted file mode 100644 index 707d7e64d3..0000000000 --- a/tests/recorded/server/experimental/openai/responses_support.py +++ /dev/null @@ -1,79 +0,0 @@ -# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Sanitize Responses evidence without merging distinct stream item identities.""" - -import json -from copy import deepcopy - -from tests.recorded.conftest import before_record_response - - -def sanitize_responses_recording(response): - """Remove routing identifiers while retaining identity equality and event order. - - One mapping spans the complete HTTP body, including all SSE events. Unlike - replacing every id with one sentinel, this preserves mismatches and distinct - items. Text, sequence numbers, output indices and extension fields survive. - The shared sanitizer still owns credential and HTTP header redaction. - """ - response = deepcopy(response) - identities = {} - - def scrub(value): - if isinstance(value, list): - return [scrub(item) for item in value] - if not isinstance(value, dict): - return value - result = {} - for key, nested in value.items(): - if key in {"id", "item_id", "response_id"} and isinstance(nested, str): - result[key] = identities.setdefault(nested, f"recorded_id_{len(identities)}") - elif key in {"created_at", "completed_at"} and nested is not None: - result[key] = 0 - elif key == "encrypted_content" and isinstance(nested, str): - result[key] = "[REDACTED_ENCRYPTED_CONTENT]" - else: - result[key] = scrub(nested) - return result - - body = response.get("body", {}).get("string") - if isinstance(body, (str, bytes)): - text = body.decode() if isinstance(body, bytes) else body - content_type = next( - (value for name, value in response.get("headers", {}).items() if name.lower() == "content-type"), [] - ) - if isinstance(content_type, str): - content_type = [content_type] - if any("text/event-stream" in value.lower() for value in content_type): - lines = [] - for line in text.splitlines(keepends=True): - if line.startswith("data: ") and line.strip() != "data: [DONE]": - try: - payload = json.loads(line[6:]) - except json.JSONDecodeError: - pass - else: - ending = line[len(line.rstrip("\r\n")) :] - line = "data: " + json.dumps(scrub(payload), separators=(",", ":")) + ending - lines.append(line) - text = "".join(lines) - else: - try: - text = json.dumps(scrub(json.loads(text))) - except json.JSONDecodeError: - pass - response["body"]["string"] = text.encode() if isinstance(body, bytes) else text - return before_record_response(response) diff --git a/tests/recorded/server/experimental/openai/test_litellm_responses.py b/tests/recorded/server/experimental/openai/test_litellm_responses.py index 3309f64c7c..3312f2b187 100644 --- a/tests/recorded/server/experimental/openai/test_litellm_responses.py +++ b/tests/recorded/server/experimental/openai/test_litellm_responses.py @@ -44,15 +44,10 @@ from nemoguardrails.server.experimental.providers.openai.errors import OPENAI_ERROR_MAPPING from nemoguardrails.server.experimental.providers.openai.responses.endpoint import RESPONSES_ENDPOINT from tests.recorded.cassette import cassette_request_jsons -from tests.recorded.server.experimental.openai.responses_support import sanitize_responses_recording pytestmark = [ pytest.mark.recorded, - pytest.mark.vcr( - ignore_hosts=["recorded-proxy.test"], - allow_playback_repeats=True, - before_record_response=sanitize_responses_recording, - ), + pytest.mark.vcr(ignore_hosts=["recorded-proxy.test"], allow_playback_repeats=True), pytest.mark.asyncio, ] diff --git a/tests/recorded/test_recorded_helpers.py b/tests/recorded/test_recorded_helpers.py index ee5810ec35..fa77dcd216 100644 --- a/tests/recorded/test_recorded_helpers.py +++ b/tests/recorded/test_recorded_helpers.py @@ -15,10 +15,13 @@ from __future__ import annotations +import json + import pytest from nemoguardrails import LLMRails from tests.recorded.assertions import assert_stream_contract +from tests.recorded.conftest import before_record_response from tests.recorded.rails.public_api.configs import INPUT_RAILS_CONFIG from tests.recorded.rails_config import load_config @@ -37,3 +40,35 @@ def test_load_config_returns_fresh_copy_when_llmrails_mutates_config(): def test_assert_stream_contract_accepts_metadata_text_chunks(): assert assert_stream_contract([{"text": "Hello", "metadata": {"usage": {}}}], expect_multiple=False) == "Hello" + + +@pytest.mark.parametrize("stream", [False, True], ids=["buffered", "streaming"]) +def test_shared_sanitizer_preserves_responses_content_and_nested_identity(stream): + """Content is evidence; only shared credential/metadata filtering applies.""" + payload = { + "id": "response-id", + "output": [ + {"id": "reasoning-id", "type": "reasoning", "summary": [], "encrypted_content": "opaque-state"}, + {"id": "message-id", "type": "message", "content": [{"type": "output_text", "text": "Hello."}]}, + ], + } + if stream: + data = {"type": "response.completed", "response": payload, "sequence_number": 4} + body = f"event: response.completed\ndata: {json.dumps(data)}\n\n" + content_type = "text/event-stream" + else: + body = json.dumps(payload) + content_type = "application/json" + response = before_record_response( + {"headers": {"content-type": [content_type], "authorization": ["Bearer test-secret"]}, "body": {"string": body}} + ) + assert "authorization" not in response["headers"] + if stream: + encoded = next(line[6:] for line in response["body"]["string"].splitlines() if line.startswith("data: ")) + event = json.loads(encoded) + assert event["sequence_number"] == 4 + assert event["response"] == payload + else: + actual = json.loads(response["body"]["string"]) + assert actual["output"] == payload["output"] + assert actual["id"] == "[RECORDED_RESPONSE_ID]" diff --git a/tests/recorded/test_responses_sanitization.py b/tests/recorded/test_responses_sanitization.py deleted file mode 100644 index 06229e8403..0000000000 --- a/tests/recorded/test_responses_sanitization.py +++ /dev/null @@ -1,68 +0,0 @@ -# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Protect relationships in sanitized Responses recordings, not real provider evidence.""" - -import json - -from tests.recorded.server.experimental.openai.responses_support import sanitize_responses_recording - - -def test_responses_sanitization_preserves_distinct_ids_and_references(): - payloads = [ - {"type": "response.created", "response": {"id": "private-routing-id", "created_at": 123}}, - {"type": "response.output_item.added", "item": {"id": "first-item"}}, - {"type": "response.output_text.delta", "item_id": "first-item", "delta": "original text", "sequence_number": 4}, - {"type": "response.output_item.done", "item": {"id": "other-item"}}, - {"type": "response.completed", "response": {"id": "private-routing-id", "completed_at": 456}}, - ] - response = { - "headers": {"content-type": ["text/event-stream"], "authorization": ["Bearer test-secret"]}, - "body": {"string": "".join(f"event: {p['type']}\ndata: {json.dumps(p)}\n\n" for p in payloads)}, - } - sanitized = sanitize_responses_recording(response) - text = sanitized["body"]["string"] - events = [json.loads(line[6:]) for line in text.splitlines() if line.startswith("data: ")] - assert events[0]["response"]["id"] == events[4]["response"]["id"] - assert events[1]["item"]["id"] == events[2]["item_id"] - assert events[1]["item"]["id"] != events[3]["item"]["id"] - assert events[2]["delta"] == "original text" - assert events[2]["sequence_number"] == 4 - assert events[0]["response"]["created_at"] == events[4]["response"]["completed_at"] == 0 - assert [line for line in text.splitlines() if line.startswith("event:")] == [ - f"event: {p['type']}" for p in payloads - ] - assert not any(value in text for value in ("private-routing-id", "first-item", "other-item")) - assert "authorization" not in sanitized["headers"] - assert "private-routing-id" in response["body"]["string"] - - -def test_responses_sanitization_preserves_unknown_fields_and_disabled_content(): - body = { - "id": "routing-id", - "created_at": 123, - "output": [{"id": "item", "type": "message", "encrypted_content": "opaque-provider-state"}], - "new_field": {"text": "do not hide"}, - "api_key": "secret", - } - sanitized = sanitize_responses_recording( - {"headers": {"content-type": ["application/json"]}, "body": {"string": json.dumps(body).encode()}} - ) - payload = json.loads(sanitized["body"]["string"]) - assert payload["new_field"] == body["new_field"] - assert payload["output"][0]["type"] == "message" - assert payload["api_key"] == "[REDACTED]" - assert payload["output"][0]["id"] != "item" - assert payload["output"][0]["encrypted_content"] == "[REDACTED_ENCRYPTED_CONTENT]" From e95e4960e863cffcd8d3e5c68bb4f0c178db84b2 Mon Sep 17 00:00:00 2001 From: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> Date: Wed, 21 Oct 2026 19:03:28 +0200 Subject: [PATCH 3/6] fix(server): inherit native Responses metadata in LiteLLM dialect Keep only the event model annotation as a dialect extension. Reuse the native response declarations and shared recording harness, with exact replay consumption and buffered and streaming blocking coverage. Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> --- .../_generated/litellm-responses.guard.yaml | 84 +++------ .../openai/dialects/litellm/responses.md | 20 ++- .../openai/dialects/litellm/responses.py | 58 +------ .../openai/test_litellm_responses.py | 164 +++++++++--------- tests/server/experimental/responses_proxy.py | 16 +- .../test_litellm_responses_dialect.py | 8 +- 6 files changed, 128 insertions(+), 222 deletions(-) diff --git a/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml b/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml index 7a6fa00ca3..ce1592a7f5 100644 --- a/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml +++ b/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml @@ -245,17 +245,15 @@ request: stream_selector_field: stream response: additionalProperties: false - description: Derive coverage and extraction from the complete dialect declaration. + description: Bind one assistant text target; replacement is deliberately unsupported. properties: access_programs: - default: null x-nemo-guardrails: classification: opaque background: x-nemo-guardrails: classification: opaque billing: - default: null x-nemo-guardrails: classification: opaque completed_at: @@ -271,7 +269,6 @@ response: x-nemo-guardrails: classification: opaque frequency_penalty: - default: null x-nemo-guardrails: classification: opaque id: @@ -460,7 +457,6 @@ response: x-nemo-guardrails: classification: opaque presence_penalty: - default: null x-nemo-guardrails: classification: opaque previous_response_id: @@ -494,7 +490,6 @@ response: x-nemo-guardrails: classification: opaque store: - default: null x-nemo-guardrails: classification: opaque temperature: @@ -507,7 +502,6 @@ response: x-nemo-guardrails: classification: opaque tool_usage: - default: null x-nemo-guardrails: classification: opaque tools: @@ -530,13 +524,13 @@ response: classification: opaque required: - output - title: openai.litellm.responses.response.text.v1 + title: openai.responses.response.text.v1 type: object x-nemo-guardrails: source: '#/components/schemas/Response' unknown_fields: forbid reject_case_aliases: true - model: LiteLLMGuardedResponses + model: ResponsesGuardedResponse integration: name: litellm_responses endpoint: @@ -1225,18 +1219,16 @@ stream: classification: opaque response: additionalProperties: false - description: Apply the same reviewed metadata to initial and terminal response - snapshots. + description: Describe a full response snapshot without demanding a message + before generation. properties: access_programs: - default: null x-nemo-guardrails: classification: opaque background: x-nemo-guardrails: classification: opaque billing: - default: null x-nemo-guardrails: classification: opaque completed_at: @@ -1252,7 +1244,6 @@ stream: x-nemo-guardrails: classification: opaque frequency_penalty: - default: null x-nemo-guardrails: classification: opaque id: @@ -1443,7 +1434,6 @@ stream: x-nemo-guardrails: classification: opaque presence_penalty: - default: null x-nemo-guardrails: classification: opaque previous_response_id: @@ -1477,7 +1467,6 @@ stream: x-nemo-guardrails: classification: opaque store: - default: null x-nemo-guardrails: classification: opaque temperature: @@ -1490,7 +1479,6 @@ stream: x-nemo-guardrails: classification: opaque tool_usage: - default: null x-nemo-guardrails: classification: opaque tools: @@ -1513,7 +1501,7 @@ stream: classification: opaque required: - output - title: LiteLLMLifecycleResponse + title: ResponsesLifecycleResponseProjection type: object x-nemo-guardrails: classification: guarded @@ -1560,18 +1548,16 @@ stream: classification: opaque response: additionalProperties: false - description: Apply the same reviewed metadata to initial and terminal response - snapshots. + description: Describe a full response snapshot without demanding a message + before generation. properties: access_programs: - default: null x-nemo-guardrails: classification: opaque background: x-nemo-guardrails: classification: opaque billing: - default: null x-nemo-guardrails: classification: opaque completed_at: @@ -1587,7 +1573,6 @@ stream: x-nemo-guardrails: classification: opaque frequency_penalty: - default: null x-nemo-guardrails: classification: opaque id: @@ -1778,7 +1763,6 @@ stream: x-nemo-guardrails: classification: opaque presence_penalty: - default: null x-nemo-guardrails: classification: opaque previous_response_id: @@ -1812,7 +1796,6 @@ stream: x-nemo-guardrails: classification: opaque store: - default: null x-nemo-guardrails: classification: opaque temperature: @@ -1825,7 +1808,6 @@ stream: x-nemo-guardrails: classification: opaque tool_usage: - default: null x-nemo-guardrails: classification: opaque tools: @@ -1848,7 +1830,7 @@ stream: classification: opaque required: - output - title: LiteLLMLifecycleResponse + title: ResponsesLifecycleResponseProjection type: object x-nemo-guardrails: classification: guarded @@ -1895,18 +1877,16 @@ stream: classification: opaque response: additionalProperties: false - description: Apply the same reviewed metadata to initial and terminal response - snapshots. + description: Describe a full response snapshot without demanding a message + before generation. properties: access_programs: - default: null x-nemo-guardrails: classification: opaque background: x-nemo-guardrails: classification: opaque billing: - default: null x-nemo-guardrails: classification: opaque completed_at: @@ -1922,7 +1902,6 @@ stream: x-nemo-guardrails: classification: opaque frequency_penalty: - default: null x-nemo-guardrails: classification: opaque id: @@ -2113,7 +2092,6 @@ stream: x-nemo-guardrails: classification: opaque presence_penalty: - default: null x-nemo-guardrails: classification: opaque previous_response_id: @@ -2147,7 +2125,6 @@ stream: x-nemo-guardrails: classification: opaque store: - default: null x-nemo-guardrails: classification: opaque temperature: @@ -2160,7 +2137,6 @@ stream: x-nemo-guardrails: classification: opaque tool_usage: - default: null x-nemo-guardrails: classification: opaque tools: @@ -2183,7 +2159,7 @@ stream: classification: opaque required: - output - title: LiteLLMLifecycleResponse + title: ResponsesLifecycleResponseProjection type: object x-nemo-guardrails: classification: guarded @@ -2230,18 +2206,16 @@ stream: classification: opaque response: additionalProperties: false - description: Apply the same reviewed metadata to initial and terminal response - snapshots. + description: Describe a full response snapshot without demanding a message + before generation. properties: access_programs: - default: null x-nemo-guardrails: classification: opaque background: x-nemo-guardrails: classification: opaque billing: - default: null x-nemo-guardrails: classification: opaque completed_at: @@ -2257,7 +2231,6 @@ stream: x-nemo-guardrails: classification: opaque frequency_penalty: - default: null x-nemo-guardrails: classification: opaque id: @@ -2448,7 +2421,6 @@ stream: x-nemo-guardrails: classification: opaque presence_penalty: - default: null x-nemo-guardrails: classification: opaque previous_response_id: @@ -2482,7 +2454,6 @@ stream: x-nemo-guardrails: classification: opaque store: - default: null x-nemo-guardrails: classification: opaque temperature: @@ -2495,7 +2466,6 @@ stream: x-nemo-guardrails: classification: opaque tool_usage: - default: null x-nemo-guardrails: classification: opaque tools: @@ -2518,7 +2488,7 @@ stream: classification: opaque required: - output - title: LiteLLMLifecycleResponse + title: ResponsesLifecycleResponseProjection type: object x-nemo-guardrails: classification: guarded @@ -2565,18 +2535,16 @@ stream: classification: opaque response: additionalProperties: false - description: Apply the same reviewed metadata to initial and terminal response - snapshots. + description: Describe a full response snapshot without demanding a message + before generation. properties: access_programs: - default: null x-nemo-guardrails: classification: opaque background: x-nemo-guardrails: classification: opaque billing: - default: null x-nemo-guardrails: classification: opaque completed_at: @@ -2592,7 +2560,6 @@ stream: x-nemo-guardrails: classification: opaque frequency_penalty: - default: null x-nemo-guardrails: classification: opaque id: @@ -2783,7 +2750,6 @@ stream: x-nemo-guardrails: classification: opaque presence_penalty: - default: null x-nemo-guardrails: classification: opaque previous_response_id: @@ -2817,7 +2783,6 @@ stream: x-nemo-guardrails: classification: opaque store: - default: null x-nemo-guardrails: classification: opaque temperature: @@ -2830,7 +2795,6 @@ stream: x-nemo-guardrails: classification: opaque tool_usage: - default: null x-nemo-guardrails: classification: opaque tools: @@ -2853,7 +2817,7 @@ stream: classification: opaque required: - output - title: LiteLLMLifecycleResponse + title: ResponsesLifecycleResponseProjection type: object x-nemo-guardrails: classification: guarded @@ -2900,18 +2864,16 @@ stream: classification: opaque response: additionalProperties: false - description: Apply the same reviewed metadata to initial and terminal response - snapshots. + description: Describe a full response snapshot without demanding a message + before generation. properties: access_programs: - default: null x-nemo-guardrails: classification: opaque background: x-nemo-guardrails: classification: opaque billing: - default: null x-nemo-guardrails: classification: opaque completed_at: @@ -2927,7 +2889,6 @@ stream: x-nemo-guardrails: classification: opaque frequency_penalty: - default: null x-nemo-guardrails: classification: opaque id: @@ -3118,7 +3079,6 @@ stream: x-nemo-guardrails: classification: opaque presence_penalty: - default: null x-nemo-guardrails: classification: opaque previous_response_id: @@ -3152,7 +3112,6 @@ stream: x-nemo-guardrails: classification: opaque store: - default: null x-nemo-guardrails: classification: opaque temperature: @@ -3165,7 +3124,6 @@ stream: x-nemo-guardrails: classification: opaque tool_usage: - default: null x-nemo-guardrails: classification: opaque tools: @@ -3188,7 +3146,7 @@ stream: classification: opaque required: - output - title: LiteLLMLifecycleResponse + title: ResponsesLifecycleResponseProjection type: object x-nemo-guardrails: classification: guarded diff --git a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md index ec99578af6..41fce52d8d 100644 --- a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md +++ b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md @@ -10,7 +10,7 @@ package, new transport, new kernel, or alternate lifecycle hooks are required. | Location | Decision | | --- | --- | | Request | Reuse the strict Responses binding unchanged: one string input, foreground generation, no tools, conversation history, or additional plaintext instructions. | -| Buffered response and lifecycle `response` snapshots | Recognize optional `access_programs`, `billing`, `frequency_penalty`, `presence_penalty`, `store`, and `tool_usage` as opaque metadata. | +| Buffered response and lifecycle `response` snapshots | Reuse the native response policy, including opaque `access_programs`, `billing`, `frequency_penalty`, `presence_penalty`, `store`, and `tool_usage` metadata. | | Root of each reviewed SSE event | Recognize optional `model` as opaque metadata, never as an event selector or text source. | | Output items, content parts, delta text, annotations, logprobs, reasoning summaries | Inherit the existing constrained/guarded policy unchanged. No Chat-specific `provider_specific_fields` bags are added. | | Event selection and lifecycle | Retain all 15 existing event rules, derived paths/coverage, native terminal requirements, identity checks, snapshot consistency, and error framing. | @@ -21,10 +21,10 @@ permission to emit tool calls. The actual output items still reject tool content Allowing `store` in the response does not introduce a new request control: it was already reviewed in the base request declaration. -These names describe the reviewed **gateway surface**, not fields all invented -by LiteLLM. For example, `access_programs` appears in the -[native OpenAI response reference](https://developers.openai.com/api/reference/resources/responses/methods/create). -This change does not broaden the strict endpoint's pinned field inventory. +Native OpenAI recordings also contain all six response metadata names. They +belong to the base Responses policy; the dialect adds only the root SSE `model` +annotation. Buffered response and nested lifecycle response declarations are +shared with the base endpoint. Unknown root members, new events and unsupported content still fail closed. The base profile's trusted nested unknown-field override remains unchanged. @@ -76,7 +76,7 @@ Two genuine cassettes live in `tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/`. They capture `openai/openai/gpt-5-nano` through NVIDIA's inference gateway, with a harmless greeting, `store: false`, and a 256-token output limit. Buffered evidence -includes all six metadata additions. Streaming evidence includes the model +includes all six shared metadata fields. Streaming evidence includes the model annotation and initial/final snapshots, reasoning-item metadata, assistant-item snapshots, content-part snapshots and text deltas/done. No new event type or Chat extension bag was observed. They are evidence for this route, not raw vLLM or @@ -116,6 +116,10 @@ uv run --locked python -m nemoguardrails.server.experimental.provider.contract_e Capture consumes the entire upstream response before policy assertions. Mandatory offline replay then checks the actual endpoint, export, text extraction, terminal behavior and passthrough bytes. A successful capture alone is not compatibility -validation. Dialect rejection cases use synthetic unit fixtures; generic blocking, -cancellation and malformed-stream tests remain with the existing Responses stack. +validation. Each replay pass consumes exactly one interaction, with explicit rewinds between +direct, inspected, and uninspected comparisons. Successful cassettes also cover +buffered blocking and first/later streaming-window blocking with synthetic +checker verdicts, forced to replay even during refresh. Dialect rejection cases +use synthetic unit fixtures; cancellation and malformed-stream tests remain +with the existing Responses stack. The exported YAML describes the effective declarations and is not a runtime input. diff --git a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.py b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.py index 4020ddad88..d6f4799212 100644 --- a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.py +++ b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.py @@ -15,8 +15,8 @@ """Bind reviewed LiteLLM Responses metadata without relaxing text boundaries. -The added fields describe the observed gateway surface, not a claim that every -field originates in LiteLLM. Requests and output-item policy remain OpenAI's. +The event model annotation extends the native surface. Requests, buffered +responses, lifecycle response objects, and output-item policy remain OpenAI's. See responses.md for source, recording provenance and explicit selection. """ @@ -27,10 +27,7 @@ from nemoguardrails.server.experimental.provider.projection_policy import ( PolicyModel, - guarded, opaque, - payload_contract, - text_location, ) from nemoguardrails.server.experimental.provider.stream import StreamBinding from nemoguardrails.server.experimental.provider.stream_classifier import StreamClassifierDefinition @@ -41,9 +38,6 @@ ) from nemoguardrails.server.experimental.providers.openai.responses import stream_projection as models from nemoguardrails.server.experimental.providers.openai.responses.endpoint import RESPONSES_ENDPOINT -from nemoguardrails.server.experimental.providers.openai.responses.response_projection import ( - ResponsesGuardedResponseProjection, -) from nemoguardrails.server.experimental.providers.openai.responses.stream_classifier import ( STREAM_CONTRACT as OPENAI_STREAM_CONTRACT, ) @@ -57,47 +51,12 @@ from nemoguardrails.server.experimental.providers.openai.responses.stream_hooks import ResponsesStreamHooks -class LiteLLMResponsesMetadata(PolicyModel): - """Preserve reviewed accounting and response-control metadata uninspected. - - These are optional response fields, never request controls or event selectors. - Containers are intentionally opaque like the inherited usage metadata; this - does not authorize new output items, plaintext reasoning, or extension bags. - """ - - access_programs: Annotated[Any, opaque()] = None - billing: Annotated[Any, opaque()] = None - frequency_penalty: Annotated[Any, opaque()] = None - presence_penalty: Annotated[Any, opaque()] = None - store: Annotated[Any, opaque()] = None - tool_usage: Annotated[Any, opaque()] = None - - -class LiteLLMResponsesProjection(ResponsesGuardedResponseProjection, LiteLLMResponsesMetadata): - """Retain one assistant text output and the inherited reasoning restrictions.""" - - -class LiteLLMGuardedResponses(LiteLLMResponsesProjection): - """Derive coverage and extraction from the complete dialect declaration.""" - - guarded_text_location = text_location(LiteLLMResponsesProjection) - projection_contract = payload_contract( - LiteLLMResponsesProjection, - projection_id="openai.litellm.responses.response.text.v1", - direction="response", - ) - - class LiteLLMResponsesEventMetadata(PolicyModel): """Accept the gateway's model annotation without using it to select events.""" model: Annotated[Any, opaque()] = None -class LiteLLMLifecycleResponse(models.ResponsesLifecycleResponseProjection, LiteLLMResponsesMetadata): - """Apply the same reviewed metadata to initial and terminal response snapshots.""" - - class LiteLLMTextDeltaEvent(models.ResponsesTextDeltaEventProjection, LiteLLMResponsesEventMetadata): """Keep the delta as the only newly generated text subject.""" @@ -133,38 +92,26 @@ class LiteLLMReasoningDoneEvent(models.ResponsesReasoningDoneEventProjection, Li class LiteLLMCreatedEvent(models.ResponsesCreatedEventProjection, LiteLLMResponsesEventMetadata): """Check the creation snapshot with the reviewed response metadata.""" - response: Annotated[LiteLLMLifecycleResponse, guarded()] - class LiteLLMInProgressEvent(models.ResponsesInProgressEventProjection, LiteLLMResponsesEventMetadata): """Check the progress snapshot without introducing a separate text target.""" - response: Annotated[LiteLLMLifecycleResponse, guarded()] - class LiteLLMQueuedEvent(models.ResponsesQueuedEventProjection, LiteLLMResponsesEventMetadata): """Recognize queued metadata without enabling background requests.""" - response: Annotated[LiteLLMLifecycleResponse, guarded()] - class LiteLLMCompletedEvent(models.ResponsesCompletedEventProjection, LiteLLMResponsesEventMetadata): """Check the native terminal snapshot against accumulated deltas.""" - response: Annotated[LiteLLMLifecycleResponse, guarded()] - class LiteLLMFailedEvent(models.ResponsesFailedEventProjection, LiteLLMResponsesEventMetadata): """Keep any output in failed responses subject to snapshot checks.""" - response: Annotated[LiteLLMLifecycleResponse, guarded()] - class LiteLLMIncompleteEvent(models.ResponsesIncompleteEventProjection, LiteLLMResponsesEventMetadata): """Keep partial output guarded even when generation is incomplete.""" - response: Annotated[LiteLLMLifecycleResponse, guarded()] - class LiteLLMErrorEvent(models.ResponsesErrorEventProjection, LiteLLMResponsesEventMetadata): """Accept model metadata on the inherited flat diagnostic error shape.""" @@ -216,6 +163,5 @@ class LiteLLMErrorEvent(models.ResponsesErrorEventProjection, LiteLLMResponsesEv RESPONSES_ENDPOINT, contract_name="litellm_responses", operation="LiteLLM Responses", - guarded_response_model=LiteLLMGuardedResponses, stream=StreamBinding(STREAM_CLASSIFIER, ResponsesStreamHooks), ) diff --git a/tests/recorded/server/experimental/openai/test_litellm_responses.py b/tests/recorded/server/experimental/openai/test_litellm_responses.py index 3312f2b187..5388f2845a 100644 --- a/tests/recorded/server/experimental/openai/test_litellm_responses.py +++ b/tests/recorded/server/experimental/openai/test_litellm_responses.py @@ -21,33 +21,30 @@ """ import json -from contextlib import asynccontextmanager +from functools import partial import httpx import pytest -from fastapi import FastAPI from jsonschema import Draft202012Validator from nemoguardrails.server.experimental._content_checker import ( ContentAllowed, + ContentBlocked, ContentInspectionPolicy, StreamBufferingPolicy, ) -from nemoguardrails.server.experimental._guarded_proxy import create_guarded_http_operation -from nemoguardrails.server.experimental._http_kernel import create_http_proxy_router -from nemoguardrails.server.experimental._provider_transport import ProviderTransport from nemoguardrails.server.experimental.provider.contract_export import export_guard_contract -from nemoguardrails.server.experimental.provider.integration import UpstreamSettings from nemoguardrails.server.experimental.provider.sse import iter_sse_events from nemoguardrails.server.experimental.provider.stream import StreamEventRole from nemoguardrails.server.experimental.providers.openai.dialects.litellm.responses import LITELLM_RESPONSES_ENDPOINT -from nemoguardrails.server.experimental.providers.openai.errors import OPENAI_ERROR_MAPPING from nemoguardrails.server.experimental.providers.openai.responses.endpoint import RESPONSES_ENDPOINT from tests.recorded.cassette import cassette_request_jsons +from tests.server.experimental.checkers import StaticChecker +from tests.server.experimental.responses_proxy import responses_proxy pytestmark = [ pytest.mark.recorded, - pytest.mark.vcr(ignore_hosts=["recorded-proxy.test"], allow_playback_repeats=True), + pytest.mark.vcr(ignore_hosts=["recorded-responses.test"]), pytest.mark.asyncio, ] @@ -55,6 +52,7 @@ MODEL = "openai/openai/gpt-5-nano" INPUT = "Reply with exactly: Hello." ENDPOINT = LITELLM_RESPONSES_ENDPOINT +proxy = partial(responses_proxy, endpoint=ENDPOINT, base_url=BASE_URL) def responses_request(stream, *, model=MODEL): @@ -68,60 +66,6 @@ def responses_request(stream, *, model=MODEL): } -class Checker: - def __init__(self): - self.calls = [] - - def inspection_policy(self): - return ContentInspectionPolicy(True, True, StreamBufferingPolicy(1, 0)) - - async def check_input(self, check): - self.calls.append(("input", check.message.content)) - return ContentAllowed() - - async def check_output(self, check): - self.calls.append(("output", check.output_content)) - return ContentAllowed() - - -@asynccontextmanager -async def proxy(api_key, checker, dispatches): - async with httpx.AsyncClient(timeout=60) as upstream: - transport = ProviderTransport(UpstreamSettings(base_url=BASE_URL, mount_path="/v1"), upstream) - - async def buffered(request): - dispatches.append(request) - return await transport.dispatch(request, require_identity_response=True) - - async def streaming(request): - dispatches.append(request) - return await transport.stream_dispatch(request, require_identity_response=True) - - app = FastAPI() - app.include_router( - create_http_proxy_router( - operations=( - create_guarded_http_operation( - ENDPOINT, - OPENAI_ERROR_MAPPING, - stream_dispatch=streaming, - max_stream_event_bytes=1024 * 1024, - max_pending_stream_bytes=1024 * 1024, - ), - ), - checker=checker, - dispatch=buffered, - render_outcome=OPENAI_ERROR_MAPPING.renderer, - ) - ) - async with httpx.AsyncClient( - transport=httpx.ASGITransport(app=app), - base_url="http://recorded-proxy.test", - headers={"authorization": f"Bearer {api_key}"}, - ) as client: - yield client - - async def capture(api_key, request): async with httpx.AsyncClient(timeout=60) as client: return await client.post( @@ -145,14 +89,7 @@ async def validate_inventory(upstream, request): Draft202012Validator(document["request"]).validate(request) if not request["stream"]: payload = upstream.json() - assert payload.keys() - base_document["response"]["properties"].keys() == { - "access_programs", - "billing", - "frequency_penalty", - "presence_penalty", - "store", - "tool_usage", - } + assert payload.keys() <= base_document["response"]["properties"].keys() assert payload["access_programs"] is None assert payload["billing"] == {"payer": "developer"} assert payload["store"] is False @@ -174,12 +111,7 @@ async def validate_inventory(upstream, request): assert payload["model"] == MODEL assert "provider_specific_fields" not in payload if "response" in payload: - assert payload["response"].keys() - base_document["response"]["properties"].keys() == { - "frequency_penalty", - "presence_penalty", - "store", - "tool_usage", - } + assert payload["response"].keys() <= base_document["response"]["properties"].keys() kinds.append(payload["type"]) classified = adapter.classify_event(event) if classified.role is StreamEventRole.GUARDED_TEXT: @@ -203,24 +135,86 @@ async def validate_inventory(upstream, request): @pytest.mark.parametrize("stream", [False, True], ids=["buffered", "streaming"]) async def test_gateway_responses_preserve_policy_and_wire( - nvidia_inference_key, record_mode, recorded_cassette_path, stream + nvidia_inference_key, record_mode, recorded_cassette_path, vcr, stream ): request = responses_request(stream) upstream = await capture(nvidia_inference_key, request) assert upstream.status_code == 200 if record_mode != "none": + assert len(vcr.requests) == 1 return + assert vcr.play_count == 1 + assert vcr.all_played + assert cassette_request_jsons(recorded_cassette_path) == [request] text = await validate_inventory(upstream, request) assert text - checker = Checker() + for inspected in (True, False): + vcr.rewind() + checker = StaticChecker( + ContentInspectionPolicy(inspected, inspected, StreamBufferingPolicy(1, 0) if inspected else None) + ) + dispatches = [] + async with proxy(nvidia_inference_key, checker, dispatches) as client: + response = await client.post("/v1/responses", json=request) + assert response.status_code == upstream.status_code + assert response.content == upstream.content + assert response.headers["content-type"] == upstream.headers["content-type"] + assert len(dispatches) == len(vcr.requests) == vcr.play_count == 1 + assert vcr.all_played + assert json.loads(dispatches[0].body) == request + if inspected: + assert checker.text_calls[0] == ("input", INPUT) + assert "".join(value for kind, value in checker.text_calls if kind == "output") == text + else: + assert checker.calls == [] + + +@pytest.mark.vcr("test_gateway_responses_preserve_policy_and_wire[buffered].yaml", record_mode="none") +async def test_gateway_recorded_buffered_block_withholds_output(nvidia_inference_key, vcr): + checker = StaticChecker(ContentInspectionPolicy(True, True), output_decisions=[ContentBlocked("blocked by test")]) + dispatches = [] + async with proxy(nvidia_inference_key, checker, dispatches) as client: + response = await client.post("/v1/responses", json=responses_request(False)) + assert response.status_code == 400 + assert response.json()["error"]["code"] == "content_blocked" + assert len(dispatches) == len(vcr.requests) == vcr.play_count == 1 + assert vcr.all_played + assert checker.text_calls[0] == ("input", INPUT) + assert checker.text_calls[1][0] == "output" + assert checker.text_calls[1][1] not in response.text + + +@pytest.mark.vcr("test_gateway_responses_preserve_policy_and_wire[streaming].yaml", record_mode="none") +@pytest.mark.parametrize("allowed_windows", [0, 1], ids=["first-window", "later-window"]) +async def test_gateway_recorded_stream_releases_only_approved_prefix(nvidia_inference_key, vcr, allowed_windows): + upstream = await capture(nvidia_inference_key, responses_request(True)) + recorded_events = await events(upstream.content) + deltas = [ + (index, payload["delta"]) + for index, event in enumerate(recorded_events) + if event.data not in (None, b"[DONE]") + and (payload := json.loads(event.data))["type"] == "response.output_text.delta" + and payload["delta"] + ] + assert len(deltas) > allowed_windows + released_end = deltas[allowed_windows - 1][0] + 1 if allowed_windows else 0 + assert vcr.play_count == 1 + vcr.rewind() + checker = StaticChecker( + ContentInspectionPolicy(True, True, StreamBufferingPolicy(1, 0)), + output_decisions=[ContentAllowed()] * allowed_windows + [ContentBlocked("blocked by test")], + ) dispatches = [] async with proxy(nvidia_inference_key, checker, dispatches) as client: - response = await client.post("/v1/responses", json=request) - assert response.status_code == upstream.status_code - assert response.content == upstream.content - assert len(dispatches) == 1 - assert json.loads(dispatches[0].body) == request - assert checker.calls[0] == ("input", INPUT) - assert "".join(value for kind, value in checker.calls if kind == "output") == text + response = await client.post("/v1/responses", json=responses_request(True)) + assert response.status_code == 200 + received = await events(response.content) + assert [event.raw for event in received[:-1]] == [event.raw for event in recorded_events[:released_end]] + assert received[-1].event_type == b"error" + assert json.loads(received[-1].data)["code"] == "content_blocked" + assert b"[DONE]" not in response.content + assert checker.text_calls == [("input", INPUT)] + [("output", text) for _, text in deltas[: allowed_windows + 1]] + assert len(dispatches) == len(vcr.requests) == vcr.play_count == 1 + assert vcr.all_played diff --git a/tests/server/experimental/responses_proxy.py b/tests/server/experimental/responses_proxy.py index fdf73d7c0f..3b55f4620c 100644 --- a/tests/server/experimental/responses_proxy.py +++ b/tests/server/experimental/responses_proxy.py @@ -27,11 +27,17 @@ @asynccontextmanager -async def responses_proxy(api_key, checker, dispatches, *, http_transport=None): +async def responses_proxy( + api_key, + checker, + dispatches, + *, + http_transport=None, + endpoint=RESPONSES_ENDPOINT, + base_url="https://api.openai.com/v1", +): async with httpx.AsyncClient(timeout=60, transport=http_transport) as upstream: - transport = ProviderTransport( - UpstreamSettings(base_url="https://api.openai.com/v1", mount_path="/v1"), upstream - ) + transport = ProviderTransport(UpstreamSettings(base_url=base_url, mount_path="/v1"), upstream) async def buffered(request): dispatches.append(request) @@ -46,7 +52,7 @@ async def streaming(request): create_http_proxy_router( operations=( create_guarded_http_operation( - RESPONSES_ENDPOINT, + endpoint, OPENAI_ERROR_MAPPING, stream_dispatch=streaming, max_stream_event_bytes=1024 * 1024, diff --git a/tests/server/experimental/test_litellm_responses_dialect.py b/tests/server/experimental/test_litellm_responses_dialect.py index f29aa16db5..4381605a99 100644 --- a/tests/server/experimental/test_litellm_responses_dialect.py +++ b/tests/server/experimental/test_litellm_responses_dialect.py @@ -31,7 +31,6 @@ from nemoguardrails.server.experimental.providers.openai.dialects.litellm.responses import ( LITELLM_RESPONSES_ENDPOINT, STREAM_RULES, - LiteLLMLifecycleResponse, ) from nemoguardrails.server.experimental.providers.openai.responses.endpoint import RESPONSES_ENDPOINT from nemoguardrails.server.experimental.providers.openai.responses.stream_classifier import STREAM_RULES as BASE_RULES @@ -50,6 +49,7 @@ def test_dialect_is_explicit_and_reuses_request_policy_and_lifecycle_hooks(): endpoint = LITELLM_RESPONSES_ENDPOINT + assert endpoint.guarded_response_model is RESPONSES_ENDPOINT.guarded_response_model assert endpoint.guarded_request_model is RESPONSES_ENDPOINT.guarded_request_model assert endpoint.stream.hooks_factory is RESPONSES_ENDPOINT.stream.hooks_factory assert endpoint.operation_paths == RESPONSES_ENDPOINT.operation_paths @@ -60,15 +60,13 @@ def test_dialect_is_explicit_and_reuses_request_policy_and_lifecycle_hooks(): @pytest.mark.parametrize("name,value", METADATA.items()) -def test_reviewed_metadata_is_response_only_and_preserves_the_original_payload(name, value): +def test_native_metadata_is_shared_and_preserves_the_original_payload(name, value): payload = {**response_payload(), name: value} original = deepcopy(payload) - with pytest.raises(ValidationError): - RESPONSES_ENDPOINT.guarded_response_model.validate_payload(payload) + RESPONSES_ENDPOINT.guarded_response_model.validate_payload(payload) projection = LITELLM_RESPONSES_ENDPOINT.guarded_response_model.validate_payload(payload) assert projection.locate_guarded_message(payload).message.content == "answer" assert payload == original - LiteLLMLifecycleResponse.model_validate(payload) exported = export_guard_contract(LITELLM_RESPONSES_ENDPOINT) assert exported["response"]["properties"][name][EXTENSION]["classification"] == "opaque" Draft202012Validator(exported["response"]).validate(payload) From fbe27d50267c5b62376abbaa2ad5c0c395ae3725 Mon Sep 17 00:00:00 2001 From: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> Date: Fri, 23 Oct 2026 10:47:35 +0200 Subject: [PATCH 4/6] fix(server): sync LiteLLM Responses storage contract Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> --- .../openai/_generated/litellm-responses.guard.yaml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml b/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml index ce1592a7f5..acb0886b8b 100644 --- a/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml +++ b/nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml @@ -144,8 +144,11 @@ request: x-nemo-guardrails: classification: opaque store: + const: false + type: boolean x-nemo-guardrails: - classification: opaque + classification: constrained + reason: core_capability.response_storage stream: default: false type: boolean @@ -235,6 +238,7 @@ request: classification: opaque required: - input + - store title: openai.responses.request.text.v1 type: object x-nemo-guardrails: From 3badd6b47a1f3f19d072dd3b4cb766f512755485 Mon Sep 17 00:00:00 2001 From: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> Date: Fri, 23 Oct 2026 11:02:56 +0200 Subject: [PATCH 5/6] test(server): validate LiteLLM rejection test baseline Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> --- tests/server/experimental/test_litellm_responses_dialect.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/server/experimental/test_litellm_responses_dialect.py b/tests/server/experimental/test_litellm_responses_dialect.py index 4381605a99..83ab200259 100644 --- a/tests/server/experimental/test_litellm_responses_dialect.py +++ b/tests/server/experimental/test_litellm_responses_dialect.py @@ -77,8 +77,11 @@ def test_native_metadata_is_shared_and_preserves_the_original_payload(name, valu [{"guardrails": []}, {"fallbacks": []}, {"billing": {}}, {"input": []}, {"tools": []}, {"instructions": "hidden"}], ) def test_dialect_does_not_enable_gateway_request_controls_or_extra_content(updates): + model = LITELLM_RESPONSES_ENDPOINT.guarded_request_model + payload = {"input": "hello", "store": False} + model.validate_payload(payload) with pytest.raises(ValidationError): - LITELLM_RESPONSES_ENDPOINT.guarded_request_model.validate_payload({"input": "hello", **updates}) + model.validate_payload({**payload, **updates}) @pytest.mark.parametrize( From f1cfe0b88ce51f0e76e75408dacfe135e911ff34 Mon Sep 17 00:00:00 2001 From: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> Date: Fri, 23 Oct 2026 11:03:13 +0200 Subject: [PATCH 6/6] docs(server): simplify LiteLLM Responses guidance Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com> --- .../openai/dialects/litellm/README.md | 5 +- .../openai/dialects/litellm/responses.md | 148 +++++++----------- tests/recorded/README.md | 32 ++-- 3 files changed, 70 insertions(+), 115 deletions(-) diff --git a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md index fe3a793dc6..eb944cef88 100644 --- a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md +++ b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md @@ -1,8 +1,7 @@ # LiteLLM Chat dialect -For the separately reviewed Responses operation, see the -[LiteLLM Responses dialect](responses.md). Chat extension bags and backend -evidence do not automatically apply to Responses. +For Responses, see the [LiteLLM Responses dialect](responses.md), which has +its own supported boundary and compatibility evidence. This experimental dialect applies the `single_text.v1` guardrail profile to reviewed LiteLLM Chat Completions wire shapes. Typed Python declarations in diff --git a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md index 41fce52d8d..6d8ef6f833 100644 --- a/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md +++ b/nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md @@ -1,102 +1,69 @@ # LiteLLM Responses dialect -The experimental [typed declaration](responses.py) binds the existing Responses -single-text policy to a reviewed LiteLLM gateway surface. It is selected explicitly; -the strict OpenAI endpoint and the LiteLLM Chat dialect are unchanged. No LiteLLM -package, new transport, new kernel, or alternate lifecycle hooks are required. - -## Policy difference - -| Location | Decision | -| --- | --- | -| Request | Reuse the strict Responses binding unchanged: one string input, foreground generation, no tools, conversation history, or additional plaintext instructions. | -| Buffered response and lifecycle `response` snapshots | Reuse the native response policy, including opaque `access_programs`, `billing`, `frequency_penalty`, `presence_penalty`, `store`, and `tool_usage` metadata. | -| Root of each reviewed SSE event | Recognize optional `model` as opaque metadata, never as an event selector or text source. | -| Output items, content parts, delta text, annotations, logprobs, reasoning summaries | Inherit the existing constrained/guarded policy unchanged. No Chat-specific `provider_specific_fields` bags are added. | -| Event selection and lifecycle | Retain all 15 existing event rules, derived paths/coverage, native terminal requirements, identity checks, snapshot consistency, and error framing. | - -Opaque metadata is passed through **without inspecting its values or validating -its nested structure**. For example, `tool_usage` is accounting metadata, not -permission to emit tool calls. The actual output items still reject tool content. -Allowing `store` in the response does not introduce a new request control: it was -already reviewed in the base request declaration. - -Native OpenAI recordings also contain all six response metadata names. They -belong to the base Responses policy; the dialect adds only the root SSE `model` -annotation. Buffered response and nested lifecycle response declarations are -shared with the base endpoint. -Unknown root members, new events and unsupported content still fail closed. -The base profile's trusted nested unknown-field override remains unchanged. - -## Explicit composition - -Use the existing application assembly with trusted endpoint declarations: +This experimental [dialect](responses.py) adds optional, opaque `model` metadata +at the root of SSE events. Requests, buffered responses, and stream lifecycle +checks use the [native Responses policy](../../responses/README.md). + +## Supported boundary + +Requests require one nonempty `input` string and explicit `store: false`. +Generation must be foreground; tools, conversation history, additional plaintext +instructions, and replacement are unsupported. Buffered and streaming output +support one assistant text response under the native content restrictions. + +Reviewed response metadata, including `store` and `tool_usage`, remains opaque: +its values and nested structure are not inspected. The event's `model` field +does not select events or supply guarded text. Unknown root fields and events, +additional content channels, and nested LiteLLM SSE errors fail closed when +output inspection is enabled. Disabling output inspection retains raw response +relay; request restrictions still apply. + +## Explicit selection + +Select the endpoint when constructing the proxy integration: ```python from dataclasses import replace from nemoguardrails.server.experimental.providers.openai.integration import create_openai_integration -from nemoguardrails.server.experimental.providers.openai.dialects.litellm.chat_completions import ( - LITELLM_CHAT_COMPLETIONS_ENDPOINT, -) from nemoguardrails.server.experimental.providers.openai.dialects.litellm.responses import ( LITELLM_RESPONSES_ENDPOINT, ) integration = replace( create_openai_integration(base_url="https://your-gateway.example/v1"), - endpoints=(LITELLM_CHAT_COMPLETIONS_ENDPOINT, LITELLM_RESPONSES_ENDPOINT), + endpoints=(LITELLM_RESPONSES_ENDPOINT,), ) ``` -Supply this integration to the existing proxy application alongside its Rails -configuration. Credentials use the application's existing HTTP authorization -path; neither YAML exports, model strings nor gateway hostnames select a dialect. -This is the transparent HTTP path, not a LangChain or LiteLLM client integration. - -## Source and recording evidence - -On 2026-10-15 the authenticated deployment OpenAPI advertised LiteLLM `1.105.1`. -Both `/responses` and `/v1/responses` declared `ResponsesAPIResponse` for buffered -success and a string SSE body for streaming. The latter is not an event schema. -The private schema is not vendored, and its advertised version does not establish -the absence of deployment patches. - -The matching public tag resolves to -[`d665dc1a6315847fcad6cc2c1264ceb47c06abce`](https://github.com/BerriAI/litellm/tree/d665dc1a6315847fcad6cc2c1264ceb47c06abce): - -- [`litellm/types/llms/openai.py`](https://github.com/BerriAI/litellm/blob/d665dc1a6315847fcad6cc2c1264ceb47c06abce/litellm/types/llms/openai.py) - declares `ResponsesAPIResponse`, lifecycle events and a separate nested error - shape. SHA-256: `99ce22ba2bb83dc37d9afb423ef2a6cf919a9cafca834eaad833c4a795088fdd`. -- [`litellm/llms/openai/responses/transformation.py`](https://github.com/BerriAI/litellm/blob/d665dc1a6315847fcad6cc2c1264ceb47c06abce/litellm/llms/openai/responses/transformation.py) - constructs a Responses object from native OpenAI JSON, with a construction - fallback on validation failure. SHA-256: `7c6f3bea7678b76b6e99bc5b4c499eef9aead8947a6a81e534dc5e42c84bfc70`. - -Two genuine cassettes live in -`tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/`. -They capture `openai/openai/gpt-5-nano` through NVIDIA's inference gateway, with a -harmless greeting, `store: false`, and a 256-token output limit. Buffered evidence -includes all six shared metadata fields. Streaming evidence includes the model -annotation and initial/final snapshots, reasoning-item metadata, assistant-item -snapshots, content-part snapshots and text deltas/done. No new event type or Chat -extension bag was observed. They are evidence for this route, not raw vLLM or -every LiteLLM backend. - -The [LiteLLM Responses documentation](https://docs.litellm.ai/docs/response_api) -also describes Chat-to-Responses bridging. These cassettes do not establish -compatibility with bridged routes. Nested LiteLLM SSE errors, additional content -channels, and other backend-specific fields remain unsupported and fail closed. -Flat diagnostic errors and other inherited events have synthetic unit coverage; -they are not claimed as recorded observations. - -Recordings use the same shared sanitizer as the Chat recordings: credentials are -removed and selected response metadata is normalized. There is no Responses-only -content redaction. Assistant text, encrypted content, nested item identifiers, -event order, sequence numbers and field presence are retained, subject to the -shared credential filters. Sanitization is not a policy repair: unsupported -fields must not be deleted to make replay pass. -Exact passthrough comparisons are against the sanitized replay bytes, not a claim -that the cassette preserves original HTTP chunk boundaries or all original bytes. +Pass this integration to the proxy application with its Rails configuration. +The transparent HTTP proxy uses the `server` extra and the application's existing +upstream authorization; no LiteLLM SDK or additional dialect-specific key is +required. The gateway URL and request model do not select the dialect. Selecting +Responses alone leaves other routes on the application's unguarded passthrough +path; restrict exposed routes as described in the native policy. + +## Compatibility evidence + +The source baseline is LiteLLM `v1.105.1`, pinned to +[`d665dc1a6315847fcad6cc2c1264ceb47c06abce`](https://github.com/BerriAI/litellm/tree/d665dc1a6315847fcad6cc2c1264ceb47c06abce). +The reviewed sources are the +[Responses types](https://github.com/BerriAI/litellm/blob/d665dc1a6315847fcad6cc2c1264ceb47c06abce/litellm/types/llms/openai.py) +and [OpenAI response transformation](https://github.com/BerriAI/litellm/blob/d665dc1a6315847fcad6cc2c1264ceb47c06abce/litellm/llms/openai/responses/transformation.py). +A deployment's advertised version does not establish the absence of patches. + +Two buffered/streaming cassettes in +`tests/recorded/server/experimental/openai/cassettes/test_litellm_responses/` +capture `openai/openai/gpt-5-nano` through NVIDIA's inference gateway with +`store: false`. They cover shared response metadata, the SSE `model` annotation, +text deltas, and repeated snapshots. This evidence is specific to that route; +Chat-to-Responses bridging and other LiteLLM backends require separate validation. +Other inherited events and rejection cases use synthetic unit fixtures. + +The shared recording sanitizer retains content and nested item identifiers, +subject to credential filtering and metadata normalization. Replay compares +sanitized bytes, not original network chunk boundaries. Unsupported fields must +not be removed to make replay pass. ## Validation and export @@ -113,13 +80,8 @@ uv run --locked python -m nemoguardrails.server.experimental.provider.contract_e --output nemoguardrails/server/experimental/contracts/openai/_generated/litellm-responses.guard.yaml ``` -Capture consumes the entire upstream response before policy assertions. Mandatory -offline replay then checks the actual endpoint, export, text extraction, terminal -behavior and passthrough bytes. A successful capture alone is not compatibility -validation. Each replay pass consumes exactly one interaction, with explicit rewinds between -direct, inspected, and uninspected comparisons. Successful cassettes also cover -buffered blocking and first/later streaming-window blocking with synthetic -checker verdicts, forced to replay even during refresh. Dialect rejection cases -use synthetic unit fixtures; cancellation and malformed-stream tests remain -with the existing Responses stack. -The exported YAML describes the effective declarations and is not a runtime input. +Capture saves the complete sanitized response; offline replay validates policy, +passthrough, and blocking with synthetic checker decisions. Replay is required +after a refresh and uses no live credentials. The exported YAML documents the +bound declarations and is not a runtime input; use `--check` instead of +`--output` to detect drift. diff --git a/tests/recorded/README.md b/tests/recorded/README.md index dece7a19c6..1233032489 100644 --- a/tests/recorded/README.md +++ b/tests/recorded/README.md @@ -128,25 +128,19 @@ exchanges with synthetic checker decisions, not real Rails configuration. Replayed-body equality is not a claim of original provider-wire byte fidelity or live streaming delivery. Real-config and socket coverage is separate. -The separately reviewed Responses recordings explicitly select the -[LiteLLM Responses dialect](../../nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md). -Refresh only `tests/recorded/server/experimental/openai/test_litellm_responses.py` -with `RECORDED_REQUIRED_KEYS=NVIDIA_INFERENCE_KEY`. Capture stores the complete -sanitized response before checking policy; the subsequent offline replay is the -acceptance gate. Responses uses the same shared sanitizer as Chat, with no extra -content redaction; encrypted content and nested item-ID relationships are retained. -These two cassettes establish neither Chat bridge nor arbitrary-backend compatibility. - -These six cassettes predate the dialect: their provider payloads are unchanged. -Two buffered successes contain message `provider_specific_fields: {refusal: null}` -and empty choice extensions. Two streaming successes contain no extension bags; -the other two responses are provider errors. The inventory tests validate every -successful payload against runtime declarations and export and pin those evidence -limits. The recordings do not establish the serving LiteLLM version or support -for arbitrary backend fields. Native finish metadata and empty stream containers -are covered by separately identified, source-backed unit fixtures, not invented -recordings. A refresh must review new fields rather than modifying payloads to -fit existing expectations. +The six Chat proxy cassettes contain two buffered successes with message +`provider_specific_fields: {refusal: null}` and empty choice extensions, two +streaming successes without extension bags, and two provider errors. Inventory +tests check successful payloads against runtime declarations and export. +Native finish metadata and empty stream containers have separate source-backed +unit fixtures. These recordings do not establish the serving LiteLLM version or +support for arbitrary backend fields; review new fields when refreshing them. + +For LiteLLM Responses, refresh +`tests/recorded/server/experimental/openai/test_litellm_responses.py` with +`RECORDED_REQUIRED_KEYS=NVIDIA_INFERENCE_KEY`. The +[Responses dialect guide](../../nemoguardrails/server/experimental/providers/openai/dialects/litellm/responses.md) +documents the two cassettes, compatibility limits, and required offline replay. `RECORDED_TESTS` is passed directly to pytest, so it can be a single test, a test class, several files, or a directory: