Repository navigation
feat(server): define OpenAI Chat contract #2412
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Pouyanpi
wants to merge
3
commits into
pouyanpi/guard-contract-foundation-1
from
pouyanpi/openai-chat-contract-2
Closed
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
48 changes: 48 additions & 0 deletions
48
nemoguardrails/server/experimental/contracts/openai/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| # OpenAI Chat Completions contract | ||
|
|
||
| Operation `createChatCompletion` uses the `single_text.v1` capability profile | ||
| for its guarded text boundary. This page summarizes that boundary and its | ||
| provider source. Neither the summary nor the source pin enables runtime | ||
| behavior. See [Guard contracts](../README.md) for the document vocabulary. | ||
|
|
||
| ## Guarded boundary | ||
|
|
||
| | Area | Single-text boundary | | ||
| | --- | --- | | ||
| | Request | One user message with non-empty string content at `messages[0].content`. | | ||
| | Buffered response | One assistant choice with non-empty string content at `choices[0].message.content`. | | ||
| | Constrained values | User/assistant roles, single-item arrays, and request `n` constrained to one. | | ||
| | Unsupported content | Tool, audio, multimodal, refusal, and separate reasoning content where explicitly disabled. | | ||
| | Opaque data | Reviewed provider-owned metadata and controls, not additional guarded subjects. | | ||
|
|
||
| This request boundary does not accept conversation histories, system/developer | ||
| messages alongside the user message, or multimodal content blocks. Disabled | ||
| fields are null-only: a non-null value does not become acceptable merely because | ||
| it requests plain text or no tools. | ||
|
|
||
| Request `logprobs` and `top_logprobs` settings are provider-owned, while non-null | ||
| buffered response `choices[0].logprobs` is unsupported. That asymmetry is a | ||
| boundary limitation, not something document validation resolves. | ||
|
|
||
| Text replacement eligibility is separate from endpoint support for replacement | ||
| outcomes. Non-empty response annotations block text replacement; their mere | ||
| presence does not. Unrelated provider data must remain intact. | ||
|
|
||
| The Python projections, bindings, and endpoint determine exact acceptance and | ||
| runtime behavior. Their machine-readable contract belongs with the integration, | ||
| not in a separately maintained handwritten policy here. Recognizing the request | ||
| `stream` flag does not itself provide a streaming endpoint. Streaming needs its | ||
| own event classification, lifecycle handling, and implementation tests; the | ||
| buffered boundary is not a claim about accepted stream events. | ||
|
|
||
| ## Provider provenance | ||
|
|
||
| [source.yaml](source.yaml) records the OpenAPI document's immutable revision, | ||
| document version, download location, and expected SHA-256 digest. The pin is a | ||
| review baseline, not a claim to cover every current OpenAI field or compatible | ||
| provider. | ||
|
|
||
| The offline metadata tests check pin structure and URL/revision consistency. | ||
| They do not fetch the document or verify its bytes against the digest. Reviewing | ||
| a provider update requires checking the source artifact and changes to guarded, | ||
| constrained, disabled, and opaque fields separately from format validation. | ||
5 changes: 5 additions & 0 deletions
5
nemoguardrails/server/experimental/contracts/openai/source.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| document_url: https://github.com/openai/openai-openapi/blob/df63773f69f542ef875b9f00c3837c25ba5f4f2a/openapi.yaml | ||
| download_url: https://raw.githubusercontent.com/openai/openai-openapi/df63773f69f542ef875b9f00c3837c25ba5f4f2a/openapi.yaml | ||
| revision: df63773f69f542ef875b9f00c3837c25ba5f4f2a | ||
| document_version: 2.3.0 | ||
| sha256: f2dae1a9aced09b91310db89edda51bf1e36ecbfb05230c3a50b239c07708469 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| """Validate pinned provider metadata without contacting the provider.""" | ||
|
|
||
| import re | ||
| from pathlib import Path | ||
| from urllib.parse import urlparse | ||
|
|
||
| import pytest | ||
| import yaml | ||
|
|
||
| REPOSITORY_ROOT = Path(__file__).parents[3] | ||
| SOURCE_PATH = REPOSITORY_ROOT / "nemoguardrails/server/experimental/contracts/openai/source.yaml" | ||
|
|
||
|
|
||
| @pytest.fixture(scope="module") | ||
| def openai_source() -> dict[str, str]: | ||
| """Load source provenance, not a separately authored operation policy.""" | ||
| source = yaml.safe_load(SOURCE_PATH.read_text(encoding="utf-8")) | ||
| assert isinstance(source, dict) | ||
| return source | ||
|
|
||
|
|
||
| def test_openai_source_declares_revision_and_digest(openai_source: dict[str, str]) -> None: | ||
| """Pin metadata has the expected shape; this does not verify upstream bytes.""" | ||
| assert set(openai_source) == { | ||
| "document_url", | ||
| "download_url", | ||
| "revision", | ||
| "document_version", | ||
| "sha256", | ||
| } | ||
| assert re.fullmatch(r"[0-9a-f]{40}", openai_source["revision"]) | ||
| assert re.fullmatch(r"[0-9a-f]{64}", openai_source["sha256"]) | ||
| assert openai_source["document_version"].strip() | ||
|
|
||
|
|
||
| @pytest.mark.parametrize( | ||
| ("key", "host", "prefix"), | ||
| [ | ||
| ("document_url", "github.com", "/openai/openai-openapi/blob"), | ||
| ("download_url", "raw.githubusercontent.com", "/openai/openai-openapi"), | ||
| ], | ||
| ) | ||
| def test_openai_source_urls_use_the_declared_revision( | ||
| openai_source: dict[str, str], key: str, host: str, prefix: str | ||
| ) -> None: | ||
| """Both locations identify the same pinned file rather than a moving branch.""" | ||
| url = urlparse(openai_source[key]) | ||
|
|
||
| assert url.scheme == "https" | ||
| assert url.netloc == host | ||
| assert url.path == f"{prefix}/{openai_source['revision']}/openapi.yaml" | ||
| assert not url.query | ||
| assert not url.fragment |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This revision removes
chat-completions.guard.yamland says the machine-readable contract belongs with the integration, but PR #2435 modifies that file rather than adding it. Its streaming test reads the file directly, so stacking that PR on this head leaves the file missing and the test fails. The buffered export in PR #2414 does not provide the stream section the test needs.Prompt To Fix With AI