[Pouyanpi]

Summary

Adds an explicitly selected LiteLLM Chat dialect for reviewed buffered and
streaming wire shapes while retaining the single-text guardrail profile.

Why

LiteLLM can add provider-specific extension fields to otherwise OpenAI-compatible
responses. Those fields need a reviewed policy so inspected exchanges can accept
supported shapes without widening the default OpenAI declaration.

How it works

Typed Python projections define the dialect. Message and choice extension
containers remain closed; null refusal, empty content arrays, and reviewed
native finish metadata have individual policies. Streaming uses rebound
projections and the existing OpenAI event semantics and lifecycle hooks.

A trusted application explicitly supplies the endpoint declaration. Omitting
it retains the default OpenAI policy. Hostnames, model strings, response fields,
and exported YAML do not select a dialect.

What Changed

• Adds reviewed buffered message and choice extensions with closed containers.
• Recognizes empty streaming extension bags and empty content arrays without accepting unsupported content.
• Derives coverage, text locations, and contract export from the bound declarations.
• Reuses the OpenAI request boundary, native errors, and per-stream lifecycle hooks.
• Adds acceptance, rejection, original-byte preservation, and export tests.
• Documents immutable LiteLLM source evidence and the limits of the supported subset.

Review Notes

Review the field policy and explicit selection boundary. This is a reviewed
LiteLLM subset, not compatibility with every release, backend, or gateway.
No additional request controls or arbitrary extension contents are enabled.
No LiteLLM runtime dependency or compiler machinery is introduced.

AI tools assisted with implementation and review.

Summary by CodeRabbit

• New Features
  • Added an experimental LiteLLM chat option that can be selected explicitly when configuring the OpenAI-compatible endpoint.
  • Added support for LiteLLM request, response, and streaming formats with strict checks on accepted content. Valid provider responses are preserved, while unreviewed content is rejected.
• Documentation
  • Documented supported formats, endpoint selection, streaming behavior, and contract export guidance.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no blocking issues were found.

Summary

Adds an explicitly selected LiteLLM Chat dialect without changing the default OpenAI policy.

• Applications can explicitly use LiteLLM Chat replies with guardrails.
• LiteLLM streams keep OpenAI text checks while accepting empty extension fields.

No actionable issues were found. Tests were inspected but not run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Trusted application] --> B{Endpoint supplied?}
  B -->|No| C[Default OpenAI declaration]
  B -->|LiteLLM| D[Reviewed LiteLLM declaration]
  C --> E[Shared request checks]
  D --> E
  E --> F[Upstream provider]
  F --> G[Selected response or stream checks]
  G --> H[Return allowed original bytes]

Loading

_{Reviews (3) · Last reviewed commit: "docs(server): remove local planning deta..." · Reviewed by Greptile}

[m-misiura]

LGTM :)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA-NeMo/Guardrails/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 7326e7a9-0ce2-4584-9151-55471a7651e8

📥 Commits

Reviewing files that changed from the base of the PR and between fc75289 and 1cb60a4.

⛔ Files ignored due to path filters: 1

• nemoguardrails/server/experimental/contracts/openai/_generated/litellm-chat-completions.guard.yaml is excluded by !**/_generated/**

📒 Files selected for processing: 8

• nemoguardrails/server/experimental/contracts/openai/README.md
• nemoguardrails/server/experimental/providers/openai/dialects/__init__.py
• nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md
• nemoguardrails/server/experimental/providers/openai/dialects/litellm/__init__.py
• nemoguardrails/server/experimental/providers/openai/dialects/litellm/chat_completions.py
• nemoguardrails/server/experimental/providers/openai/dialects/litellm/stream.py
• nemoguardrails/server/experimental/providers/openai/integration.py
• tests/server/experimental/test_litellm_chat_dialect.py

📝 Walkthrough

This change adds an experimental LiteLLM Chat dialect with constrained buffered and streaming response policies. The OpenAI router accepts an optional endpoint so applications can select the dialect explicitly. Tests cover policy validation, contract export, and router behavior.

Changes: LiteLLM Chat Dialect

Layer / File(s)	Summary
Buffered response policy nemoguardrails/server/experimental/providers/openai/dialects/litellm/chat_completions.py, nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md, tests/server/experimental/test_litellm_chat_dialect.py	Adds a guarded response projection with one choice and constrained message and choice fields. Tests check accepted empty fields, opaque metadata, and rejection of unreviewed response fields.
Streaming response policy nemoguardrails/server/experimental/providers/openai/dialects/litellm/stream.py, tests/server/experimental/test_litellm_chat_dialect.py	Adds LiteLLM stream projections and a policy-based classifier. Tests check extension and content constraints, classifier behavior, and provider errors.
Endpoint selection and validation nemoguardrails/server/experimental/providers/openai/integration.py, nemoguardrails/server/experimental/providers/openai/dialects/*, nemoguardrails/server/experimental/providers/openai/dialects/litellm/README.md, nemoguardrails/server/experimental/contracts/openai/README.md, tests/server/experimental/test_litellm_chat_dialect.py	Adds an optional router endpoint while retaining the OpenAI error mapping and OpenAI endpoint as the default. Tests cover explicit dialect selection, contract export, request validation, and buffered and streaming requests.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Application
  participant OpenAIRouter as create_openai_chat_router
  participant GuardedOperation
  participant Checker as ContentChecker
  participant Dispatch as HttpDispatch
  Application->>OpenAIRouter: Supply LiteLLM endpoint
  OpenAIRouter->>GuardedOperation: Create operation with endpoint and OpenAI error mapping
  GuardedOperation->>Checker: Inspect request when inspection is enabled
  GuardedOperation->>Dispatch: Forward request
  Dispatch-->>GuardedOperation: Return provider response
  GuardedOperation->>Checker: Inspect response when inspection is enabled

Loading

Merge Risk: ⚪ Minimal · up to 1cb60

The reviewed LiteLLM dialect change is ready to merge after normal checks; no actionable merge-blocking risk remains.

Pre-merge checks: 5 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 6.06% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 33 functions across 6 files. (2 skipped: 2…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 5 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: adding a reviewed LiteLLM Chat dialect to the server.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes		The PR is a major feature change: it adds a LiteLLM dialect, endpoint selection, generated contract, and 433 lines of focused tests. The PR description documents testing information, including accepta…

Full details: Test Results For Major Changes

Explanation

The PR is a major feature change: it adds a LiteLLM dialect, endpoint selection, generated contract, and 433 lines of focused tests. The PR description documents testing information, including acceptance, rejection, byte-preservation, and export tests. The supplied objectives also record 678 focused offline tests and pre-commit validation. This satisfies the check.

Full details: Docstring Coverage

Explanation

Docstring coverage is 6.06% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 33 functions across 6 files. (2 skipped: 2 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

• Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}