feat: add OpenAI-compatible HTTP server for mellea backends

docs/examples/openai_compat_server/README.md

@@ -0,0 +1,137 @@
+# OpenAI-Compatible Server Examples
+
+This directory contains examples for using mellea's OpenAI-compatible HTTP server.
+
+## Overview
+
+The OpenAI-compatible server allows you to wrap any mellea backend and expose it through OpenAI's API format. This means you can use the official OpenAI Python SDK or any other OpenAI-compatible client to interact with local models, Ollama, or other backends supported by mellea.
+
+## Files
+
+- **`basic_server.py`**: Shows how to start the server with basic configuration
+- **`client_example.py`**: Demonstrates using the OpenAI Python SDK to interact with the server
+- **`curl_examples.sh`**: Command-line examples using curl
+
+## Quick Start
+
+### 1. Start the Server
+
+```python
+from mellea.integrations.openai_compat.server import ServerConfig, run_server
+
+config = ServerConfig(
+    backend_name="ollama",
+    model_id="granite4:micro",
+)
+
+run_server(host="0.0.0.0", port=8000, config=config)
+```
+
+Or run the example:
+
+```bash
+uv run python docs/examples/openai_compat_server/basic_server.py
+```
+
+### 2. Use the OpenAI SDK
+
+```python
+import openai
+
+client = openai.OpenAI(
+    base_url="http://localhost:8000/v1",
+    api_key="not-needed",
+)
+
+response = client.chat.completions.create(
+    model="granite4:micro",
+    messages=[
+        {"role": "user", "content": "Hello!"}
+    ],
+)
+
+print(response.choices[0].message.content)
+```
+
+### 3. Or Use curl
+
+```bash
+curl http://localhost:8000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "granite4:micro",
+    "messages": [{"role": "user", "content": "Hello!"}]
+  }'
+```
+
+## Supported Endpoints
+
+- **`POST /v1/chat/completions`**: Chat completions (with streaming support)
+- **`GET /v1/models`**: List available models
+- **`GET /health`**: Health check
+
+## Interactive API Documentation
+
+FastAPI automatically provides interactive API documentation when the server is running:
+
+- **Swagger UI**: http://localhost:8000/docs
+- **ReDoc**: http://localhost:8000/redoc
+- **OpenAPI JSON**: http://localhost:8000/openapi.json
+
+Visit the Swagger UI to explore all endpoints, view request/response schemas, and test the API directly from your browser.
+
+## Configuration Options
+
+The `ServerConfig` class accepts the following parameters:
+
+- **`backend_name`**: Name of the mellea backend (e.g., "ollama", "openai", "hf")
+- **`model_id`**: Model identifier to use
+- **`base_url`**: Base URL for the backend API (if applicable)
+- **`api_key`**: API key for the backend (if applicable)
+- **`default_model_options`**: Default model options for all requests
+- **`**backend_kwargs`**: Additional arguments passed to the backend
+
+## Supported Features
+
+✅ Chat completions
+✅ Streaming responses
+✅ System messages
+✅ Temperature, top_p, max_tokens
+✅ Seed for reproducibility
+✅ Tool calling (when supported by backend)
+✅ Token usage tracking
+
+## Backend Support
+
+The server works with any mellea backend:
+
+- **Ollama**: Local models via Ollama
+- **OpenAI**: OpenAI API models
+- **HuggingFace**: Local HuggingFace models
+- **Watsonx**: IBM Watsonx models
+- **LiteLLM**: Any LiteLLM-supported provider
+
+## Deployment
+
+The server can be deployed as a standalone service:
+
+```bash
+# Using uvicorn directly
+uvicorn mellea.integrations.openai_compat.server:app --host 0.0.0.0 --port 8000
+
+# Or using the run_server function
+python -c "from mellea.integrations.openai_compat.server import run_server; run_server()"
+```
+
+## Testing
+
+Run the tests:
+
+```bash
+uv run pytest test/integrations/test_openai_compat_server.py
+```
+
+Run the client example (requires server to be running):
+
+```bash
+uv run python docs/examples/openai_compat_server/client_example.py

docs/examples/openai_compat_server/basic_server.py

@@ -0,0 +1,61 @@
+# pytest: skip_always
+"""Basic example of running the OpenAI-compatible server with mellea.
+
+This example demonstrates how to start a server that wraps a mellea backend
+and exposes it via OpenAI's API format.
+"""
+
+import socket
+
+from mellea.integrations.openai_compat.server import ServerConfig, run_server
+
+
+def find_unused_port(start_port: int = 8000, max_attempts: int = 100) -> int:
+    """Find an unused port starting from start_port.
+
+    Args:
+        start_port: Port to start searching from.
+        max_attempts: Maximum number of ports to try.
+
+    Returns:
+        An available port number.
+
+    Raises:
+        RuntimeError: If no available port is found.
+    """
+    for port in range(start_port, start_port + max_attempts):
+        try:
+            with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+                s.bind(("", port))
+                return port
+        except OSError:
+            continue
+    raise RuntimeError(
+        f"Could not find an unused port in range {start_port}-{start_port + max_attempts}"
+    )
+
+
+def main():
+    """Run the OpenAI-compatible server with Ollama backend."""
+    # Configure the server to use Ollama with granite4:micro
+    config = ServerConfig(
+        backend_name="ollama",
+        model_id="granite4:micro",
+        default_model_options={"temperature": 0.7},
+    )
+
+    # Find an available port starting from 8000
+    port = find_unused_port(start_port=8000)
+
+    # Start the server
+    print(f"Starting OpenAI-compatible server on http://localhost:{port}")
+    print(f"API endpoint: http://localhost:{port}/v1/chat/completions")
+    print(f"Models endpoint: http://localhost:{port}/v1/models")
+    print(f"Health check: http://localhost:{port}/health")
+    print(f"API docs: http://localhost:{port}/docs")
+
+    run_server(host="0.0.0.0", port=port, config=config)
+
+
+if __name__ == "__main__":
+    main()

docs/examples/openai_compat_server/client_example.py

@@ -0,0 +1,68 @@
+# pytest: skip_always
+"""Example of using the OpenAI Python SDK with the mellea server.
+
+This example shows how to use the official OpenAI Python SDK to interact
+with a mellea backend through the OpenAI-compatible server.
+"""
+
+import openai
+
+
+def test_openai_sdk_compatibility():
+    """Test that the OpenAI SDK works with mellea server."""
+    # Configure the OpenAI client to point to our mellea server
+    client = openai.OpenAI(
+        base_url="http://localhost:8000/v1",
+        api_key="not-needed",  # mellea server doesn't require auth by default
+    )
+
+    # Make a simple chat completion request
+    response = client.chat.completions.create(
+        model="granite4:micro",
+        messages=[
+            {"role": "system", "content": "You are a helpful assistant."},
+            {"role": "user", "content": "What is the capital of France?"},
+        ],
+        temperature=0.7,
+        max_tokens=50,
+    )
+
+    # Access the response
+    print(f"Response: {response.choices[0].message.content}")
+    assert response.choices[0].message.content is not None
+    assert len(response.choices[0].message.content) > 0
+
+
+def test_streaming_with_openai_sdk():
+    """Test streaming responses with the OpenAI SDK."""
+    client = openai.OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
+
+    # Make a streaming request
+    stream = client.chat.completions.create(
+        model="granite4:micro",
+        messages=[{"role": "user", "content": "Count from 1 to 5"}],
+        stream=True,
+        max_tokens=50,
+    )
+
+    # Collect streamed chunks
+    full_response = ""
+    for chunk in stream:
+        if chunk.choices[0].delta.content is not None:
+            content = chunk.choices[0].delta.content
+            full_response += content
+            print(content, end="", flush=True)
+
+    print()  # New line after streaming
+    assert len(full_response) > 0
+
+
+if __name__ == "__main__":
+    print("Testing OpenAI SDK compatibility...")
+    print("\n1. Basic completion:")
+    test_openai_sdk_compatibility()
+
+    print("\n2. Streaming completion:")
+    test_streaming_with_openai_sdk()
+
+    print("\nAll tests passed!")

docs/examples/openai_compat_server/curl_examples.sh

@@ -0,0 +1,93 @@
+#!/bin/bash
+# Example curl commands for the OpenAI-compatible server
+
+# Base URL for the server
+BASE_URL="http://localhost:8000"
+
+echo "=== OpenAI-Compatible Server Examples ==="
+echo ""
+
+# 1. Health check
+echo "1. Health Check:"
+curl -s "${BASE_URL}/health" | jq .
+echo ""
+
+# 2. List models
+echo "2. List Models:"
+curl -s "${BASE_URL}/v1/models" | jq .
+echo ""
+
+# 3. Basic chat completion
+echo "3. Basic Chat Completion:"
+curl -s "${BASE_URL}/v1/chat/completions" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "granite4:micro",
+    "messages": [
+      {"role": "user", "content": "What is 2+2?"}
+    ],
+    "max_tokens": 50
+  }' | jq .
+echo ""
+
+# 4. Chat completion with system message
+echo "4. Chat Completion with System Message:"
+curl -s "${BASE_URL}/v1/chat/completions" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "granite4:micro",
+    "messages": [
+      {"role": "system", "content": "You are a helpful math tutor."},
+      {"role": "user", "content": "Explain what a prime number is."}
+    ],
+    "temperature": 0.7,
+    "max_tokens": 100
+  }' | jq .
+echo ""
+
+# 5. Streaming chat completion
+echo "5. Streaming Chat Completion:"
+curl -s "${BASE_URL}/v1/chat/completions" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "granite4:micro",
+    "messages": [
+      {"role": "user", "content": "Count from 1 to 5"}
+    ],
+    "stream": true,
+    "max_tokens": 50
+  }'
+echo ""
+
+# 6. Chat completion with seed for reproducibility
+echo "6. Chat Completion with Seed:"
+curl -s "${BASE_URL}/v1/chat/completions" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "granite4:micro",
+    "messages": [
+      {"role": "user", "content": "Pick a random number"}
+    ],
+    "seed": 42,
+    "max_tokens": 20
+  }' | jq .
+echo ""
+
+# 7. Chat completion with custom parameters
+echo "7. Chat Completion with Custom Parameters:"
+curl -s "${BASE_URL}/v1/chat/completions" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "granite4:micro",
+    "messages": [
+      {"role": "user", "content": "Write a haiku about coding"}
+    ],
+    "temperature": 0.9,
+    "top_p": 0.95,
+    "max_tokens": 100,
+    "presence_penalty": 0.5,
+    "frequency_penalty": 0.5
+  }' | jq .
+echo ""
+
+echo "=== All examples completed ==="

mellea/integrations/openai_compat/__init__.py

@@ -0,0 +1,5 @@
+"""OpenAI API-compatible server for mellea."""
+
+from .server import create_app, run_server
+
+__all__ = ["create_app", "run_server"]