Modal OpenAI-Compatible Proxy

A lightweight serverless proxy that allows OpenAI-compatible applications to securely access private Modal-hosted AI models.

Why This Exists

Many modern AI applications support custom OpenAI-compatible endpoints but only allow a single API key for authentication.

Modal's protected endpoints require custom authentication headers:

• Modal-Key
• Modal-Secret

which are not supported by many clients.

This project bridges that gap by exposing a standard OpenAI-compatible API while securely handling Modal authentication on the server side.

The result is a reusable gateway that allows private Modal-hosted models to be used from any OpenAI-compatible application without exposing Modal credentials.

---

Primary Use Cases

This project was built to support a variety of personal AI workflows, including:

• AI meeting assistants
• Meeting transcription analysis
• Action item extraction
• Document summarization
• Personal chatbots
• Knowledge assistants
• Retrieval-Augmented Generation (RAG)
• Custom AI applications built on private Modal deployments

---

Architecture

OpenAI-Compatible Client
        │
        │ Bearer API Key
        ▼
Modal OpenAI Proxy
        │
        ├── Validates API Key
        ├── Injects Modal Authentication
        └── Forwards Requests
        ▼
Modal Endpoint
        ▼
Gemma / Qwen / Llama / Custom Models

---

Features

• OpenAI-compatible Chat Completions API
• Works with any OpenAI-compatible client
• Supports private Modal endpoints
• Server-side credential management
• Bearer token authentication
• Streaming support
• Serverless deployment
• Scale-to-zero friendly
• Simple and lightweight

---

Environment Variables

Variable	Description
PROXY_API_KEY	API key used by client applications
MODAL_ENDPOINT_URL	Full Modal endpoint URL
MODAL_KEY	Modal authentication key
MODAL_SECRET	Modal authentication secret

Example:

PROXY_API_KEY=sk-my-private-key
MODAL_ENDPOINT_URL=https://your-endpoint.modal.direct/v1/chat/completions
MODAL_KEY=wk_xxxxxxxxx
MODAL_SECRET=ws_xxxxxxxxx

---

Deployment

Deploy directly to Vercel:

1. Clone the repository
2. Import into Vercel
3. Configure environment variables
4. Deploy

No additional infrastructure is required.

---

Example Client Configuration

Base URL:
https://your-deployment.vercel.app/v1

API Key:
sk-my-private-key

Model:
google/gemma-4-E4B-it

---

Example Request

curl -X POST \
  https://your-deployment.vercel.app/v1/chat/completions \
  -H "Authorization: Bearer sk-my-private-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-4-E4B-it",
    "messages": [
      {
        "role": "user",
        "content": "Summarize this document."
      }
    ]
  }'

---

Future Plans

• Multi-model routing
• Model fallback support
• Request analytics
• Rate limiting
• RAG integrations
• Long-term memory support
• Tool calling support
• Multi-user authentication

---

License

MIT