Add MCP ToolHive examples

README.md

@@ -10,6 +10,7 @@ For full platform documentation, see [docs.prokube.ai](https://docs.prokube.ai/)
 ├── hparam-tuning    # hyperparameter tuning examples (Katib)
 ├── images           # custom container images used by examples
 ├── mlflow           # MLflow experiment tracking examples
+├── mcp-servers      # MCP server examples with ToolHive
 ├── notebooks        # Jupyter notebook examples (Dask, MNIST VAE, etc.)
 ├── pipelines        # Kubeflow Pipelines examples
 ├── rstudio          # RStudio examples
@@ -41,6 +42,7 @@ Some examples require a minimum prokube platform version. If an example is not l
 
 | Example | Min platform version | Notes |
 |---|---|---|
+| `mcp-servers` | TBD | Requires MCP servers / ToolHive support |
 | `serving/minimal-s3-model` | v1.7.0 | Requires s3creds secret with KServe support |
 
 ## Contributing

mcp-servers/README.md

@@ -0,0 +1,15 @@
+# MCP server examples
+
+These examples show how to run MCP servers on prokube with ToolHive.
+
+- [`deploy-upstream-mcp-server`](./deploy-upstream-mcp-server/): deploy an existing upstream MCP server image with a ToolHive `MCPServer` resource.
+- [`build-custom-mcp-server`](./build-custom-mcp-server/): build and deploy a small custom FastMCP server image.
+
+Use the prokube **MCP** UI for the normal workflow. These examples are useful when you want to understand the underlying Kubernetes resources, test GitOps-style deployment, or build your own MCP server image.
+
+## Prerequisites
+
+- Access to a prokube workspace with MCP servers enabled.
+- A prokube Lab or notebook running in the target workspace.
+- `kubectl` configured for the current workspace namespace.
+- Permission to create ToolHive `MCPServer` resources in the workspace namespace.

mcp-servers/build-custom-mcp-server/README.md

@@ -0,0 +1,57 @@
+# Build and deploy a custom MCP server
+
+This example builds a small FastMCP server for workspace-specific operational runbooks.
+
+The server stores Markdown files in `/data/runbooks`, which is backed by a PVC in `workspace-runbooks.yaml`. It does not use RAG, embeddings, or a vector database. Search is simple case-insensitive text matching over Markdown files.
+
+The example demonstrates:
+
+- a readable custom MCP server implementation;
+- a non-root container image;
+- read-only root filesystem compatibility;
+- persistent workspace data;
+- seeding initial Markdown files into the PVC on first start.
+
+## Tools
+
+The server exposes:
+
+- `list_runbooks`
+- `get_runbook`
+- `save_runbook`
+- `search_runbooks`
+- `delete_runbook`
+
+## How runbooks get into the server
+
+There are two paths:
+
+- Seed files in `runbook-server/seed-runbooks/` are copied into `/data/runbooks` when the server starts and the PVC is empty.
+- MCP clients can create or update runbooks later with the `save_runbook` tool.
+
+Because `/data` is backed by a PVC, runbooks created through MCP tools survive pod restarts.
+
+## Build and push the image
+
+```bash
+export IMAGE=<registry>/<project>/workspace-runbooks-mcp:0.1.0
+docker build -t "$IMAGE" runbook-server
+docker push "$IMAGE"
+```
+
+## Deploy
+
+Replace `IMAGE_PLACEHOLDER` with the pushed image and apply the manifest:
+
+```bash
+sed "s|IMAGE_PLACEHOLDER|$IMAGE|g" workspace-runbooks.yaml | kubectl apply -f -
+kubectl get mcpservers.toolhive.stacklok.dev
+```
+
+After the server is running, open **MCP** in the prokube UI and copy the server URL from the deployed servers table or details page.
+
+## Clean up
+
+```bash
+sed "s|IMAGE_PLACEHOLDER|$IMAGE|g" workspace-runbooks.yaml | kubectl delete -f -
+```

mcp-servers/build-custom-mcp-server/runbook-server/Dockerfile

@@ -0,0 +1,19 @@
+FROM python:3.12-slim
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1 \
+    RUNBOOK_DIR=/data/runbooks \
+    SEED_RUNBOOK_DIR=/app/seed-runbooks
+
+WORKDIR /app
+
+COPY requirements.txt ./
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY server.py ./server.py
+COPY seed-runbooks ./seed-runbooks
+
+RUN useradd --create-home --uid 10001 appuser
+USER 10001
+
+ENTRYPOINT ["python", "server.py"]

mcp-servers/build-custom-mcp-server/runbook-server/requirements.txt

@@ -0,0 +1 @@
+fastmcp>=3.4,<4

mcp-servers/build-custom-mcp-server/runbook-server/seed-runbooks/model-endpoint-503.md

@@ -0,0 +1,17 @@
+# Model Endpoint 503
+
+## Symptoms
+
+- Client requests to a model endpoint return `503`.
+- The endpoint is listed in the UI, but predictions fail.
+
+## Checks
+
+1. Check whether the backing pod is running.
+2. Check recent pod events for image pull, quota, or readiness errors.
+3. Check the model server logs.
+4. Confirm that the endpoint is not scaling from zero and still warming up.
+
+## Escalate
+
+Escalate to the platform owner if pods cannot be scheduled because of quota or node capacity.

mcp-servers/build-custom-mcp-server/runbook-server/seed-runbooks/pipeline-quota.md

@@ -0,0 +1,17 @@
+# Pipeline Quota
+
+## Symptoms
+
+- Pipeline steps stay pending.
+- New pods are not created in the workspace namespace.
+
+## Checks
+
+1. Check workspace pod quota in the prokube UI.
+2. Delete completed pods if the workspace is at its pod limit.
+3. Check whether other workloads are consuming CPU or memory requests.
+4. Retry the run after capacity is available.
+
+## Escalate
+
+Escalate if active workloads are expected and the workspace needs a quota increase.

mcp-servers/build-custom-mcp-server/runbook-server/server.py

@@ -0,0 +1,95 @@
+import os
+import re
+import shutil
+from pathlib import Path
+
+from fastmcp import FastMCP
+
+
+RUNBOOK_DIR = Path(os.environ.get("RUNBOOK_DIR", "/data/runbooks"))
+SEED_RUNBOOK_DIR = Path(os.environ.get("SEED_RUNBOOK_DIR", "/app/seed-runbooks"))
+
+RUNBOOK_DIR.mkdir(parents=True, exist_ok=True)
+
+mcp = FastMCP("workspace-runbooks")
+
+
+def _seed_runbooks() -> None:
+    if any(RUNBOOK_DIR.glob("*.md")) or not SEED_RUNBOOK_DIR.exists():
+        return
+    for source in SEED_RUNBOOK_DIR.glob("*.md"):
+        shutil.copyfile(source, RUNBOOK_DIR / source.name)
+
+
+def _safe_name(name: str) -> str:
+    slug = name.removesuffix(".md").strip().lower()
+    slug = re.sub(r"[^a-z0-9._-]+", "-", slug).strip("-._")
+    if not slug:
+        raise ValueError("Runbook name must contain at least one letter or number")
+    return f"{slug}.md"
+
+
+def _runbook_path(name: str) -> Path:
+    path = (RUNBOOK_DIR / _safe_name(name)).resolve()
+    if RUNBOOK_DIR.resolve() not in path.parents:
+        raise ValueError("Invalid runbook path")
+    return path
+
+
+@mcp.tool
+def list_runbooks() -> list[str]:
+    """List available runbook names."""
+    return sorted(path.stem for path in RUNBOOK_DIR.glob("*.md"))
+
+
+@mcp.tool
+def get_runbook(name: str) -> str:
+    """Return the Markdown content of a runbook."""
+    path = _runbook_path(name)
+    if not path.exists():
+        raise ValueError(f"Runbook not found: {name}")
+    return path.read_text(encoding="utf-8")
+
+
+@mcp.tool
+def save_runbook(name: str, content: str) -> str:
+    """Create or replace a runbook with Markdown content."""
+    path = _runbook_path(name)
+    path.write_text(content.rstrip() + "\n", encoding="utf-8")
+    return path.stem
+
+
+@mcp.tool
+def search_runbooks(query: str) -> list[dict[str, str]]:
+    """Search runbook names and content with simple case-insensitive text matching."""
+    needle = query.strip().lower()
+    if not needle:
+        raise ValueError("Query must not be empty")
+
+    matches: list[dict[str, str]] = []
+    for path in sorted(RUNBOOK_DIR.glob("*.md")):
+        content = path.read_text(encoding="utf-8")
+        haystack = f"{path.stem}\n{content}".lower()
+        if needle not in haystack:
+            continue
+        lines = [line.strip() for line in content.splitlines() if needle in line.lower()]
+        matches.append({
+            "name": path.stem,
+            "snippet": lines[0] if lines else content[:160],
+        })
+    return matches
+
+
+@mcp.tool
+def delete_runbook(name: str) -> bool:
+    """Delete a runbook. Returns true when a file was deleted."""
+    path = _runbook_path(name)
+    if not path.exists():
+        return False
+    path.unlink()
+    return True
+
+
+if __name__ == "__main__":
+    _seed_runbooks()
+    mcp.run()

mcp-servers/build-custom-mcp-server/workspace-runbooks.yaml

@@ -0,0 +1,45 @@
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: workspace-runbooks-data
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: 1Gi
+---
+apiVersion: toolhive.stacklok.dev/v1beta1
+kind: MCPServer
+metadata:
+  name: workspace-runbooks
+spec:
+  image: IMAGE_PLACEHOLDER
+  transport: stdio
+  proxyPort: 8080
+  resources:
+    requests:
+      cpu: 50m
+      memory: 128Mi
+    limits:
+      cpu: 250m
+      memory: 256Mi
+  podTemplateSpec:
+    spec:
+      volumes:
+        - name: runbooks
+          persistentVolumeClaim:
+            claimName: workspace-runbooks-data
+      containers:
+        - name: mcp
+          volumeMounts:
+            - name: runbooks
+              mountPath: /data
+          securityContext:
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+            runAsNonRoot: true
+            runAsUser: 10001
+            capabilities:
+              drop:
+                - ALL

mcp-servers/deploy-upstream-mcp-server/README.md

@@ -0,0 +1,20 @@
+# Deploy an upstream MCP server
+
+This example deploys the upstream Time MCP server image with a ToolHive `MCPServer` resource.
+
+```bash
+kubectl apply -f time-server.yaml
+kubectl get mcpservers.toolhive.stacklok.dev
+```
+
+If your kubeconfig can access multiple namespaces, set the target workspace namespace explicitly, for example with `kubectl --namespace <workspace-namespace> apply -f time-server.yaml`.
+
+The server exposes the `get_current_time` and `convert_time` tools.
+
+The current upstream image requires root because of its image layout. Use this as a minimal ToolHive smoke test, not as a production baseline.
+
+## Clean up
+
+```bash
+kubectl delete -f time-server.yaml
+```

mcp-servers/deploy-upstream-mcp-server/time-server.yaml

@@ -0,0 +1,16 @@
+apiVersion: toolhive.stacklok.dev/v1beta1
+kind: MCPServer
+metadata:
+  name: time
+spec:
+  image: docker.io/mcp/time:latest@sha256:9c46a918633fb474bf8035e3ee90ebac6bcf2b18ccb00679ac4c179cba0ebfcf
+  transport: stdio
+  proxyPort: 8080
+  podTemplateSpec:
+    spec:
+      containers:
+        - name: mcp
+          securityContext:
+            runAsUser: 0
+            runAsNonRoot: false
+            readOnlyRootFilesystem: false