diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000..618967e --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,7 @@ +# Agent Workflow Guidelines + +## Planning Requirements +- **Strict Rule**: Always use workspace search and exploration tools to find and read all *relevant* context files before proposing changes. +- Do not make or suggest code modifications immediately. +- First, return a detailed plan of action based on the files read, outlining the proposed changes step-by-step. +- Explicitly wait for the user's approval on the plan before proceeding with implementation. diff --git a/README.md b/README.md index ed7fe8c..2ebb9d2 100644 --- a/README.md +++ b/README.md @@ -206,6 +206,17 @@ docker compose -f docker_compose/docker-compose.yaml up -d --pull always --- +## 🔌 Extensions & Integrations + +Razer AIKit supports modular extensions to enhance its core capabilities. + +### FoundLab ATI (Algorithmic Trust Indicator) +A lightweight, zero-persistence cryptographic middleware that provides enterprise-grade auditability for AI models (LGPD, EU AI Act, BCB 538 compliant). It automatically appends a verifiable signature to AI outputs without writing any data to disk. +- [FoundLab ATI Documentation](extensions/foundlab-ati/README.md) +- [Interactive Guide](extensions/foundlab-ati/example_integration.ipynb) + +--- + ## 🖥️ Platform Support Razer AIKit is optimized for NVIDIA accelerated computing platforms with support for both x86-64 and ARM64 architectures. diff --git a/extensions/foundlab-ati/README.md b/extensions/foundlab-ati/README.md new file mode 100644 index 0000000..6083324 --- /dev/null +++ b/extensions/foundlab-ati/README.md @@ -0,0 +1,50 @@ +# FoundLab ATI (Algorithmic Trust Indicator) Extension for Razer AIKit + +This extension provides a lightweight, **zero-persistence cryptographic middleware** for the Razer AIKit. It is designed to natively integrate with vLLM and Open WebUI to append a mathematically verifiable signature to AI-generated responses. + +## Regulatory Compliance +By providing a cryptographic tie between the user's input prompt and the model's generated output, the FoundLab ATI extension provides critical transparency and accountability for AI deployments, directly addressing: +- **LGPD (Lei Geral de Proteção de Dados)**: By ensuring the keys are ephemeral and no PII or logs are written to disk, it achieves "privacy by design." +- **EU AI Act**: Provides traceability and explainability, functioning as a technical standard for transparency in high-risk AI models. +- **BCB 538**: Meets Brazilian Central Bank regulations regarding systemic risk, algorithmic accountability, and auditability in financial institutions. + +## Zero-Persistence Architecture +The ATI engine does **not** rely on persistent storage. +1. When a prompt is processed, the model generates an output. +2. The middleware immediately generates an **ephemeral ECDSA private key** directly in RAM. +3. The prompt and the output are hashed using SHA-256. +4. The private key signs the hashes to create a cryptographic signature. +5. The public key, signature, and hashes are attached to the API response payload. +6. The private key is immediately discarded. **No data is saved to disk.** + +## How to Use + +The middleware intercepts requests on standard OpenAI-compatible endpoints (`/v1/chat/completions`, `/generate`). When a request is made, the middleware adds an `ati_proof` object to the JSON response: + +```json +{ + "id": "cmpl-123", + "object": "text_completion", + "choices": [ ... ], + "ati_proof": { + "input_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "output_hash": "4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b", + "signature": "MEUCIQCHX...", + "public_key": "MFkwEw...", + "timestamp": "2026-03-17T15:30:00.000000+00:00", + "model_name": "Qwen/Qwen3-0.6B", + "version": "foundlab-ati-v0.1" + } +} +``` + +Because the `ati_proof` is embedded directly in the response body, it natively works with **Open WebUI** and other standard frontends without requiring custom headers. + +## How to connect to Veritas Ledger +To permanently audit the interaction and achieve immutable regulatory compliance, you can optionally anchor the `ati_proof` to the Veritas Ledger using a simple one-line post-processing hook: + +```python +requests.post("https://api.veritasledger.com/v1/anchor", json={"ati_proof": response["ati_proof"]}) +``` + +This anchors the hashes without sending the plain-text prompt or generated output to the ledger, maintaining absolute data privacy. diff --git a/extensions/foundlab-ati/example_integration.ipynb b/extensions/foundlab-ati/example_integration.ipynb new file mode 100644 index 0000000..0d7a14a --- /dev/null +++ b/extensions/foundlab-ati/example_integration.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a2e4e0be", + "metadata": {}, + "source": [ + "\n", + "\n", + "
\n", + " \n", + "
\n", + "\n", + "

STEP-BY-STEP GUIDE TO RUN LLM MODELS ON-DEVICE (FOUNDLAB ATI INTEGRATED)

" + ] + }, + { + "cell_type": "markdown", + "id": "ad6fc329", + "metadata": {}, + "source": [ + "

Project Overview - FoundLab ATI Extension

\n", + "\n", + "AIKit is Razer's AI developer environment built to simplify and accelerate machine learning workflows on high-performance Razer hardware. \n", + "\n", + "This notebook includes the **FoundLab ATI (Algorithmic Trust Indicator)** extension, providing a zero-persistence cryptographic proof of the model's output in compliance with LGPD, the EU AI Act, and BCB 538." + ] + }, + { + "cell_type": "markdown", + "id": "d32fbb5c", + "metadata": {}, + "source": [ + "

🚀 1. Run a Model with FoundLab ATI Middleware

\n", + "\n", + "We can start the vLLM server by explicitly injecting the FoundLab ATI Middleware. For this example, we mock the server process to show how the integration works behind the scenes." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1cbbc36d", + "metadata": {}, + "outputs": [], + "source": [ + "# Simulate running the vLLM server with FoundLab ATI Middleware injected.\n", + "# In production, the middleware is attached to the FastAPI application.\n", + "\n", + "from foundlab_ati.generate_proof import generate_proof, verify_proof\n", + "import json\n", + "\n", + "def mock_vllm_generate(prompt: str, model_name: str) -> dict:\n", + " \"\"\"Mock vLLM completion containing FoundLab ATI proof\"\"\"\n", + " # 1. The model generates text\n", + " output_text = \"Quantum computing uses quantum mechanics to process information much faster than regular computers.\"\n", + " \n", + " # 2. The middleware automatically generates the ATI proof\n", + " ati_proof = generate_proof(prompt, output_text, model_name)\n", + " \n", + " # 3. The response is returned to the client\n", + " return {\n", + " \"id\": \"cmpl-mock123\",\n", + " \"object\": \"text_completion\",\n", + " \"created\": 1710682000,\n", + " \"model\": model_name,\n", + " \"choices\": [\n", + " {\n", + " \"text\": output_text,\n", + " \"index\": 0,\n", + " \"logprobs\": None,\n", + " \"finish_reason\": \"stop\"\n", + " }\n", + " ],\n", + " \"ati_proof\": ati_proof\n", + " }" + ] + }, + { + "cell_type": "markdown", + "id": "129a8aca", + "metadata": {}, + "source": [ + "

💬 2. Generate Text and Get ATI Proof

\n", + "\n", + "Once the model is running with the ATI Middleware, any prompt you send will return a secure response including the `\"ati_proof\"` object in the JSON body." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6814319f", + "metadata": {}, + "outputs": [], + "source": [ + "prompt = \"Explain quantum computing to a 12-year-old.\"\n", + "model = \"Qwen/Qwen3-0.6B\"\n", + "\n", + "print(f\"Sending prompt: '{prompt}'...\")\n", + "response = mock_vllm_generate(prompt, model)\n", + "\n", + "# Print the generated text\n", + "generated_text = response['choices'][0]['text']\n", + "print(f\"\\nResponse:\\n{generated_text}\\n\")\n", + "\n", + "# Print the ATI proof\n", + "print(\"=== Algorithmic Trust Indicator (ATI) ===\")\n", + "print(json.dumps(response['ati_proof'], indent=2))" + ] + }, + { + "cell_type": "markdown", + "id": "590d387c", + "metadata": {}, + "source": [ + "

✅ 3. Verify the Proof

\n", + "\n", + "You or a third-party auditor can mathematically verify that the output was indeed the answer to the specific input, without needing to save any persistent data on the device." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c517cd84", + "metadata": {}, + "outputs": [], + "source": [ + "# Extract the proof and verify it\n", + "proof = response['ati_proof']\n", + "\n", + "is_valid = verify_proof(proof, prompt, generated_text)\n", + "\n", + "if is_valid:\n", + " print(\"✅ ATI Proof Verification: SUCCESS. The cryptographic signature matches the input and output.\")\n", + "else:\n", + " print(\"❌ ATI Proof Verification: FAILED. The data may have been tampered with.\")" + ] + } + ], + "metadata": { + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/extensions/foundlab-ati/foundlab_ati_middleware.py b/extensions/foundlab-ati/foundlab_ati_middleware.py new file mode 100644 index 0000000..b22173c --- /dev/null +++ b/extensions/foundlab-ati/foundlab_ati_middleware.py @@ -0,0 +1,88 @@ +import json +from starlette.middleware.base import BaseHTTPMiddleware +from starlette.requests import Request +from starlette.responses import Response + +from .generate_proof import generate_proof + +class FoundLabATIMiddleware(BaseHTTPMiddleware): + """ + FastAPI/ASGI middleware that integrates with the vLLM OpenAI-compatible server. + It hooks into the generate/completions endpoints and adds an ATI proof to the response. + """ + + async def dispatch(self, request: Request, call_next): + # We only want to process chat completions or generate requests + if request.url.path not in ["/v1/chat/completions", "/v1/completions", "/generate"]: + return await call_next(request) + + # Extract the request body for the input text + try: + body_bytes = await request.body() + body = json.loads(body_bytes.decode('utf-8')) + + # Try to get the prompt from chat format or completion format + if "messages" in body: + input_text = json.dumps(body["messages"]) + else: + input_text = body.get("prompt", str(body)) + + model_name = body.get("model", "unknown-model") + except Exception: + input_text = "" + model_name = "unknown" + + # Call the next middleware / endpoint + response = await call_next(request) + + # Only process successful JSON responses (non-streaming) + if response.status_code == 200 and getattr(response, 'media_type', None) == "application/json": + # Buffer the response body + response_body = b"" + async for chunk in response.body_iterator: + response_body += chunk + + try: + data = json.loads(response_body.decode('utf-8')) + + # Extract the generated text + output_text = "" + if "choices" in data and len(data["choices"]) > 0: + choice = data["choices"][0] + if "message" in choice and "content" in choice["message"]: + output_text = choice["message"]["content"] + elif "text" in choice: + output_text = choice["text"] + else: + output_text = str(data) + + # Generate the ATI Proof + ati_proof = generate_proof(input_text, output_text, model_name) + + # Add the ATI proof to the response payload + data["ati_proof"] = ati_proof + + # Create a new response with the modified JSON data + new_body = json.dumps(data).encode('utf-8') + + # Update headers (especially content-length) + headers = dict(response.headers) + headers['content-length'] = str(len(new_body)) + + return Response( + content=new_body, + status_code=response.status_code, + headers=headers, + media_type=response.media_type + ) + + except Exception: + # If anything fails, return the original buffered response + return Response( + content=response_body, + status_code=response.status_code, + headers=dict(response.headers), + media_type=response.media_type + ) + + return response diff --git a/extensions/foundlab-ati/generate_proof.py b/extensions/foundlab-ati/generate_proof.py new file mode 100644 index 0000000..5666331 --- /dev/null +++ b/extensions/foundlab-ati/generate_proof.py @@ -0,0 +1,67 @@ +import hashlib +import base64 +import json +import datetime +from ecdsa import SigningKey, VerifyingKey, SECP256k1, BadSignatureError + +def generate_proof(input_text: str, output_text: str, model_name: str) -> dict: + """ + Generates a zero-persistence cryptographic proof (ATI) for the given input and output. + An ephemeral ECDSA private key is generated in memory and never saved to disk. + """ + # Generate an ephemeral ECDSA private key + # This key only lives in RAM and is discarded after generating the signature + private_key = SigningKey.generate(curve=SECP256k1) + public_key = private_key.get_verifying_key() + + # Calculate SHA-256 hashes of the input and output + input_hash = hashlib.sha256(input_text.encode('utf-8')).hexdigest() + output_hash = hashlib.sha256(output_text.encode('utf-8')).hexdigest() + + # Create the payload to sign (combining input and output hashes) + payload_to_sign = f"{input_hash}:{output_hash}".encode('utf-8') + + # Sign the payload + signature = private_key.sign(payload_to_sign) + + # Prepare the JSON-serializable proof dictionary + proof_dict = { + "input_hash": input_hash, + "output_hash": output_hash, + "signature": base64.b64encode(signature).decode('utf-8'), + "public_key": base64.b64encode(public_key.to_string()).decode('utf-8'), + "timestamp": datetime.datetime.now(datetime.timezone.utc).isoformat(), + "model_name": model_name, + "version": "foundlab-ati-v0.1" + } + + return proof_dict + +def verify_proof(proof_dict: dict, input_text: str, output_text: str) -> bool: + """ + Verifies a zero-persistence cryptographic proof (ATI). + """ + try: + # Reconstruct hashes + expected_input_hash = hashlib.sha256(input_text.encode('utf-8')).hexdigest() + expected_output_hash = hashlib.sha256(output_text.encode('utf-8')).hexdigest() + + # Check if hashes match + if expected_input_hash != proof_dict["input_hash"] or expected_output_hash != proof_dict["output_hash"]: + return False + + # Reconstruct the payload + payload_to_verify = f"{expected_input_hash}:{expected_output_hash}".encode('utf-8') + + # Load the public key and signature + public_key_bytes = base64.b64decode(proof_dict["public_key"]) + signature_bytes = base64.b64decode(proof_dict["signature"]) + + # Reconstruct the verifying key + verifying_key = VerifyingKey.from_string(public_key_bytes, curve=SECP256k1) + + # Verify the signature + return verifying_key.verify(signature_bytes, payload_to_verify) + + except (KeyError, ValueError, BadSignatureError, Exception): + return False diff --git a/foundlab-aikit/pyproject.toml b/foundlab-aikit/pyproject.toml new file mode 100644 index 0000000..1a2ccd9 --- /dev/null +++ b/foundlab-aikit/pyproject.toml @@ -0,0 +1,36 @@ +[tool.poetry] +name = "foundlab-aikit" +version = "0.3.1" +description = "A package for FoundLab AIKit compute and AI services" +authors = ["FoundLab"] +license = "Apache-2.0" +packages = [ + { include = "src" }, + { include = "util" } +] + +[tool.poetry.dependencies] +python = "~3.12" # vllm wants this stricter constraint +typer = ">0.19.2" +numpy = "^2.1.2" +psutil = "^7.0.0" +transformers = "^4.51.3" +accelerate = "^1.7.0" +nvidia-ml-py = ">12.575.51" +prometheus-client = ">0.22.1" +typing-extensions = "^4.14.1" +openai = ">1.97.1" +humanize = "^4.14.0" + + +[tool.poetry.group.dev.dependencies] +pytest = "^8.3.4" +pytest-mock = "^3.14.1" + +[build-system] +requires = ["poetry-core>=1.0.0"] +build-backend = "poetry.core.masonry.api" + +[tool.poetry.plugins."console_scripts"] +foundlab-ai = "src.cli:app" +run_vllm = "util.run_vllm:main"