diff --git a/docs/inference.md b/docs/inference.md new file mode 100644 index 0000000..ae80e72 --- /dev/null +++ b/docs/inference.md @@ -0,0 +1,485 @@ +# Inference System — Integration Guide + +> **Audience**: Embedded / firmware team +> **Last Updated**: 2026-05-09 +> **Related**: [ML–Control Interface Specification](./ml_control_interface.md) + +This document covers everything needed to get torque predictions from the TCN model at runtime. It does not repeat the tensor spec or feature ordering — those are fully defined in [`ml_control_interface.md`](./ml_control_interface.md). Read that first. + +--- + +## Table of Contents + +1. [Overview](#overview) +2. [Prerequisites](#prerequisites) +3. [Deployment Modes](#deployment-modes) +4. [Mode A: HTTP Server](#mode-a-http-server) + - [Starting the Server](#starting-the-server) + - [Running as a systemd Service](#running-as-a-systemd-service) + - [Endpoint Reference](#endpoint-reference) + - [POST /predict (JSON)](#post-predict-json) + - [POST /predict_msgpack (Binary)](#post-predict_msgpack-binary) + - [GET /health](#get-health) + - [C++ Integration Example](#c-integration-example) +5. [Mode B: Serial (Direct Arduino)](#mode-b-serial-direct-arduino) + - [Protocol](#protocol) + - [Running the Script](#running-the-script) +6. [Choosing a Mode](#choosing-a-mode) +7. [Model Export](#model-export) +8. [Validation & Testing](#validation--testing) +9. [Error Handling](#error-handling) +10. [Configuration Reference](#configuration-reference) + +--- + +## Overview + +The ML system runs on a **Raspberry Pi** and exposes the trained TCN model for real-time torque prediction. At each control cycle, your system sends a window of sensor readings and receives 4 joint torque values: + +| Output | Unit | Description | +|--------|------|-------------| +| `hip_left` | Nm/kg | Left hip biological joint moment | +| `hip_right` | Nm/kg | Right hip biological joint moment | +| `knee_left` | Nm/kg | Left knee biological joint moment | +| `knee_right` | Nm/kg | Right knee biological joint moment | + +> **Important**: Outputs are normalized by body mass. Multiply by participant mass (kg) to get absolute torque in Nm. +> See [ml_control_interface.md — Output Specification](./ml_control_interface.md#output-specification) for full details. + +There is one production-ready deployment mode and one reference implementation: + +| Mode | Script | Interface | Status | +|------|--------|-----------|--------| +| **A — HTTP Server** | `scripts/server.py` | HTTP POST (JSON or msgpack) | Production-ready | +| **B — Serial** | `scripts/endpoint_tcn.py` | UART serial | Reference implementation only | + +--- + +## Prerequisites + +### Hardware +- Raspberry Pi (tested on Pi 4 / Pi 5) +- ONNX model file at `outputs/model.onnx` on the Pi (see [Model Export](#model-export)) + +### Python dependencies (install on the Pi) + +```bash +pip install fastapi "uvicorn[standard]" onnxruntime numpy msgpack +``` + +For serial mode only: + +```bash +pip install pyserial +``` + +For C++ test client only: + +```bash +sudo apt install libcurl4-openssl-dev nlohmann-json3-dev libmsgpack-dev +``` + +--- + +## Deployment Modes + +### Mode A: HTTP Server + +The server loads the ONNX model once at startup and serves predictions over HTTP. The C++ control system calls `POST /predict` or `POST /predict_msgpack` on each control cycle. + +``` +C++ control loop + │ + │ POST /predict_msgpack (recommended) + │ POST /predict (JSON fallback) + ▼ +FastAPI server (scripts/server.py) + │ + ▼ +ONNX Runtime (outputs/model.onnx) + │ + ▼ +{ hip_left, hip_right, knee_left, knee_right, inference_ms } +``` + +#### Starting the Server + +```bash +# From the exoskeleton-ai repo root on the Pi: + +# Accessible only from localhost (recommended for same-Pi integration): +uvicorn scripts.server:app --host 127.0.0.1 --port 8000 + +# Accessible over the network (e.g. laptop → Pi): +uvicorn scripts.server:app --host 0.0.0.0 --port 8000 +``` + +The server prints confirmation when the model is loaded: + +``` +Loading ONNX model: /home/pi/exoskeleton-ai/outputs/model.onnx +Model loaded. Input: 'input' | Expected shape: (1, 187, 28) +``` + +--- + +#### Running as a systemd Service + +For unattended deployments (auto-start at boot, auto-restart on crash), use the systemd unit at [`deploy/exoskeleton-server.service`](../deploy/exoskeleton-server.service). It runs `uvicorn scripts.server:app --host 0.0.0.0 --port 8000` from the repo's virtualenv and restarts on failure. + +**Install on the Pi** + +```bash +# 1. Confirm the paths in the unit file match your Pi. +# Defaults assume: +# WorkingDirectory=/home/dylan-exo/ml_stuff/exoskeleton-ai +# ExecStart=/home/dylan-exo/ml_stuff/exoskeleton-ai/.venv/bin/uvicorn ... +# User=dylan-exo +# Edit deploy/exoskeleton-server.service if your layout differs. + +# 2. Install the unit: +sudo cp deploy/exoskeleton-server.service /etc/systemd/system/ +sudo systemctl daemon-reload + +# 3. Enable at boot and start now: +sudo systemctl enable --now exoskeleton-server + +# 4. Verify: +sudo systemctl status exoskeleton-server +curl http://127.0.0.1:8000/health +``` + +**Common operations** + +```bash +sudo systemctl restart exoskeleton-server # after updating model.onnx or server.py +sudo systemctl stop exoskeleton-server # stop the server +sudo journalctl -u exoskeleton-server -f # tail logs +``` + +> The unit binds to `0.0.0.0:8000` (network-accessible). For localhost-only deployments, edit `ExecStart` to use `--host 127.0.0.1`. + +--- + +#### Endpoint Reference + +##### `POST /predict` (JSON) + +Standard JSON request/response. Use this for debugging or if your C++ environment does not have a msgpack library. + +**Request** + +- Content-Type: `application/json` +- Body: JSON object with a single key `window` — a 2D array of shape `[WINDOW_SIZE][28]` + +```json +{ + "window": [ + [f0, f1, ..., f27], + [f0, f1, ..., f27], + ... + ] +} +``` + +`window` must have exactly `WINDOW_SIZE` rows (187 for the standard TCN). Each row must have exactly 28 values in the feature order defined in [`ml_control_interface.md`](./ml_control_interface.md#feature-ordering). + +**Response** + +- Content-Type: `application/json` +- HTTP 200 on success, 422 on shape validation error, 503 if model not loaded + +```json +{ + "hip_left": 0.1234, + "hip_right": 0.1102, + "knee_left": -0.0873, + "knee_right": -0.0941, + "inference_ms": 3.241 +} +``` + +| Field | Type | Description | +|-------|------|-------------| +| `hip_left` | float | Left hip moment (Nm/kg) | +| `hip_right` | float | Right hip moment (Nm/kg) | +| `knee_left` | float | Left knee moment (Nm/kg) | +| `knee_right` | float | Right knee moment (Nm/kg) | +| `inference_ms` | float | ONNX session time only — does not include HTTP or JSON overhead | + +--- + +##### `POST /predict_msgpack` (Binary) + +Preferred endpoint for the C++ control loop. Uses msgpack binary encoding — lower serialization overhead than JSON, especially important at 100 Hz. + +**Request** + +- Content-Type: `application/x-msgpack` +- Body: msgpack-encoded flat array of `WINDOW_SIZE * 28 = 5236` float32 values, **row-major order** + - i.e. timestep 0's 28 features, then timestep 1's 28 features, ..., then timestep 186's 28 features + +**Response** + +- Content-Type: `application/x-msgpack` +- Body: msgpack-encoded map with the same 5 keys as the JSON response + +``` +{ + "hip_left": , + "hip_right": , + "knee_left": , + "knee_right": , + "inference_ms": +} +``` + +**C++ Integration Example** + +```cpp +#include +#include +#include +#include +#include + +// Build request body: flat float32 array, row-major, shape (187, 28) +std::string build_msgpack_request(const std::vector& flat) { + msgpack::sbuffer buf; + msgpack::pack(buf, flat); + return std::string(buf.data(), buf.size()); +} + +// Parse response body into prediction values +struct Prediction { + double hip_left, hip_right, knee_left, knee_right, inference_ms; +}; + +Prediction parse_msgpack_response(const std::string& body) { + msgpack::object_handle oh = msgpack::unpack(body.data(), body.size()); + std::map result; + oh.get().convert(result); + return { + result["hip_left"], + result["hip_right"], + result["knee_left"], + result["knee_right"], + result["inference_ms"], + }; +} + +// In your control loop: +// 1. Fill flat[] with your (187 * 28) sensor window, row-major +// 2. POST to http://127.0.0.1:8000/predict_msgpack +// 3. Parse response and use prediction values +``` + +For a complete working example including libcurl boilerplate, health check, and latency benchmarking, see `scripts/test_endpoint.cpp`. + +--- + +##### `GET /health` + +Liveness check. Returns HTTP 200 when the model is loaded and ready, HTTP 503 otherwise. Poll this on startup before sending predictions. + +**Response (200)** +```json +{ "status": "ok" } +``` + +--- + +### Mode B: Serial (Reference Implementation) + +> **This is not a production-ready script.** `scripts/endpoint_tcn.py` is a proof-of-concept demonstrating how a serial inference loop would work once an Arduino with IMU hardware is available. It is not suitable for deployment as-is — it is missing argument parsing, graceful shutdown, and runtime configuration. Use it as a starting point when the embedded hardware is ready. + +The script runs on the Pi and communicates with an Arduino over UART (`/dev/ttyACM0`, 115200 baud). It reads one sample at a time from serial, maintains a rolling window, runs ONNX inference, and writes 4 torque values back. + +#### Intended Protocol + +**Arduino → Pi (input)** + +Send one line of comma-separated floats over serial at each sample: + +``` +f0,f1,f2,...,f27\n +``` + +- 28 values per line, matching the feature order in [`ml_control_interface.md`](./ml_control_interface.md#feature-ordering) +- Baud rate: **115200** +- One sample per line, terminated with `\n` + +The Pi maintains a rolling window of the last 100 samples internally. No need to send the full window — just one sample at a time. + +**Pi → Arduino (output)** + +After each inference, the Pi sends back one line: + +``` +hip_left,hip_right,knee_left,knee_right\n +``` + +- 4 comma-separated floats, 4 decimal places each +- Units: Nm/kg (multiply by body mass for absolute torque) +- Example: `0.1234,0.1102,-0.0873,-0.0941\n` + +#### Known Limitations + +- Serial port and baud rate are hardcoded — cannot be overridden at runtime without editing the file +- `argparse` is imported but not wired up +- No graceful shutdown (Ctrl+C will leave the serial port open) +- Window size (100) does not match the HTTP server (187) — accuracy implications TBD with the ML team +- Docstring is incomplete + +--- + +## Choosing a Mode + +**Use Mode A (HTTP Server)** for any current integration work. It is the only production-ready option. + +**Mode B (Serial)** describes the intended protocol for direct Arduino ↔ Pi communication but requires further development before it can be deployed. Refer to it when building out the embedded serial interface. + +| Consideration | HTTP Server (Mode A) | Serial (Mode B) | +|--------------|----------------------|-----------------| +| Status | Production-ready | Reference implementation | +| Interface | HTTP POST | UART | +| Overhead | Low (msgpack) / Moderate (JSON) | Minimal | +| Latency at 100 Hz | Typically passes 10 ms budget (verify with benchmark) | Lowest possible | +| Architecture | Pi runs server; C++ calls it | Pi reads/writes serial directly | +| Debugging | Easy — curl / Postman / browser | Requires serial monitor | + +--- + +## Model Export + +The deployed model must be in ONNX format (`outputs/model.onnx`). If the ML team has given you a `.onnx` file, copy it to the Pi: + +```bash +scp outputs/model.onnx pi@:~/exoskeleton-ai/outputs/model.onnx +``` + +If you need to regenerate the ONNX file from a trained PyTorch checkpoint: + +```bash +# Default path (outputs/2026-01-08/18-29-05/best_model.pt): +python scripts/export_onnx.py + +# Specify a checkpoint: +python scripts/export_onnx.py --model-path outputs//