NEURIM

NEURIM is an online human-in-the-loop optimizer. It searches a diffusion model's anchor space using one scalar reward decoded from EEG.

It does not reconstruct images from brain activity. The EEG pipeline produces a noisy approval signal; the optimizer uses that signal to choose which image to render next.

Architecture

EEG -> FAA reward -> Optimizer -> z -> Private diffusion server -> PNG
                            |                              |
                            +-> local live_frame.png <-----+

The runtime is split across two machines:

Component	Location	Package
Frontend API, EMOTIV lifecycle, and session lifecycle	Local machine	src/server/api/
Prompt curation	Local machine	src/session/curation.py
EEG/mock optimizer session	Local machine	src/session/
Manifest-driven diffusion renderer	Private GPU machine	src/server/diffusion/

Files in scripts/ are compatibility CLI entrypoints. Runtime logic belongs under src/.

Manifest-Driven Flow

1. Curate the user's prompt into a seven-anchor manifest:

python scripts/run_prompt_curation.py \
  --user-prompt "Woolly mammoths" \
  --out data/processed/prompt_sessions/mammoths.json \
  --verbose

2. Copy the manifest to the private GPU machine and start its render server:

python scripts/run_general_stable_diffusion.py \
  --session-manifest data/processed/prompt_sessions/mammoths.json \
  --host 0.0.0.0 \
  --port 8766

Select a specific GPU when the private machine exposes multiple CUDA devices:

CUDA_VISIBLE_DEVICES=4 python scripts/run_general_stable_diffusion.py \
  --session-manifest data/processed/prompt_sessions/mammoths.json \
  --host 0.0.0.0 \
  --port 8766

The private server exposes:

• GET /manifest: active manifest metadata and render endpoint.
• POST /render: accepts {"z": [...], "frame_size": 512} and returns PNG bytes.

3. Start the local API bridge. It immediately attempts to connect to EMOTIV Cortex, retries every 60 seconds on failure, and calibrates for 30 seconds once connected:

python scripts/api_server.py --host 127.0.0.1 --port 8000

4. Validate the optimizer without EEG hardware:

python scripts/run_mock_optimizer.py --server-url http://GPU_HOST:8766 --seed 3

5. Run the EEG optimizer directly, without the frontend API:

python scripts/run_real_eeg_optimizer.py --server-url http://GPU_HOST:8766

Use --mock for synthetic EEG while retaining the FAA and optimizer wiring:

python scripts/run_real_eeg_optimizer.py \
  --mock \
  --server-url http://GPU_HOST:8766

Both optimizer clients atomically update data/processed/live_frame.png. The basic viewer at frontend/live_view.html polls that file. Real EEG sessions also save session_start.png and session_end.png.

Frontend API

Start the local API bridge:

python scripts/api_server.py --host 127.0.0.1 --port 8000

Run the frontend separately:

cd frontend-app
NEURIM_API_URL=http://127.0.0.1:8000 npm run dev

The API exposes:

• GET /health
• GET /eeg/status
• POST /eeg/retry
• POST /session/start
• POST /session/stop
• GET /session/status
• GET /session/logs

The API owns the EMOTIV connection. On startup it connects to Cortex, retries every 60 seconds if no headset is available, and runs a 30-second FAA baseline calibration immediately after connection. POST /eeg/retry forces an immediate retry.

When POST /session/start receives a prompt, the API curates a local manifest, compares it with the private diffusion server's GET /manifest response, and starts the optimizer only if they match. The private diffusion server must already be running with that manifest.

API-managed same-machine diffusion

For a same-machine GPU setup, api_server.py can own the diffusion process. In this mode, each frontend prompt curates a new manifest, restarts scripts/run_general_stable_diffusion.py with that manifest, waits for GET /manifest, validates the active manifest, and then starts the optimizer.

Enable it before starting the API bridge:

NEURIM_MANAGE_DIFFUSION=true \
NEURIM_DIFFUSION_HOST=127.0.0.1 \
NEURIM_DIFFUSION_PORT=8766 \
NEURIM_DIFFUSION_CUDA_VISIBLE_DEVICES=4 \
python scripts/api_server.py --host 127.0.0.1 --port 8000

On PowerShell:

$env:NEURIM_MANAGE_DIFFUSION="true"
$env:NEURIM_DIFFUSION_HOST="127.0.0.1"
$env:NEURIM_DIFFUSION_PORT="8766"
$env:NEURIM_DIFFUSION_CUDA_VISIBLE_DEVICES="4"
python scripts/api_server.py --host 127.0.0.1 --port 8000

Optional environment variables:

• NEURIM_DIFFUSION_PYTHON: Python executable used to launch the diffusion server.
• NEURIM_DIFFUSION_MODEL: model id passed to --model, default stabilityai/sd-turbo.
• NEURIM_DIFFUSION_STARTUP_TIMEOUT_S: seconds to wait for the restarted server, default 300.

Do not run run_mock_optimizer.py or run_real_eeg_optimizer.py separately in this mode. The frontend starts sessions through api_server.py, and the API owns the optimizer loop.

API-managed remote GPU diffusion

When the EPOC X headset is connected to the local machine but CUDA is on a private GPU server, keep api_server.py local and run a lightweight diffusion supervisor on the GPU server.

On the GPU server:

NEURIM_DIFFUSION_HOST=127.0.0.1 \
NEURIM_DIFFUSION_PORT=8766 \
NEURIM_DIFFUSION_PUBLIC_URL=http://GPU_HOST:8766 \
NEURIM_DIFFUSION_CUDA_VISIBLE_DEVICES=4 \
python scripts/diffusion_supervisor.py --host 0.0.0.0 --port 8010

On the local machine with EMOTIV Launcher and the EPOC X:

$env:NEURIM_DIFFUSION_SUPERVISOR_URL="http://GPU_HOST:8010"
python scripts/api_server.py --host 127.0.0.1 --port 8000

Then run the frontend locally:

cd frontend-app
NEURIM_API_URL=http://127.0.0.1:8000 npm run dev

In this mode, the local API curates the prompt and owns EEG/optimizer. The GPU supervisor receives the curated manifest, restarts the diffusion renderer on the GPU server, and returns the render URL to the local API.

Core Services

• src/signal_service/: EEG sources, baseline calibration, and frontal alpha asymmetry reward computation.
• src/optimizer/: optimizer algorithms and the CALIBRATE -> EXPLORE -> REFINE -> SETTLE state machine.
• src/generator/: manifest validation, interpolation, and generator support.
• src/session/: diffusion HTTP client, atomic frame storage, shared optimizer render loop, and real/mock session runners.
• src/server/api/: FastAPI app, request models, settings, logs, and subprocess lifecycle management.
• src/server/diffusion/: anchor renderer, HTTP transport, pipeline loading, and GPU server composition.

config/config.yaml controls FAA windows, optimizer behavior, seven-dimensional anchor search, frame size, and model defaults.

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -r requirements-diffusion.txt  # GPU diffusion machine only
pytest

Prompt curation requires OPENAI_API_KEY. Real EEG acquisition requires the EMOTIV credentials described in .env.example.

Documentation

• Preference reward pipeline