🏎️ NeuroDrive

A lidar-driven autonomous racing environment for reinforcement learning.

Learn to drive a car around procedurally generated tracks using nothing but range-finder beams — a compact, fast, fully Gymnasium-compatible environment with a shaped, exploit-resistant reward and a trained PPO baseline.

A PPO agent driving a track it has never seen, from lidar alone.

---

Why this exists

NeuroDrive is a small, self-contained study in reinforcement-learning environment design: the parts that actually decide whether an agent learns something useful — the observation, the action interface, and above all a reward that can't be gamed. It is deliberately fast (vectorised NumPy physics and sensing, no heavyweight simulator) so a capable policy trains on a laptop CPU in minutes.

Features

• 🎮 Clean Gymnasium API — passes gymnasium.utils.env_checker, works out of the box with Stable-Baselines3.
• 🛰️ Learn from sensors, not the map — the agent sees a fan of normalised lidar beams plus its own speed and heading; it never gets the track handed to it.
• 🎲 Procedural tracks — every episode is a new smooth closed circuit (periodic Catmull-Rom spline), so the policy has to generalise, not memorise.
• 🧮 Shaped, exploit-resistant reward — forward progress along the centerline, a monotonic lap bonus that cannot be farmed by oscillating across the line, and a crash penalty. (See the reward design note.)
• 🚗 Kinematic bicycle-model physics — smooth, believable car dynamics in a few lines.
• ✅ Tested & linted — behaviour tests (including a reward-hack regression test) and CI on Python 3.10–3.12.
• 🎥 One-command GIF / MP4 recording for any policy.

Random vs. trained

Untrained (random actions)	Trained PPO
spins out immediately	completes laps on unseen tracks

Quickstart

git clone https://github.com/yferc/neurodrive.git
cd neurodrive
pip install -e ".[train,media]"

Drive with the trained baseline and watch it live:

import gymnasium as gym
import neurodrive                      # registers "NeuroDrive-v0"
from stable_baselines3 import PPO

env = gym.make("NeuroDrive-v0", render_mode="human")
model = PPO.load("models/best/best_model.zip")

obs, _ = env.reset()
done = False
while not done:
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, term, trunc, info = env.step(action)
    env.render()
    done = term or trunc

How it works

Observation — Box(shape=(n_beams + 3,))

Component	Meaning
[0 : n_beams]	normalised lidar ranges in [0, 1] (1 = clear to max range)
[n_beams]	speed normalised by top speed
[n_beams + 1]	sin of heading error to the road direction
[n_beams + 2]	cos of heading error to the road direction

The lidar is cast vectorised: every beam is intersected against every track boundary segment in one NumPy operation (neurodrive/sensors.py).

Action — Box(shape=(2,), [-1, 1])

[steering, throttle] — continuous. -1/-1 is full-left / full-brake, +1/+1 is full-right / full-throttle.

Reward

+ progress        forward metres along the centerline this step
+ lap_bonus       once per newly-completed lap (monotonic — never farmable)
- time_cost       small per-step cost, so faster is better
- crash_penalty   on leaving the track (episode ends)

The lap bonus is only ever paid when the agent reaches a new highest lap. An earlier version awarded it whenever the lap counter changed, and PPO promptly discovered it could vibrate across the start line to farm bonuses forever — a textbook reward hack, now covered by a regression test (test_no_phantom_laps_when_idle).

Training

python scripts/train.py --timesteps 1200000 --n-envs 8
tensorboard --logdir logs        # optional: watch it learn

PPO (Stable-Baselines3) with an MLP policy trains a competent driver on CPU. The best policy by evaluation reward is saved to models/best/.

Recording media

python scripts/record.py --model models/best/best_model.zip --out docs/media/trained
python scripts/record.py --random --out docs/media/random

Results

PPO trained for 1.2M timesteps (8 parallel envs, ~11 minutes on a laptop CPU), then evaluated with a deterministic policy on 20 random procedurally-generated tracks:

Metric	Value
Mean episode reward	1128 ± 11
Mean laps per episode	3.05 (max 4)
Mean distance driven	1097 units / episode
Crash rate	0 / 20 (0%)
Full-episode completion	20 / 20

The agent drives every unseen track to the time limit without crashing — it generalises across track shapes from lidar alone, rather than memorising a single circuit. Reproduce with python scripts/train.py then the evaluation snippet, or just load models/best/best_model.zip.

Project structure

neurodrive/
├── neurodrive/
│   ├── env.py        # NeuroDriveEnv — the Gymnasium environment
│   ├── car.py        # kinematic bicycle-model dynamics
│   ├── track.py      # procedural closed-track generation + geometry cache
│   ├── sensors.py    # vectorised lidar ray-casting
│   └── render.py     # pygame rendering (human + rgb_array)
├── scripts/
│   ├── train.py      # PPO training
│   └── record.py     # roll out a policy → GIF + MP4
├── tests/            # behaviour + reward-hack regression tests
└── .github/workflows/ci.yml

Testing

pip install -e ".[dev]"
pytest -q

License

MIT — see LICENSE.