The reinforcement-learning engine behind Retro Speedlab.

datenwissenschaften turns classic-game emulators into reproducible training systems. It provides visual and state-aware environments, recurrent PPO agents, parallel execution, durable checkpoints, episode recording, and a live browser dashboard in one focused Python package.

This repository contains the reusable engine. For game runners, end-to-end examples, and user-facing documentation, start with Retro Speedlab.

• Exploration for sparse rewards — adaptive multi-input CNN-LSTM PPO combines visual frames, normalized RAM, temporal memory, and normalized, clipped, annealed Random Network Distillation (RND).
• Efficient execution — vectorized environments, automatic worker selection, CUDA tuning, and CPU fallback.
• Reliable training runs — atomic checkpoints, resumable model state, and .bk2 replay capture.
• Operational visibility — a local Vue dashboard reports episode outcomes, reward distributions, environment details, PPO parameters, and RND progress.
• Game-oriented infrastructure — ROM discovery, RAM models, state machines, visual encoders, and configurable action translation.

AdaptiveRecurrentRNDModel is the recommended starting point for visual agents. RND encourages the policy to visit novel observations, while its influence decays during training so learned external rewards increasingly drive behavior. The model also auto-configures score-staleness windows, missing-win windows, exploration multipliers, entropy, learning rate, clip range, and RND update pressure from the action space, rollout size, training horizon, fitness volatility, score staleness, and win staleness. The predictor, fixed target, optimizer, reward statistics, adaptation state, and annealing progress are all preserved in checkpoints. Environment wrappers always use RGB observations and one emulator step per selected action; these are fixed engine defaults rather than game-level options. The default profile uses longer 512-step rollouts, a 256-unit LSTM, gamma=0.999, gae_lambda=0.98, and a slower 5-million-step RND decay. These settings preserve more temporal context and delayed reward information than the shorter arcade baseline while retaining conservative PPO updates.

The package requires Python 3.12.

pip install datenwissenschaften

For local development:

git clone https://github.com/datenwissenschaften/datenwissenschaften.git
cd datenwissenschaften
poetry install
cp config.example.yaml config.yaml

Enable the dashboard with ui.enable: true, then open http://127.0.0.1:18080. It refreshes live training telemetry without interrupting the learner.

Dashboard history and non-file training state are restored from and persisted to Redis. This includes best-episode references and metrics, callback state, and target memory. Best episodes are scoped by game identity and game savestate (for example level1-1), independently of the active training objective. Model checkpoints and .bk2 episode recordings remain on disk. The default Redis URL is redis://127.0.0.1:6379/0, and history keys use the datenwissenschaften:history prefix. The ui mapping accepts enable, host, port, max_episodes, redis_url, and history_key_prefix. Snapshots retain the latest 1,000 episodes by default and include summarized totals for discarded episodes; set max_episodes to another positive integer or null for unlimited retained rows. Binding to 0.0.0.0 makes the dashboard reachable on the local network; use that only on a trusted network and open it through the machine's actual IP address.

1. A game package defines RAM structures, training states, rewards, and action translation.
2. The environment factory creates vectorized emulator workers and processed visual observations.
3. A model builder creates or restores the selected policy.
4. The trainer coordinates learning, checkpoints, replay capture, telemetry, and optional uploads.
5. The dashboard exposes the active run without coupling the learner to a separate monitoring service.

Run Python quality checks:

ruff check src
black --check src
python -m compileall -q src

Build the dashboard assets after changing the Vue frontend:

cd src/datenwissenschaften/ui/frontend
npm ci
npm run build

Copyright © datenwissenschaften contributors. Distributed under the GNU General Public License v3.0.