Public archive note: I built AgentVille in 2020 and am making it public now. The project and benchmark are kept in their 2020 form.
Unity 2020.1.4f1 ML-Agents / DOTS distributed RL playground.
Open ml-agents/Project in Unity. Custom code lives in ml-agents/Project/Assets/AgentVille.
- Tabular Q-learning grid navigation with body-cost reward shaping.
- Switchable AI/player agents, camera modes, deterministic moving obstacles, and presentation controls.
- DOTS/ECS factory-courier scale scene with
100,000simulated actors. - Exploration cohorts, novelty reward, prioritized replay, recurrent learner, fixed-seed greedy eval, and ablation-style policy comparison.
- Best measured policy in current evals is tabular-greedy.
Internal Unity DOTS factory-courier benchmark, 100,000 agents, 10 x 120s, TimeScale 20, fixed seeds 1337-1346, learning off, novelty off, epsilon 0. Goals are completed deliveries.
| Policy | Runs | Goals/run | 95% CI | Goals/sec | Decisions/sec | Success | Mean steps | Result |
|---|---|---|---|---|---|---|---|---|
| zero-Q tie-random | 10 | 102.4 | +/- 72.9 | 0.85 | 6.41M | 0.10% | 2,125.2 | weak baseline |
| tabular-greedy | 10 | 1,222,687.1 | +/- 15,000.6 | 10,189.06 | 7.65M | 91.12% | 455.3 | strongest measured policy |
| recurrent-greedy | 10 | 451,964.3 | +/- 216,953.1 | 3,766.37 | 1.22M | 65.17% | 169.6 | high variance recurrent policy |
Results are based on a custom internal benchmark.
Scale Run combines 100k ECS actors, exploration cohorts, novelty reward, prioritized replay, recurrent Q learning, fixed-seed greedy eval, and policy ablations.
- Uses Unity DOTS/Entities preview packages.
- ECS agents run pickup/dropoff courier tasks, track reward/progress, and retarget after deliveries.
- Q-grid controls:
Tabswitches controlled agent,Vcycles camera views, WASD/arrows move the agent. - Main curriculum controls switch stages, camera/view, Demo Run/Scale Run, reset, and zoom.