UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videos

ICLR, 2026

---

Mingxuan Liu^1,2,*, Honglin He^1,*, Elisa Ricci^2,3, Wayne Wu¹, Bolei Zhou¹ (*Equal contribution)

_{¹University of California, Los Angeles    ²University of Trento    ³Fondazione Bruno Kessler}

---

Introducing UrbanVerse — a system that converts real-world urban scenes from city-tour videos into physics-aware, interactive simulation environments enabling scalable robot learning in urban spaces with real-world generalization.

Click the image below to watch the introductory video.

Updates

• Sep 05, 2026 - Sim-ready conversion complete: all 387 scenarios and 102,438 assets are on HuggingFace — see below
• Sep 02, 2026 - Sim-ready (Isaac Sim ≥ 5) releases on HuggingFace: CraftBench, Training Scenes and Assets — see below
• Aug 11, 2026 - Converting all assets & scenarios to sim-ready: in progress — see live progress
• Jul 08, 2026 - UrbanVerse simulation-ready urban scenes are released here
• Mar 19, 2026 - CraftBench scenes are released here
• Mar 06, 2026 - UrbanVerse-100K 3D asset database is released here
• Oct 20, 2025 - UrbanVerse paper is now available here

Sim-Ready Conversion Progress

All UrbanVerse assets & scenarios have been converted to simulation-ready (Isaac Sim ≥ 5) format and published on HuggingFace.

Assets: █████████████████████████ 102,438 / 102,438 (100.0%)

Scenarios: █████████████████████████ 387 / 387 (100.0%)

_{Assets: 102,438 of the 102,444 GLBs in UrbanVerse-100K; the other 6 are empty files upstream (ids in scenes/assets_batches_state.json). Scenarios: 375 training scenes + 12 CraftBench scenes. Bars are maintained with scripts/update_progress.py.}

Releases

	HuggingFace	source	what changed
Training scenes	UCLA-VAIL/UrbanVerse-Training-Scenes-Sim-Ready	all 375 generated scenes (66 city walks; the original UrbanVerse-Training-Scenes release holds the first 107)	ground-material texture_scale re-authored as float2 so roads/sidewalks/terrain render in Isaac Sim 5.x; one scene.tar per scene (1.0 TB total) plus one preview still per city walk
CraftBench scenes	UCLA-VAIL/UrbanVerse-CraftBench-Sim-Ready	UrbanVerse-CraftBench (12)	same fix, same per-scene tar packaging
Assets	UCLA-VAIL/UrbanVerse-Assets-Sim-ready	UrbanVerse-100K	GLB → USD: Z-up, metric, base on z=0, rigid body + convex-decomposition collision; one tar per asset (usd/<uid[:2].lower()>/std_<uid>.tar, 1.78 TB)

_{Why scenes needed fixing: Isaac Sim 4.5 rejected the scenes' int texture_scale and fell back to each MDL's default tile size; Isaac Sim 5.x rejects it too but then renders the surface flat black/white. The sim-ready scenes bake that default in as a proper float2, so 5.x shows what 4.5 showed. Tooling: scripts/upgrade_isaacsim5.md, scripts/convert_scenes_simready.py; per-scene provenance in scenes/.}

Open-Source Roadmap (priority order)

• UrbanVerse-100K Dataset: release annotations and rescaled assets
• UrbanVerse-100K Dataset Toolkit: release Python package urbanverse-asset
• CraftBench scenes: release the 10 professionally designed test-only scenes
• Scene Toolkit: release Python package urbanverse-scene
• UrbanVerse Scenes Repository: release the simulation-ready urban scenes generated by UrbanVerse — available here
• All Assets & Scenarios Sim-Ready: full corpus converted for Isaac Sim ≥ 5 — details; releases: CraftBench, training scenes, assets
• UrbanVerse-Gen Pipeline: release the real-to-sim automatic scene construction code
• RL Training Pipeline: release in-simulation policy training and evaluation code

Status will be updated as each component is released. If you use UrbanVerse, please cite the paper.

Citation

@inproceedings{
    liu2026urbanverse,
    title={UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videos},
    author={Mingxuan Liu and Honglin He and Elisa Ricci and Wayne Wu and Bolei Zhou},
    booktitle={The Fourteenth International Conference on Learning Representations},
    year={2026},
}