scan2simready

Convert raw 3D scan meshes into NVIDIA Isaac Sim SimReady USD assets.

Scans from photogrammetry, LiDAR, or structured light carry no collision geometry, no mass, no inertia, and no friction, and they usually arrive in the wrong units and up-axis. Getting one to behave correctly in Isaac Sim normally means manual mesh cleanup, convex decomposition, and USD physics schema authoring by hand. scan2simready takes a scan and writes a .usd asset with a cleaned visual mesh, CoACD convex collision hulls, a measured mass and inertia tensor, friction defaults, and semantic labels.

Mass and inertia on open scans

Volume, mass, centre of mass, and inertia are defined only for a closed surface, and scans are rarely closed. Measuring the convex hull instead is safe but wrong for anything concave, and a single speck of floating scan noise can inflate the hull to several times the object's real size.

The pipeline reconstructs a closed body before it measures anything:

1. Drop disconnected floaters (scan noise) below a fraction of surface area.
2. Fix face winding and normal orientation.
3. Fill boundary loops. trimesh.fill_holes only closes 3- and 4-vertex holes, so it never touches the large opening left where the scanner never saw the underside. Open loops of any length are capped with a centroid triangle fan.
4. If the mesh still will not close, voxelise the surface, morphologically close the scanner's gaps, flood-fill the interior, and integrate mass properties over the occupied voxels. This follows concavities that a convex hull bridges over.
5. Fall back to the convex hull only if all of the above fails, and report that in the log, in the CLI output, in the USD's customData, and in the batch manifest.

On a test scan of a 0.2 × 0.3 × 0.1 m box with its underside missing and one floating speck, the convex-hull fallback measures 0.0255 m³ and repair measures 0.006000 m³, the exact true volume. At steel density that is a 200 kg asset versus a 47 kg one.

Features

Current

• Load OBJ / GLB / GLTF / PLY / STL scan meshes (via trimesh)
• Mesh cleanup: degenerate/duplicate face removal, vertex welding, optional quadric decimation to a target face count
• Scan repair: floater removal, winding/normal fixes, arbitrary-length hole filling, voxel solid reconstruction. The volume's provenance and trustworthiness are reported throughout
• Convex decomposition with CoACD (configurable concavity threshold, single-hull fallback)
• Mass from measured volume × density, or set directly with --mass
• Inertia tensor integrated over the reconstructed solid and authored as diagonalInertia + principalAxes. Without it PhysX derives inertia from the collision hulls, which are coarser than the real shape, and the asset tumbles wrong
• Material presets (--material steel) supplying density and friction
• Visual materials: UsdPreviewSurface with base colour from the scan's texture, baseColorFactor, or mean vertex colour; textures extracted alongside the USD and referenced relatively
• Semantic labels via UsdSemantics.LabelsAPI and the legacy Isaac Replicator attributes, so both current and older tooling can read them
• Batch mode: a directory of scans in, an asset library plus manifest.json out, with per-asset error isolation
• Isaac Sim validation script that drops each asset on a ground plane and reports settling, penetration, drift, and mass round-trip
• Sim conventions: kilograms, metres (metersPerUnit = 1), Z-up

Roadmap

• Full PBR material graphs (normal/roughness/metallic from scan textures)
• Per-part material segmentation rather than one density for the whole asset
• Articulated assets (joints, drawers, hinges) rather than single rigid bodies
• Scan-to-scan registration for multi-capture objects

Install

Requires Python 3.10+.

git clone https://github.com/steefpls/scan2simready.git
cd scan2simready
python -m venv .venv
# Windows: .venv\Scripts\activate    |    Linux/macOS: source .venv/bin/activate
pip install -e .[dev,decimation]

Usage

# single asset
scan2simready input.obj -o output.usd

# a scanned steel bracket, labelled for perception, decimated
scan2simready bracket.ply -o bracket.usd --material steel \
    --label bracket --target-faces 20000

# you measured the real mass; use it instead of guessing from density
scan2simready chair.glb -o chair.usd --mass 6.4

# a whole scan library -> an asset library + manifest.json
scan2simready ./scans -o ./library --label furniture

Key options (--help for the full list):

Flag	Default	Meaning
-o, --output	<input>.usd	Output USD path, or output directory in batch mode
--material	none	Preset supplying density and friction (steel, pla, wood, …)
--density	700	Material density in kg/m³ for mass estimation
--mass	none	Set mass in kg directly and back-solve density
--target-faces	off	Decimate visual mesh to ~N faces
--coacd-threshold	0.05	CoACD concavity threshold (lower = tighter hulls)
--label	none	Semantic label; repeatable
--no-repair	off	Skip repair entirely (mass then falls back to the convex hull)
--min-component-fraction	0.01	Drop islands below this fraction of surface area
--voxel-resolution	64	Voxels along the longest edge when reconstructing a solid
--max-hole-edges	none	Refuse to fill boundary loops longer than this
--static-friction / --dynamic-friction	0.5 / 0.4	Friction coefficients
--restitution	0.1	Bounciness
--skip-existing	off	Batch mode: leave already-converted assets alone

The CLI exits non-zero if any asset fails.

Authored USD layout

/<AssetName>                     Xform, defaultPrim, kind=component
    + UsdPhysics.RigidBodyAPI
    + UsdPhysics.MassAPI         mass, centre of mass, diagonalInertia, principalAxes
    + UsdSemantics.LabelsAPI     (+ legacy Isaac semantic attributes)
    /Visual/<AssetName>          UsdGeom.Mesh with normals, st primvar
    /Looks/<AssetName>Material   UsdPreviewSurface (+ UsdUVTexture if textured)
    /Collisions/hull_NN          UsdGeom.Mesh, purpose=guide, CollisionAPI,
                                 convexHull approximation
    /PhysicsMaterials/           friction, restitution, density

customData["scan2simready"] records volume, volumeSource, and volumeIsTrustworthy, so a downstream consumer can see how the mass was derived without re-running the pipeline.

Validating in Isaac Sim

Schema-correct USD is not necessarily physically sane. The validation script drops each asset on a ground plane:

# with Isaac Sim's own interpreter, not a plain python
./python.sh scripts/validate_in_isaac.py library/ --json report.json

It reports, per asset: whether PhysX read back the authored mass, how far the body sank into the floor (hull/visual mismatch), how long it took to settle, and how far it drifted and tumbled. Exit code is 0 only if every asset passes.

Tests

pytest

The suite covers the mass-property maths against closed-form answers (a solid box's inertia tensor), the repair path against a synthetic bad scan, the authored USD contract, and batch error isolation.

Status

Early stage. The end-to-end pipeline runs, mass properties are validated against analytic values, and every estimate reports its own provenance. Expect rough edges on messy real-world scans; APIs and CLI flags may change without notice.