Add cameras as first-class objects in ModelBuilder/Model

[eric-heiden]

Description

Adds cameras as first-class objects in Newton, targeted at dev/sensor-batched-camera so the model-side camera data can feed SensorBatchedCamera (newton-physics#3276) directly.

Camera data model & core module (newton/_src/core/cameras.py)

• CameraProjection descriptors: CameraPinhole (physical/USD form + from_fov), CameraFisheyeOpenCV / CameraFisheyeFTheta / CameraFisheyeKannalaBrandt, and CameraCustomRays for user-supplied ray bundles (custom camera models).
• Equal parametric projections dedupe at finalize(); CameraCustomRays shares by object identity — rays are never duplicated and never stored on the Model. Resolution is a per-camera hint that the renderer overrides (custom-ray cameras derive it from the ray array shape).
• Shared camera math: ray-generation kernels (moved from warp_raytrace/camera_utils.py with back-compat re-exports), compute_camera_rays, eval_camera_world_xforms, pitch/yaw basis + fov/focal helpers now used by ViewerGL/Viser/RTX (removes the duplicated viewer math).

ModelBuilder / Model

• builder.add_camera(body=-1, *, xform, projection, resolution, enabled, label, custom_attributes); cameras follow world semantics (camera_world, camera_world_start), have labels, optional rigid-body attachment (camera_body), CameraFlags, and AttributeFrequency.CAMERA custom attributes; full add_builder/replicate support (shared projections across replicas).

Importers

• USD: UsdGeom.Camera prims (load_cameras=True, path_camera_map, body attachment via nearest rigid-body ancestor, orthographic warn+skip).
• MJCF: <camera> elements incl. fovy/focal/sensorsize/principal/resolution, frames, defaults; non-fixed modes import as fixed with the original mode preserved in the mjcf:camera_mode custom attribute.

Viewers

• viewer.show_cameras batched frustum visualization (single instanced draw, scales to hundreds of cameras, respects visible worlds) and viewer.set_camera_from_model(index_or_label) + ViewerGL camera dropdown with follow mode.

Integration with this branch

• Rebuilt directly on dev/sensor-batched-camera — no upstream-main delta is included; the diff is the camera feature only (26 files).
• The camera work was adapted to this branch's builder/model machinery (dict-based attribute frequencies, add_builder offset table extended with the camera entity) and leaves SensorBatchedCamera untouched (its 22 tests stay green).

Note: fisheye descriptor defaults follow the existing ray-helper defaults (max_fov=2π, FTheta k1=1.0) rather than the draft design doc, so descriptors map 1:1 onto the ray helpers.

Follow-up (separate PR): SensorBatchedCamera.select_cameras() consuming model cameras, lazy per-(projection, resolution) ray-bundle cache vs in-kernel ray computation with a deciding benchmark.

Checklist

• New or existing tests cover these changes
• The documentation is up to date with these changes (docs/concepts/cameras.rst, API registry)
• CHANGELOG.md has been updated

Test plan

uv run --extra dev -m newton.tests -k test_camera_projections
uv run --extra dev -m newton.tests -k test_camera_rays
uv run --extra dev -m newton.tests -k test_model_cameras
uv run --extra dev -m newton.tests -k test_import_mjcf_cameras
uv run --extra dev -m newton.tests -k test_import_usd_cameras
uv run --extra dev -m newton.tests -k test_viewer_model_cameras
uv run --extra dev -m newton.tests -k test_sensor_batched_camera   # 22/22 on this branch
uv run --extra dev -m newton.tests -k test_sensor_tiled_camera     # 34/34
uv run --extra dev -m newton.tests                                  # full suite: 4045 tests OK

New feature / API change

import math
import newton

builder = newton.ModelBuilder()
builder.add_mjcf("robot.xml")   # <camera> elements imported automatically
builder.add_usd("scene.usda")   # UsdGeom.Camera prims imported automatically

cam = builder.add_camera(
    body=hand_body,             # optional rigid-body attachment (-1 = world-fixed)
    xform=wp.transform((0.0, 0.0, 0.1), wp.quat_identity()),
    projection=newton.CameraPinhole.from_fov(math.radians(60.0)),
    resolution=(640, 480),      # hint; renderer-side resolution overrides
    label="wrist_cam",
)

model = builder.finalize()
model.camera_transform          # wp.array[wp.transform], user-mutable
model.camera_projections        # deduplicated descriptor table
xforms = newton.eval_camera_world_xforms(model, state)  # follows body_q

viewer.show_cameras = True                  # batched frustum visualization
viewer.set_camera_from_model("wrist_cam")   # adopt a model camera's viewpoint

🤖 Generated with Claude Code