A personal lab environment for hands-on experimentation and benchmarking of 3D reconstruction pipelines — SfM, Gaussian Splatting variants, and mesh extraction engines — on Blackwell hardware.
- NVIDIA GPU with CUDA support (16GB+ VRAM recommended)
- CUDA Toolkit 12.8+ (required for
sm_120; 12.4+ for older architectures) - PyTorch 2.11.0+cu128 (Stable,
sm_120native CUBIN required for Blackwell)
- GPU: NVIDIA GeForce RTX 5070 Ti (Blackwell Architecture,
sm_120, 16GB VRAM) - CUDA: CUDA 12.8 Toolkit / PyTorch 2.11.0+cu128
- Host OS: Ubuntu 24.04 LTS (GCC 12 host compiler via
-ccbin /usr/bin/g++-12) - CUBIN Binaries: Native
sm_120SASS CUBIN with12.0+PTXfallback - Rasterizer Performance Benchmark (RTX 5070 Ti, 1,112-frame indoor scene):
- PTX JIT Fallback (
sm_90SASS + JIT onsm_120):~37.18 it/s - Native
sm_120SASS CUBIN:~66.09 it/s(~1.78xthroughput performance gain)
- PTX JIT Fallback (
# 1. Apply tracked patches and verify environment
./scripts/utils/apply_patches.sh
./scripts/utils/verify_patches.sh
# 2. Run Step 1 Camera Pose Estimation (COLMAP, hloc, VGGT, FastMap)
./scripts/run_3drc.sh sfm
# 3. Run Step 2 Gaussian Training (Inria 3DGS / PlanarGS / 2DGS / Scaffold-GS)
./scripts/run_3drc.sh train 3dgs
./scripts/run_3drc.sh train planargs
./scripts/run_3drc.sh train 2dgs
./scripts/run_3drc.sh train scaffoldgs
# 4. Run Step 3 Mesh Reconstruction (SuGaR / MILo / TSDF)
./scripts/run_3drc.sh sugar
./scripts/run_3drc.sh milo
./scripts/run_3drc.sh tsdf
# 5. Evaluate 3D Mesh Topology & Geometric Accuracy
./scripts/run_3drc.sh eval outputs/<scene_name>/mesh/milo/mesh_cleaned_largest.ply
./scripts/run_3drc.sh eval outputs/<scene_name>/mesh/2dgs/tsdf_mesh.ply data/<scene_name>/lidar.pcd
# 6. Inspect Generated Artifacts
./scripts/run_3drc.sh outputs3drc: Main pipeline orchestrator, Step 1 SfM (COLMAP, hloc, VGGT, FastMap), preprocessing (dense_undistort), and Step 5 geometric evaluation (eval_mesh).gs_train: Inria 3DGS, PlanarGS, and 2DGS training (dr_aa&diff-surfel-rasterization), plus Open3D TSDF mesh extraction.gs_sugar: SuGaR surface-aligned mesh extraction environment.gs_milo: MILo differentiable mesh-in-the-loop training and extraction environment.gs_group: Gaussian Grouping 3D SAM instance segmentation environment.
| Tool | Pipeline Step | Core Role | Methodology |
|---|---|---|---|
| COLMAP | Step 1 (SfM) | Camera pose estimation | SIFT keypoint extraction + Incremental SfM BA |
| hloc | Step 1 (SfM) | Feature-matched camera pose estimation | SuperPoint + SuperGlue with COLMAP BA |
| VGGT-Omega | Step 1 (SfM) | Feed-forward pose estimation | Visual Geometry Transformer pose initialization |
| FastMap | Step 1 (SfM) | GPU-accelerated pose estimation | Fast keypoint matching and mapping |
| 3DGS (Inria) | Step 2 (Training) | 3D Gaussian scene optimization | Differentiable 3D Gaussian rasterization |
| PlanarGS | Step 2 (Training) | Planar-regularized 3DGS | Planar priors on detected indoor surfaces |
| 2DGS | Step 2c (Training) | Planar surfel representation | Exact ray-splat intersection with 2D oriented disks |
| Scaffold-GS | Step 2d (Training) | Anchor-based 3DGS representation | Learnable anchors for view-dependent attribute prediction |
| SuGaR | Step 3 (Mesh Extraction) | UV-textured OBJ polygon mesh | Surface-Aligned Gaussian Regularization |
| MILo | Step 3b (Mesh Extraction) | Compact collision mesh | Differentiable Mesh-in-the-loop MT SDF |
| TSDF (Open3D) | Step 3c (Mesh Extraction) | Volumetric polygon mesh | Volumetric Open3D TSDF Integration from depth/normals |
| Gaussian Grouping | Step 4 (Segmentation) | 3D object instance segmentation | 3D Identity Embedding from SAM masks |
All submodules in third_party/ reference official upstream repositories directly. Custom modifications (e.g. Blackwell sm_120 CUBIN build fixes, camera model extensions) are maintained as versioned patch files in patches/.
For the complete patch catalog, patch application, and verification workflows, refer to the Submodule Patches Guide.
For in-depth technical guides, execution options, and evaluation methodologies, refer to the documentation in docs/:
- Pipeline Architecture Guide: Complete end-to-end workflow from Dataset Preparation (Step 0) to SfM, Gaussian Training, Mesh Extraction, and Quantitative Evaluation (Step 5).
- Blackwell (sm_120) Build Notes: Troubleshooting matrix and native sm_120 CUBIN build guide for RTX 50 Series.
- Submodule Patches Guide: Patch maintenance,
apply_patches.sh, andverify_patches.shmechanisms for official upstream submodules. - Architecture Decision Records (ADR): Architecture decision records with operational writing rules.