About the survey From physical fidelity to generative priors: a comprehensive survey on inverse rendering
Inverse rendering recovers scene quantities that a forward renderer needs—geometry, surface materials, lighting, environment, etc.—from observed images so that re-projection, relighting, or edits stay visually consistent. The task is usually severely ill-posed and relies on explicit or implicit assumptions and priors.
The survey's narrative runs from physical fidelity—physical shading, differentiable rendering, and explicit or semi-explicit representations fit in an optimization loop to match the imaging model—toward generative priors: neural representations learned at scale (e.g. radiance fields), explicit point or Gaussian primitives, and diffusion and other generative models that reduce ambiguity and improve recovery in real captures. On that basis it typically organizes methods, capabilities (decomposition granularity, editability, novel views), and trends.
- Physical fidelity — explicit geometry / differentiable rendering
- Neural implicit — NeRF-style decomposition
- 3D Gaussian splatting
- Generative priors — intrinsics, diffusion & generative models
- Direct prediction — end-to-end & classical pipelines
- MIRReS — Mirror-aware Inverse Rendering with Synthetic-data Supervision · [ICLR'25] [Paper]
- NVdiffrec — Extracting Triangular 3D Models, Materials, and Lighting From Images · [CVPR'22] [Paper] [Code]
- NVdiffrecmc — Extracting Objects and Their Materials From Images Using Differentiable Rendering and Denoising · [NeurIPS'22] [Paper] [Code]
- DIB-R++ — Learning to Predict 3D Objects With an Interpolation-Based Differentiable Renderer · [NeurIPS'21] [Paper]
- JOC — Joint Objective-Consistent Inverse Rendering · [SIGGRAPH'23] [Paper]
- INvRender — [CVPR'22] [Paper]
- NeRD — NeRD: Neural Reflectance Decomposition from Image Collections · [ICCV'21] [Paper] [Project Page] [Code]
- PhySG — PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting · [CVPR'21] [Paper] [Code]
- NeRFactor — NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination · [SIGGRAPH Asia / TOG'21] [Paper] [Project Page] [Code]
- TensoIR — TensoIR: Tensorial Inverse Rendering · [Paper]
- TensoSDF — [Paper]
Gaussian splatting is closely related to inverse rendering. The inverse-rendering block is split by material / appearance complexity: diffuse-dominated and simpler appearance vs specular highlights, transmission, and harder BRDFs.
- Gaussian Surfels — High-quality Surface Reconstruction using Gaussian Surfels · [SIGGRAPH'24] [Paper] [Project Page]
- GlossGau — [Paper]
- GlossyGS — Inverse Rendering of Glossy Objects with 3D Gaussian Splatting · [TVCG'25] [Paper] [Project Page]
- GS-2DGS — [CVPR'25] [Paper]
- gs-ssr — [Paper]
- GUS-IR — [Paper]
- InvGS — [Paper]
- RGS-DR — [Paper]
- Geosplatting — [ICCV'25] [Paper]
- GI-GS — [ICLR'25] [Paper]
- GIR — [TPAMI'24] [Paper]
- GS-IR — 3D Gaussian Splatting for Inverse Rendering · [CVPR'24] [Paper] [Project Page] [Code]
- PhyGap — [CVPR'26] [Paper]
- Ref-Gaussian — [ICLR'25] [Paper]
- Ref-GS — [CVPR'25] [Paper]
- RTR-GS — [Paper]
- Sun SVG-IR — [CVPR'25] [Paper]
- SVG-IR Supplementary — (same work as Sun / SVG-IR; supplementary PDF only)
- TransparentGS — [SIGGRAPH'25] [Paper]
Image-space and learned-prior methods (diffusion, intrinsic decomposition); complements explicit 3D + differentiable rendering sections.
- Diffusion Ren — [CVPR'25] [Paper]
- DNF-Intrins — [ICCV'25] [Paper]
- IntrinsicAnything — [Paper]
- IntrinsicDiffusion — [Paper]
- Kocsis — [CVPR'24] [Paper]
- MaterialFusion — [Paper]
- rgbx — [Paper]
- DL3DV-10K — A Large-Scale Scene Dataset for Deep Learning-based 3D Vision · [CVPR'24] (large-scale scene video; common for 3D / inverse rendering / NVS) · [Paper] [Project Page] [Code]