3D Gaussian Splatting

Representing a scene as many colored 3D Gaussian ellipsoids that can be rendered from any new viewpoint in real time.

3D Gaussian splatting was proposed by Bernhard Kerbl and colleagues at Inria and other institutions, published at SIGGRAPH 2023. It represents a scene as a large number of 3D Gaussian distributions — imaginable as semi-transparent colored ellipsoids — each with its own position, shape and orientation, opacity, and color that can vary with viewing angle. The method first runs structure-from-motion on a set of multi-view photos to recover camera poses and a sparse point cloud for initialization, then uses differentiable rasterization to repeatedly compare a rendered image against the real photos and optimize the Gaussians accordingly. The paper reports real-time novel-view synthesis at 1080p, no less than 30 frames per second, much faster than neural radiance fields (NeRF). In embodied AI it's often used to bring real scenes into simulation, to perform Gaussian-splatting SLAM, or to render novel views for data augmentation.

ExampleWalking around a table taking photos, then using COLMAP to recover the camera poses and a sparse point cloud and training 3DGS on them, lets you view that table from any angle in real time on a computer.

Also called

3DGS, Gaussian Splatting, GS

Related

Neural Radiance Fields · Novel View Synthesis · Structure from Motion · Gaussian Splatting-based Simulation · Gaussian Splatting SLAM · Real-to-Sim

Sources

3D Gaussian Splatting for Real-Time Radiance Field Rendering (project page, Inria)
arXiv 2308.04079: 3D Gaussian Splatting for Real-Time Radiance Field Rendering