Gaussian Splatting SLAM

A family of SLAM methods that use 3D Gaussian splatting as the map, building a photorealistically renderable scene while localizing.

This is a family of SLAM methods that use 3D Gaussian splatting — representing a scene with a large number of colored, translucent 3D Gaussian ellipsoids that render very fast — as the map representation, appearing in a cluster starting in late 2023. Landmark examples include SplaTAM (Carnegie Mellon University and others, CVPR 2024, taking RGB-D video as input) and MonoGS (Imperial College London, Andrew Davison’s group, CVPR 2024, the first monocular version, running in real time at about 3 frames per second). During tracking, the current map is rendered into an image and compared against the real observation, optimizing the camera pose in reverse; during mapping, Gaussians are added, removed, and adjusted. Compared with a point-cloud or TSDF map, this approach gets a dense map that can be rendered from any new viewpoint at the same time as localization, which suits reality-to-simulation, digital twins, and navigation mapping for robots.

ExampleSplaTAM scans a room once with a handheld RGB-D camera, estimating the camera trajectory while building a Gaussian map at the same time; afterward, the room can be rendered from a viewpoint that was never actually photographed.

Also called

3DGS SLAM, SplaTAM, MonoGS

Related

3D Gaussian Splatting · Simultaneous Localization and Mapping · Visual SLAM · Neural Radiance Fields · Novel View Synthesis · Real-to-Sim

Sources

SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM (arXiv 2312.02126)
Gaussian Splatting SLAM / MonoGS (arXiv 2312.06741)

As of

2024-06