Pointmap

An array the same size as an image, where each pixel stores a 3D coordinate instead of a color.

A pointmap arranges the 3D coordinate (x, y, z) corresponding to every pixel of an image into an H×W×3 array, giving a one-to-one correspondence between pixels and 3D points. DUSt3R, proposed in 2023 by Naver Labs Europe and others, made this its core output: given two images, the network directly regresses two pointmaps, both expressed in the camera coordinate frame of the first image, so it needs no prior knowledge of camera intrinsics (such as focal length) or camera poses. Compared with a depth map, a pointmap implicitly encodes depth, camera parameters, and pixel correspondence between the two images all at once, and the focal length, relative pose, and matching points can all be recovered from it. Later feed-forward 3D reconstruction models such as MASt3R, VGGT, and π³ adopted or remained compatible with this representation, letting robots quickly recover a scene’s 3D structure from ordinary RGB images.

ExampleTwo phone photos of a tabletop are fed into DUSt3R, producing two pointmaps that, combined, form a dense point cloud of the tabletop scene.

Also called

Point Map, Per-Pixel 3D Point Map

Related

Feed-Forward 3D Reconstruction · DUSt3R · VGGT · Depth Map · Point Cloud · Camera Intrinsics

Sources

DUSt3R: Geometric 3D Vision Made Easy (arXiv 2312.14132)