Habitat-Matterport 3D Dataset

A dataset of 1,000 real buildings, 3D-reconstructed by Meta and Matterport, used to train navigation agents.

HM3D is an indoor scene dataset released jointly in 2021 by Meta AI (FAIR) and the property-scanning company Matterport. It contains 1,000 building-scale, textured 3D meshes covering homes, stores, and public buildings, with 112,500 square meters of navigable area combined — 1.4 to 3.7 times more than earlier datasets like Matterport3D and Gibson — and with fewer reconstruction defects. It's mainly used together with the Habitat simulator: an agent practices tasks like point-goal navigation and object-goal navigation inside digital copies of these real buildings. The paper shows that a point-goal navigation agent trained on HM3D also performs best when tested on other datasets. A later release, HM3D-Semantics v0.2, added 142,646 object-instance annotations across 216 of the spaces; v0.1 was the basis for the Habitat 2022 Object-Goal Navigation Challenge. The data is licensed for academic, non-commercial use only.

ExampleLoading one HM3D house into Habitat, a robot agent starts at the front door and must find the location of the “sofa” using only its camera feed.

Also called

HM3D, HM3D-Semantics (HM3DSem)

Related

Habitat · Matterport3D · Object-Goal Navigation · Point-Goal Navigation · Navigation · ScanNet

Sources

Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI (arXiv)
HM3D - AI Habitat
HM3D-Semantics - AI Habitat