Multi-View

Observing the same scene from multiple cameras or multiple angles at once.

Multi-view refers to observing the same scene from two or more positions, whether that's several cameras shooting at once or a single camera moving while it shoots. It mainly shows up in two places: 3D vision, where images from different angles are used for triangulation, multi-view stereo reconstruction, or training a NeRF; and robot learning, where a robot commonly feeds a head camera, a wrist camera (close to the gripper), and a fixed third-person camera into the policy together. The benefit is reduced occlusion and depth information that a single image can't provide; research has also found that adding a wrist view improves training efficiency and generalization to new scenes. The cost is more overhead for calibration, synchronization, and compute, and every camera's intrinsics and extrinsics need to be recorded.

ExampleThe DROID dataset's collection rig pairs two adjustable-position ZED 2 stereo cameras with one wrist-mounted ZED Mini, collecting 76,000 trajectories in total across 1,417 camera viewpoints.

Also called

Multi-Camera

Related

Wrist Camera · Head Camera · Third-Person Camera · Multi-View Stereo · Occlusion · Camera Extrinsics

Sources

DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset
Vision-Based Manipulators Need to Also See from Their Hands (ICLR 2022)