Multi-Sensor Time Synchronization

多传感器时间同步Common

Making sure data from different sensors used together actually comes from the same moment in time, not slightly different ones.

A robot's camera, depth camera, lidar, IMU, and joint encoders each run on their own clock and sample rate. Time synchronization ensures that the different pieces of data used together in a computation actually come from the same instant. It works on two levels: hardware synchronization, where a trigger line makes multiple devices expose at the same moment, or PTP (the IEEE 1588 Precision Time Protocol, which can achieve sub-microsecond accuracy on a local network) lets devices share a single clock; and software alignment, where every piece of data gets a timestamp and is then paired or interpolated by timestamp — ROS's message_filters package, for instance, offers exact-match and approximate-match policies for this. Even a gap of a few tens of milliseconds can misalign a point cloud and an image when the robot is moving quickly, degrading the accuracy of visual-inertial odometry, and it can likewise misalign the observation-action pairs collected for training, making a trained policy's actions lag behind what it sees. It's a prerequisite for multi-sensor fusion, calibration, and data collection generally.

ExampleDuring dual-arm teleoperation data collection, several cameras output frames at 30 Hz while joint states are reported at a higher frequency. When saving the data, each image's timestamp must be matched to the closest joint reading, or the action labels end up offset from the images.

Also called: Timestamp Alignment, Hardware Synchronization, Hard Sync / Soft Sync
Related: Multi-Sensor Fusion · Precision Time Protocol · Camera-IMU Calibration · Observation-Action Pair · Visual-Inertial Odometry · ROS Bag
Sources: Wikipedia: Precision Time Protocol
ROS 2 message_filters 文档（ExactTime / ApproximateTime 同步策略） (Chinese)
Orbbec Gemini 335L（硬件触发与多机统一硬件时间戳） (Chinese)