FFmpeg

FFmpeg（音视频编解码工具）Advanced

An open-source audio and video processing toolkit, used for transcoding, cutting, frame extraction, and compression.

FFmpeg is a set of open-source audio and video processing tools and libraries, including command-line programs like ffmpeg and ffprobe as well as codec libraries such as libavcodec, and it supports nearly every common audio and video format. In embodied AI, robot datasets need to store huge amounts of camera footage, and storing every frame as a separate image file takes up far too much space, so footage is usually encoded into video files like MP4 and decoded back into frames at training time. The LeRobot dataset format stores each camera's footage as video, with encoding and decoding relying on FFmpeg or libraries built on it. It's also used for extracting frames for annotation, aligning sample rates, and compressing demo videos.

ExampleRunning ffmpeg -i episode_0.mp4 -vf fps=10 frames/%05d.png extracts a 30-frames-per-second demonstration video into images at 10 frames per second.

Related: LeRobotDataset · Data Cleaning · Multi-sensor Time Synchronization / Timestamp Alignment · Demonstration Data
Sources: FFmpeg 官网 (Chinese)
LeRobot GitHub