ONNX Runtime

Microsoft's open-source, cross-platform engine for running models saved in the ONNX format.

ONNX Runtime is an open-source inference engine from Microsoft that executes models in the ONNX format, a general-purpose file format for exchanging neural networks between frameworks. After a model is trained in a framework like PyTorch, it can be exported to ONNX and then run with ONNX Runtime on Windows, Linux, phones, and embedded devices, without needing the original training framework installed. It connects to different hardware backends through “execution providers” — CPU, CUDA, TensorRT, OpenVINO, CoreML, and others — so the same model can use the right hardware acceleration just by changing a setting. In robotics, control policies trained with reinforcement learning are often deployed as ONNX files and run in real time on an onboard computer using ONNX Runtime.

ExampleExport a quadruped walking policy trained in Isaac Lab to policy.onnx, then run it at 50 Hz on the robot dog's onboard CPU with ONNX Runtime.

Also called

ORT, onnxruntime

Related

Open Neural Network Exchange (ONNX) · NVIDIA TensorRT · Intel OpenVINO · Inference Deployment · On-Device / Edge Deployment · RL-based Locomotion Control

Sources

ONNX Runtime 官网 (Chinese)
microsoft/onnxruntime (GitHub)