9D Rotation Representation

Lets a network output nine unconstrained numbers, then uses SVD to project them onto the nearest valid rotation matrix.

The 9D rotation representation is a way for a neural network to predict a 3D rotation: the network outputs nine unconstrained real numbers arranged into a 3×3 matrix M, which is then factored by singular value decomposition, M = UΣVᵀ. Taking R = U·diag(1, 1, det(UVᵀ))·Vᵀ gives the rotation matrix closest to M, with the det term ensuring the result is not a mirror reflection. Levinson et al. gave a systematic analysis in their NeurIPS 2020 paper 'An Analysis of SVD for Deep Rotation Estimation': the mapping is smooth almost everywhere, unlike quaternions or Euler angles, which can jump discontinuously; under noisy inputs, the expected reconstruction error is about half that of the Gram-Schmidt orthogonalization used by the 6D representation; and it achieved state-of-the-art results at the time on tasks including point-cloud alignment, object pose estimation, and inverse kinematics. Whenever a network must directly regress an object's or end-effector's orientation, this is a common alternative to quaternions or Euler angles, alongside the 6D representation.

ExampleThe paper's official code implements this in about ten lines: reshape the network's [batch, 9] output into 3×3 matrices, run SVD, correct the sign of the last column using det(UVᵀ), and obtain a [batch, 3, 3] rotation matrix that can be attached directly to the end of any regression network for end-to-end training.

Also called

SVD Orthogonalization, Symmetric Orthogonalization, 9D Rotation

Related

6D Rotation Representation · Rotation Matrix · Quaternion · Special Orthogonal Group SO(3) · Geodesic Distance on SO(3) · Euler Angles

Sources

An Analysis of SVD for Deep Rotation Estimation (arXiv 2006.14616, NeurIPS 2020)
google-research/special_orthogonalization README
On the Continuity of Rotation Representations in Neural Networks (arXiv 1812.07035)