CUDA

NVIDIA's general-purpose computing platform for GPUs, which is what lets deep learning run on a graphics card.

CUDA is NVIDIA's parallel computing platform and programming model, including a compiler, runtime libraries, and math libraries, that lets developers call an NVIDIA GPU directly for general-purpose computation from languages such as C/C++. Deep learning's huge number of matrix operations get their speed from GPU parallelism, and frameworks such as PyTorch call into CUDA and its companion libraries (such as cuDNN, which implements neural-network operators) whenever they run on an NVIDIA card. Newcomers usually meet it first while setting up their environment: the GPU build of PyTorch installed via pip bundles its own matching CUDA runtime libraries, so it works as long as the graphics driver is new enough to support that CUDA version — too old a driver either errors out or falls back to CPU only. Only when compiling a custom CUDA extension yourself does the locally installed CUDA Toolkit version need to match PyTorch's. GPU parallelism in simulation (such as Isaac Lab) and deployment acceleration (such as TensorRT) are both built on top of CUDA too.

ExampleRunning torch.cuda.is_available() in Python returns True only when PyTorch can actually use the GPU.

Also called

Compute Unified Device Architecture, CUDA Toolkit

Related

CUDA Deep Neural Network Library (cuDNN) · PyTorch · NVIDIA TensorRT · GPU Memory (VRAM) · NVIDIA

Sources

NVIDIA CUDA Toolkit
CUDA Programming Guide（NVIDIA 官方文档） (Chinese)
PyTorch Forums: Cuda versioning and pytorch compatibility（PyTorch 二进制包自带 CUDA 运行库，只需驱动支持） (Chinese)