Multilayer Perceptron

The most basic neural network: stacked fully-connected layers with nonlinear activations in between.

A multilayer perceptron is the most basic feedforward neural network: an input layer, several hidden layers, and an output layer, with every neuron in one layer connected to every neuron in the next. Each layer applies a linear transformation followed by a nonlinear activation function (such as ReLU). It traces back to Rosenblatt's 1958 perceptron; only after the backpropagation algorithm spread in 1986 could multi-layer versions be trained effectively. A single-layer perceptron can only separate linearly separable data; adding hidden layers and nonlinearity lets it fit far more complex functions. MLPs are everywhere in today's models: the feedforward block inside each Transformer layer is itself a two-layer MLP; vision-language models such as LLaVA-1.5 use an MLP to project visual features into the language model's input space; and in robotics, reinforcement-learning locomotion policies with modest input dimensions, action heads, and state encoders are often just an MLP.

ExampleThe default policy network in the legged_gym reinforcement-learning framework for legged robots is an MLP with three hidden layers of width 512, 256, and 128, using ELU activations, taking in proprioception and velocity commands, and outputting target joint positions.

Also called

MLP, Feedforward Neural Network

Related

Neural Network · Activation Function · Transformer · Projector / Connector · Action Head · Backpropagation

Sources

Wikipedia: Multilayer perceptron
legged_gym: legged_robot_config.py
Improved Baselines with Visual Instruction Tuning (LLaVA-1.5, arXiv:2310.03744)