SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-alexnet-dropout-regularization

What did AlexNet's authors use to reduce overfitting in its fully connected layers?

A new regularization method, dropout, which randomly switches off units during training.

Start playing free

  1. Dropout regularization ✔
  2. A smaller dataset
  3. A higher learning rate

Why: AlexNet was big enough to overfit. Its authors used a then-new regularization method, dropout, which removes a random selection of units at each training step so the network cannot lean on any one of them.

Huh, didn't know that: Dropout is now a standard tool for training neural networks.

The dad joke
Dropout is like randomly benching players at practice. The whole team gets better.
Like this card?
Play the game

References

Answer: Dropout regularization. AlexNet was big enough to overfit. Its authors used a then-new regularization method, dropout, which removes a random selection of units at each training step so the network cannot lean on any one of them.

  1. NeurIPS 2012: ImageNet Classification with Deep Convolutional Neural Networks (abstract) papers.nips.cc/paper_files/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html
    To reduce overfitting in the globally connected layers we employed a new regularization method that proved to be very effective.
    Checked 2026-10-10.
  2. Google Machine Learning Glossary: dropout regularization developers.google.com/machine-learning/glossary
    Dropout regularization removes a random selection of a fixed number of the units in a network
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhat does dropout randomly remove during a training step?Artificial IntelligenceWhat does L2 regularization penalize?Artificial IntelligenceWhich regularization can push useless features' weights to exactly zero?

That's stuff everyone should know.