SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-dropout-randomly-removes-units

What does dropout randomly remove during a training step?

It removes selected units and their connections.

Start playing free

  1. All training labels
  2. Random units and connections ✔
  3. The optimizer permanently

Why: A new random subset of activations is suppressed during training. This discourages the network from depending too heavily on particular cooperating units.

Huh, didn't know that: The original dropout paper describes each training pass as sampling a thinned network.

The dad joke
Dropout skipped class, but only as regularization.
Like this card?
Play the game

References

Answer: Random units and connections. A new random subset of activations is suppressed during training. This discourages the network from depending too heavily on particular cooperating units.

  1. Dropout: A Simple Way to Prevent Neural Networks from Overfitting www.jmlr.org/papers/v15/srivastava14a.html
    The key idea is to randomly drop units (along with their connections) from the neural network during training.
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhat did AlexNet's authors use to reduce overfitting in its fully connected layers?Artificial IntelligenceA model scores great on its training data but badly on new data. What is that called?Artificial IntelligenceTraining error is far lower than test error. Which problem does that point to?

That's stuff everyone should know.