SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-dropout-inference-network

How is standard dropout commonly handled at inference time?

Use one unthinned network with adjusted scaling.

Start playing free

  1. Delete the trained model
  2. Drop every unit
  3. Use the full scaled network ✔

Why: Training samples many thinned subnetworks. At test time, a single full network with appropriate scaling approximates averaging their predictions.

Huh, didn't know that: This avoids explicitly running an exponential number of dropout masks at inference.

The dad joke
At test time, every dropout neuron showed up for roll call.
Like this card?
Play the game

References

Answer: Use the full scaled network. Training samples many thinned subnetworks. At test time, a single full network with appropriate scaling approximates averaging their predictions.

  1. Dropout: A Simple Way to Prevent Neural Networks from Overfitting www.jmlr.org/papers/v15/srivastava14a.html
    At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceIn machine learning, what is inference?Artificial IntelligenceWhat did AlexNet's authors use to reduce overfitting in its fully connected layers?Artificial IntelligenceWhat does dropout randomly remove during a training step?

That's stuff everyone should know.