SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-inference-multiplies-energy

Why can inference consume more total energy than one training run?

A deployed model may be used millions of times.

Start playing free

  1. Repeated inference multiplies small costs ✔
  2. Inference always retrains the model
  3. Training uses no electricity

Why: Training may be a large one-time event, while every prediction consumes resources. At high deployment volume, those repeated costs can dominate the model's operational energy.

Huh, didn't know that: Per-request efficiency matters most when a service runs at very large scale.

The dad joke
One tiny inference invited a million identical friends.
Like this card?
Play the game

References

Answer: Repeated inference multiplies small costs. Training may be a large one-time event, while every prediction consumes resources. At high deployment volume, those repeated costs can dominate the model's operational energy.

  1. Compute and Energy Consumption Trends in Deep Learning Inference arxiv.org/abs/2109.05472
    we focus on inference costs rather than training costs, as the former account for most of the computing effort, solely because of the multiplicative factors
    Checked 2026-10-11.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceIn machine learning, what is inference?Artificial IntelligenceHow is standard dropout commonly handled at inference time?Artificial IntelligenceDoes layer normalization compute differently at training and inference time?

That's stuff everyone should know.