SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-greedy-decoding-repetitive

If a language model always picks the most likely words, what kind of text tends to come out?

Bland and strangely repetitive text.

Start playing free

  1. Bland and strangely repetitive text ✔
  2. The most creative possible text
  3. Perfectly factual text

Why: Researchers found that using likelihood as the decoding goal gives dull text that loops. Human writing is not always the most probable next word, so some randomness actually reads more naturally.

Huh, didn't know that: The same trained model can produce very different quality text just by changing the decoding method.

The dad joke
I always pick the most likely word too. That's why my speeches are so boring, so boring, so boring.
Like this card?
Play the game

References

Answer: Bland and strangely repetitive text. Researchers found that using likelihood as the decoding goal gives dull text that loops. Human writing is not always the most probable next word, so some randomness actually reads more naturally.

  1. Holtzman et al., The Curious Case of Neural Text Degeneration (arXiv 1904.09751) arxiv.org/abs/1904.09751
    using likelihood as a decoding objective leads to text that is bland and strangely repetitive.
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceHow do AI, machine learning and deep learning fit together?Artificial IntelligenceWhat did Alan Turing call his 1950 test of machine intelligence?Artificial IntelligenceTuring predicted that by about 2000, judges would ID the machine at most what percent of the time?

That's stuff everyone should know.