SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-llm-next-token-prediction

At its core, what is a large language model trained to do?

Predict the next token from the tokens that came before.

Start playing free

  1. Predict the next token from earlier tokens ✔
  2. Look up answers in a stored database
  3. Copy whole sentences it memorized

Why: Language models are autoregressive: each new token is predicted from the tokens before it, then added to the text, and the process repeats. A whole answer is built one token at a time.

Huh, didn't know that: Google's crash course calls LLMs autocomplete mechanisms that can predict thousands of tokens.

The dad joke
I asked a language model to finish my sentence. Now it won't stop finishing my sandwiches too.
Like this card?
Play the game

References

Answer: Predict the next token from earlier tokens. Language models are autoregressive: each new token is predicted from the tokens before it, then added to the text, and the process repeats. A whole answer is built one token at a time.

  1. Google Machine Learning Glossary: Generative AI (auto-regressive model) developers.google.com/machine-learning/glossary/generative
    auto-regressive language models predict the next token based on the previously predicted tokens.
    Checked 2026-10-10.
  2. Google ML Crash Course: LLMs and Transformers developers.google.com/machine-learning/crash-course/llm/transformers
    LLMs are essentially autocomplete mechanisms that can automatically predict (complete) thousands of tokens.
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhy does an autoregressive Transformer decoder mask later positions?Artificial IntelligenceWhat does an LLM's context window measure?Artificial IntelligenceWhat two kinds of memory does retrieval-augmented generation (RAG) combine?

That's stuff everyone should know.