SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-mmlu-57-subjects

The MMLU benchmark tests language models across how many subjects, from math to law?

57 tasks, from elementary math to US history and law.

Start playing free

  1. 12
  2. 57 ✔
  3. 500

Why: MMLU (Massive Multitask Language Understanding) is multiple choice across 57 subjects. When it came out in 2020, most models scored near random chance and the largest GPT-3 was only about 20 points better.

Huh, didn't know that: The authors found models frequently do not know when they are wrong.

The dad joke
MMLU has 57 subjects. My report card had 6 and I still needed a tutor.
Like this card?
Play the game

References

Answer: 57. MMLU (Massive Multitask Language Understanding) is multiple choice across 57 subjects. When it came out in 2020, most models scored near random chance and the largest GPT-3 was only about 20 points better.

  1. Hendrycks et al., Measuring Massive Multitask Language Understanding arxiv.org/abs/2009.03300
    The test covers 57 tasks including elementary mathematics, US history, computer science, law, and more.
    Checked 2026-10-11.
  2. Hendrycks et al., Measuring Massive Multitask Language Understanding arxiv.org/abs/2009.03300
    Models also have lopsided performance and frequently do not know when they are wrong.
    Checked 2026-10-11.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhat is benchmark contamination for a language model?Artificial IntelligenceResearchers wrote GSM1k, fresh math problems like GSM8K. What happened to some models' scores?Artificial IntelligenceA 2025 paper by OpenAI researchers says hallucinations persist partly because most evaluations reward what?

That's stuff everyone should know.