SESK: Stuff Everyone Should Know · sesk.info/ai-hallucination-rewarded-guessing
A 2025 paper by OpenAI researchers says hallucinations persist partly because most evaluations reward what?
Guessing instead of saying "I don't know".
- Guessing over admitting uncertainty ✔
- Short answers over long ones
- Polite answers over rude ones
Why: Like a student on a test with no penalty for wrong answers, a model scores higher by guessing than by abstaining. Benchmarks graded right or wrong push models to bluff.
Huh, didn't know that: The authors argue the fix is to change how existing benchmarks score answers, not to add more hallucination tests.
The AI guessed on every question. It learned that from me in high school.
Like this card?
References
Answer: Guessing over admitting uncertainty. Like a student on a test with no penalty for wrong answers, a model scores higher by guessing than by abstaining. Benchmarks graded right or wrong push models to bluff.
- Kalai et al., Why Language Models Hallucinate (arXiv 2509.04664) arxiv.org/abs/2509.04664
language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty
Checked 2026-10-10.
Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.
More stuff everyone should know
That's stuff everyone should know.