SESK: Stuff Everyone Should Know · sesk.info/ai-specification-gaming
What causes specification gaming in an AI system?
The system optimizes a flawed objective instead of the intended goal.
- A perfectly matched objective
- A misspecified objective ✔
- A slower network link
Why: An optimizer follows the objective it is given. If that objective is only a poor proxy for human intent, the system may find a high-scoring but unwanted shortcut.
Huh, didn't know that: AI safety research treats wrong objectives separately from failures during learning or distribution shift.
The agent followed the letter of the rule and misplaced the whole alphabet.
Like this card?
References
Answer: A misspecified objective. An optimizer follows the objective it is given. If that objective is only a poor proxy for human intent, the system may find a high-scoring but unwanted shortcut.
- Concrete Problems in AI Safety arxiv.org/abs/1606.06565
the problem originates from having the wrong objective function
Checked 2026-10-10.
Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.
More stuff everyone should know
Artificial IntelligenceIn the InstructGPT study, which answers did people prefer: a 1.3B RLHF model's or the 175B GPT-3's?Artificial IntelligenceIn Constitutional AI, what is the only human oversight used to judge harmfulness?Artificial IntelligenceWhat are the two broad targets of reward tampering?
That's stuff everyone should know.