SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-instruction-tuning-flan

What is instruction tuning?

Fine-tuning on many tasks written as instructions, so the model follows new ones.

Start playing free

  1. Typing longer instructions into the prompt
  2. Teaching humans to write better prompts
  3. Fine-tuning on many tasks phrased as instructions ✔

Why: Google's 2021 FLAN paper fine-tuned a 137 billion parameter model on over 60 tasks written as natural language instructions. It got much better at tasks it had never seen.

Huh, didn't know that: FLAN beat zero-shot 175B GPT-3 on 20 of the 25 tasks tested.

The dad joke
I tried instruction tuning on my kids. They now follow instructions on 0 of 60 tasks.
Like this card?
Play the game

References

Answer: Fine-tuning on many tasks phrased as instructions. Google's 2021 FLAN paper fine-tuned a 137 billion parameter model on over 60 tasks written as natural language instructions. It got much better at tasks it had never seen.

  1. Wei et al., Finetuned Language Models Are Zero-Shot Learners (arXiv 2109.01652) arxiv.org/abs/2109.01652
    instruction tuning -- finetuning language models on a collection of tasks described via instructions -- substantially improves zero-shot performance
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhat changes when a pretrained model is fine-tuned rather than only used frozen?Artificial IntelligenceQLoRA made it possible to fine-tune a 65 billion parameter model on what hardware?Artificial IntelligenceHow does LoRA fine-tune a large model so cheaply?

That's stuff everyone should know.