SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-finetuning-unfreezes-weights

What changes when a pretrained model is fine-tuned rather than only used frozen?

Some pretrained weights are unfrozen and updated.

Start playing free

  1. Pretrained weights are updated ✔
  2. All tokens become parameters
  3. The dataset is deleted

Why: Fine-tuning continues training some or all pretrained layers on new data. A low learning rate helps adapt useful representations without overwriting them abruptly.

Huh, didn't know that: A common workflow first trains a new head with the base frozen, then unfreezes selected upper layers.

The dad joke
Fine-tuning adjusted the model without changing the whole station.
Like this card?
Play the game

References

Answer: Pretrained weights are updated. Fine-tuning continues training some or all pretrained layers on new data. A low learning rate helps adapt useful representations without overwriting them abruptly.

  1. TensorFlow: Transfer learning and fine-tuning www.tensorflow.org/guide/keras/transfer_learning
    unfreezing the entire model you obtained above (or part of it), and re-training it on the new data
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhat happens if the learning rate is set too high?Artificial IntelligenceWhich of these is a hyperparameter you set, not something the model learns?Artificial IntelligenceWhat is instruction tuning?

That's stuff everyone should know.