SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-transformer-removes-recurrence

Which two sequence mechanisms did the original Transformer dispense with?

It removed recurrence and convolution from its core architecture.

Start playing free

  1. Weights and biases
  2. Recurrence and convolution ✔
  3. Tokens and embeddings

Why: The original Transformer relied on attention plus feedforward layers. It did not require recurrent or convolutional layers to model its sequences.

Huh, didn't know that: Removing recurrence made training computations across positions much more parallelizable.

The dad joke
The Transformer dropped recurrence because it disliked repeating itself.
Like this card?
Play the game

References

Answer: Recurrence and convolution. The original Transformer relied on attention plus feedforward layers. It did not require recurrent or convolutional layers to model its sequences.

  1. Attention Is All You Need arxiv.org/html/1706.03762
    based solely on attention mechanisms, dispensing with recurrence and convolutions entirely
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhich 2017 paper introduced the Transformer architecture?Artificial IntelligenceWhere is a neural network's hidden layer located?Artificial IntelligenceAfter query-key scores become weights, what does attention combine?

That's stuff everyone should know.