SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-encoder-maps-input-sequence

What does a Transformer encoder produce from an input sequence?

It produces a sequence of continuous representations.

Start playing free

  1. One random class
  2. A finished output sentence
  3. Continuous input representations ✔

Why: The encoder transforms every input position into a contextual representation. A decoder can then use that encoded sequence while generating outputs.

Huh, didn't know that: The original encoder layer paired multi-head self-attention with a position-wise feedforward network.

The dad joke
The encoder packed the sequence without losing its train of thought.
Like this card?
Play the game

References

Answer: Continuous input representations. The encoder transforms every input position into a contextual representation. A decoder can then use that encoded sequence while generating outputs.

  1. Attention Is All You Need arxiv.org/html/1706.03762
    to a sequence of continuous representations
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhat lets a recurrent neural network carry information through a sequence?Artificial IntelligenceWhat makes an attention mechanism self-attention?Artificial IntelligenceWhat is a key benefit of using multiple attention heads?

That's stuff everyone should know.