SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-attention-weighted-values

After query-key scores become weights, what does attention combine?

It forms a weighted sum of value vectors.

Start playing free

  1. The value vectors ✔
  2. Only the query vectors
  3. The token spellings

Why: A query is compared with keys to determine relevance weights. Those weights mix the corresponding values into the attention output.

Huh, didn't know that: Keys decide how much to attend, while values carry the information that gets combined.

The dad joke
Attention valued the answer after checking the keys.
Like this card?
Play the game

References

Answer: The value vectors. A query is compared with keys to determine relevance weights. Those weights mix the corresponding values into the attention output.

  1. Attention Is All You Need arxiv.org/html/1706.03762
    The output is computed as a weighted sum of the values
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhat calculation does a basic artificial neuron perform before activation?Artificial IntelligenceWhich 2017 paper introduced the Transformer architecture?Artificial IntelligenceWhich two sequence mechanisms did the original Transformer dispense with?

That's stuff everyone should know.