SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-embedding-token-vector

What does a learned embedding assign to each input token?

It assigns a vector of numbers.

Start playing free

  1. A complete webpage
  2. A numeric vector ✔
  3. A fixed grammar rule

Why: An embedding converts a discrete token ID into a continuous vector. Training can arrange those vectors so useful relationships are reflected geometrically.

Huh, didn't know that: The original Transformer used token embeddings with the same dimension as its model states.

The dad joke
The token found direction after joining a vector space.
Like this card?
Play the game

References

Answer: A numeric vector. An embedding converts a discrete token ID into a continuous vector. Training can arrange those vectors so useful relationships are reflected geometrically.

  1. Attention Is All You Need arxiv.org/html/1706.03762
    we use learned embeddings to convert the input tokens and output tokens to vectors
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceIn word2vec, what is vector(King) minus vector(Man) plus vector(Woman) closest to?Artificial IntelligenceWhat property does cosine similarity ignore when comparing two embeddings?Artificial IntelligenceWhy does a Transformer add positional information to token embeddings?

That's stuff everyone should know.