SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-time-to-first-token

Which latency measure matters most for how fast a streaming chatbot feels?

Time to first token (TTFT).

Start playing free

  1. Total training time
  2. Time to first token ✔
  3. Number of parameters

Why: TTFT is the wait from sending the prompt to the first token of the reply. With streaming, users start reading as soon as that token appears, so a short TTFT feels fast even if the full answer takes longer.

Huh, didn't know that: Fewer tokens in the prompt and the answer means a faster response.

The dad joke
Time to first token is like a waiter saying I'll be right with you. You still wait, but you feel better.
Like this card?
Play the game

References

Answer: Time to first token. TTFT is the wait from sending the prompt to the first token of the reply. With streaming, users start reading as soon as that token appears, so a short TTFT feels fast even if the full answer takes longer.

  1. Claude API Docs: Reducing latency platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-latency
    This metric measures the time it takes for the model to generate the first token of the response, from when the prompt was sent.
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceSpeculative decoding speeds up a big model by drafting tokens with a smaller one. What happens to the output?Artificial IntelligenceIn machine learning, what is inference?Artificial IntelligenceHow is standard dropout commonly handled at inference time?

That's stuff everyone should know.