Learn · The Inference Pipeline · Measuring Performance
TTFT — Time To First Token
In plain English
TTFT is the waiting game. It's how long you stare at the screen before the AI starts typing! ⏳
What is actually going on
Time To First Token is the latency from when a prompt is submitted to when the first token is generated. It encapsulates network latency (if remote) and the entire prefill compute time.
Why it matters
High TTFT destroys user experience. Users tolerate slow typing, but they hate staring at a blank loading screen.
Another way to think about it
The time between placing your order at a restaurant and getting the breadbasket. 🥖
Check yourself
What does TTFT stand for?
- Time To Final Token
- Time To First Token
- Total Tokens For Training
- Text To Fast Text
Show answer
Time To First Token — It measures the latency before the first output appears.
A massive input prompt will usually increase the TTFT.
- True
- False
Show answer
True — Larger prompts require more computation during the prefill phase, delaying the first token.
Which phase primarily determines TTFT?
- Decode
- Prefill
- Training
- Quantization
Show answer
Prefill — The prefill phase must complete before the first token can be decoded.