AI comparison
Streaming vs not streaming
Stream when a person is watching the reply appear, so they can start reading at once; wait for the whole reply when code must parse or check it first.
| Aspect | Streaming | Whole reply |
|---|---|---|
| First words appear | Within a moment | When the whole reply is done |
| Total time | About the same | About the same |
| Handling in code | Events to join and partial text | One response to parse |
| Checking before showing | Hard: the text is already shown | Easy |
Choose Streaming when
- A person reads the reply as it is written
- Replies are long
- You want the feature to feel responsive
Choose Whole reply when
- The reply is JSON for your code
- You must check or filter it before anyone sees it
- Replies are short
Learn it hands-on
These free lessons teach it with commands to run in a lab and checked tasks. Read them whole with no account.
Related terms
- Streaming
- Streaming sends a model's reply piece by piece as it is generated, so a user sees the first words in a moment instead of waiting for the whole answer.
- Latency
- Latency is how long a user waits for a response; for a language model it is usually split into time to first token and total time.
- Structured output
- Structured output is asking a language model to reply in a fixed machine-readable shape, such as JSON with named fields, instead of free text.