smart_toy Artificial Intelligence › Article

Response streaming: showing AI output while it is being written

Response streaming shows an AI model's text as it is generated, instead of waiting for the end. How it works with SSE and how to design the interface for it.

info Quick Definition

Response streaming is the way to deliver an AI model’s output in chunks, as it is generated, so the person can start reading before it ends. It is usually sent over SSE (server-sent events). The model does not get faster: what changes is when the text appears.

What is it and how does it work?

A language model generates the response token by token (if tokens are new to you, see tokens). Without streaming, the server waits until it has the full text and sends it in one piece. With streaming, it sends each fragment as soon as it has it.

SSE is a one-way web mechanism: the server answers with the text/event-stream content type and the client receives events while the connection stays open. You cannot send events from the client to the server over that same channel (MDN).

In the Claude API you turn it on with "stream": true, and the flow always follows the same order:

Event	What it carries	What your interface can do
`message_start`	A message with empty `content`	Show the initial indicator
`content_block_start`	Opens a content block (text, tool use, thinking)	Create the block’s container
`content_block_delta`	A new fragment; in text blocks, a `text_delta`	Append the text to what is already shown
`content_block_stop`	Closes the block	Apply the block’s final formatting
`message_delta`	Changes to the message, such as `stop_reason`; token usage is cumulative	Store the stop reason
`message_stop`	End of the stream	Remove the stop button
`ping`	Keeps the connection alive; can arrive any number of times	Ignore it
`error`	For example `overloaded_error`	Mark the response as incomplete

Tool-use and thinking blocks also arrive as deltas (input_json_delta and thinking_delta); decide whether to show them. New event types may appear, so your code should ignore the ones it does not know.

You do not have to read this protocol by hand: the official Python and TypeScript SDKs offer helpers that yield only the text (text_stream in Python, the text event in TypeScript). If you do not need to show anything half-finished, the SDK can use streaming internally and hand you the complete message, useful with large max_tokens values to avoid HTTP timeouts.

Why does it matter to a designer?

Because it changes what the person perceives as waiting. Nielsen Norman Group recalls the three classic limits: 0.1 s feels instantaneous, 1 s is the most that keeps a person’s flow of thought uninterrupted, and 10 s is the limit for keeping their attention on the task. With an eight-second answer that appears all at once, the person stares at a still screen; with text from the first second, they are already reading.

Be careful: streaming does not speed up the response. It improves the time to the first useful content: if the model takes six seconds to generate everything, those six seconds are still there; what changes is what the person sees during them.

Three design decisions depend on this:

What to show before any text arrives. A brief indicator (a pulsing dot, “Thinking”) that disappears when the text starts. To choose between a spinner, a skeleton or a bar, see loading states in AI interfaces.
How to show partial text. Text that jumps with every token is tiring. Batch fragments and keep what is already written stable.
What happens if it stops. If the connection drops or there is an error, the person should not lose what they already read without knowing it.

When to use streaming and when not to?

Situation	Stream?	Why
Long summary, draft email, multi-paragraph explanation	Yes	People read while it arrives and the partial text is already useful
One-word or one-number answer	No	It is understood at a glance; streaming only adds flicker
Action confirmed at the end (creating a record)	No	A partial result is no use for deciding
Structured output (JSON that other code will read)	No	Half a JSON cannot be parsed until it closes
Multi-step agent	Yes, but steps	Showing what it is doing matters more than the text

A practical rule: if the person needs the complete result to act, show a clear loading state and deliver the whole result. If they can start reading, use streaming.

For agents with tools, the agent loop explains why the interface should show steps. To measure the wait, see latency and tokens per second.

How do you implement it well in the interface?

Batch the updates. Repainting on every token is expensive. Accumulate fragments and update once per browser animation frame.
Do not force the scroll. If the person scrolled up to reread, do not bring them back down with every fragment. Follow the end only if they were already there.
Render formatting carefully. A code block or table that is half-written can break the layout. Show the block in progress as plain text until it closes.
Look after screen reader users. MDN recommends aria-live="polite" for important changes that are not so rapid as to be annoying, and asks you to create the region before updating its content. Announce the start and the end of the response, not every token.
Offer a way out. A button to stop generation, visible from the first fragment. The full pattern is in interrupting and steering an agent.
Check why it ended. A stream can close without an error and still be truncated. The stop_reason in message_delta tells you: end_turn is a normal ending; max_tokens, for example, means the limit cut the response short.

What to do when the stream is cut?

The API can send errors inside the stream, for example overloaded_error during periods of high usage, which arrive as an error event and not as an HTTP status code. Treat this as a normal case.

The Claude documentation describes a three-step recovery strategy: save what was received, build a new request that includes that partial text, and keep streaming from there. On Claude 4.6 and later, the partial text goes in a user message with an instruction to continue. Limit: tool-use and thinking blocks cannot be recovered partially; you can only resume from the most recent text block.

In design terms: keep the partial text, mark it as incomplete and offer “Retry” or “Continue”.

Example

Illustrative example: an internal tool summarizes 1,500-word sales reports. The first version shows a spinner for about seven seconds and then the whole summary. Some people think the app has frozen and reload, which doubles the requests.

The team switches to streaming. The first headings appear almost immediately and the text grows underneath. They add a fixed notice (“Generating summary”) and a stop button. The total duration does not change, but there is something to read from the first second. If the stream fails, the received text stays on screen with an “Incomplete response” notice and a retry button.

Common mistakes

Streaming very short answers, where partial text adds nothing.
Updating the DOM on every token without batching, which makes the screen slow and flickery.
Not handling an error in the middle of the stream, and leaving a cut-off answer with no notice.
Assuming a response is complete just because the stream ended, without checking the stop_reason.
Removing the option to stop generation, or announcing every fragment to screen readers.

Sources

Streaming messages (Claude documentation): stream events, errors in the stream, SDK helpers and recovery from interrupted streams.
Using server-sent events (MDN): what SSE is and the text/event-stream type.
Response Times: The 3 Important Limits (Nielsen Norman Group): the 0.1 s, 1 s and 10 s limits.
ARIA live regions (MDN): aria-live="polite" and regions created before they are updated.

Frequently asked questions

What is response streaming in AI?

It is the technique of sending the model’s response in chunks as it is generated, so it can be shown on screen without waiting for the end. The model generates the same way; what changes is when the person sees the text.

Does streaming make the AI respond faster?

No. The total time to generate the response stays the same. What changes is the time until the first visible content, which is what people perceive as waiting.

What happens if there is an error in the middle of a stream?

The response has already started arriving, so the interface must handle a partial error: keep what was received, mark it as incomplete and offer to retry. The Claude API can send an error event inside the same stream.

Should I use streaming across the whole AI interface?

Use it for long responses that people read while they arrive. For short results or actions that are confirmed at once, a loading indicator and the complete result are usually better.