Server-Sent Events (SSE) is a lightweight, unidirectional streaming protocol that enables servers to push data to clients over standard HTTP connections. In the context of LLM applications, SSE is the dominant protocol for delivering token-by-token responses from language models to end users, creating the familiar “typing” effect seen in ChatGPT, Claude, and virtually every modern AI chat interface.
SSE has become the de facto standard for LLM streaming because it perfectly matches the communication pattern: the server generates tokens sequentially and pushes them to the client as they become available. OpenAI, Anthropic, Google, and most major LLM API providers use SSE natively for their streaming endpoints. Unlike WebSockets, SSE requires no protocol upgrade, works seamlessly with existing HTTP infrastructure (load balancers, proxies, CDNs), and provides built-in reconnection with event ID tracking.
For AI engineers, understanding SSE is essential because it directly impacts perceived latency, user experience, and system architecture. Instead of waiting 5-30 seconds for a complete LLM response, SSE delivers the first token within milliseconds of generation, giving users immediate feedback and dramatically improving time-to-first-byte (TTFB) metrics.
SSE operates over standard HTTP. The client makes a regular HTTP request, and the server responds with the MIME type text/event-stream. Instead of closing the connection after sending a response, the server keeps it open and continues sending data as events become available. Each event is a plain-text block separated by two newline characters (\n\n).
SSE messages are UTF-8 encoded plain text. Each message consists of one or more fields (key-value pairs separated by :), with each field on its own line. Messages are delimited by a blank line (\n\n). The four standard fields are data, event, id, and retry.
The basic format of an SSE message looks like this:
data: {"token": "Hello"}
data: {"token": " world"}
data: {"token": "!"}
data: [DONE]Each line beginning with data: contains the payload. The blank line after each data line signals the end of that event. A colon as the first character creates a comment (often used as a heartbeat to keep connections alive). The id: field sets an event identifier that enables resumption after reconnection. The retry: field tells the client how long to wait before attempting reconnection, in milliseconds.
Browsers provide the EventSource API specifically for consuming SSE streams. It handles connection management, automatic reconnection, and event parsing natively:
const source = new EventSource('/api/chat/stream');
source.onmessage = (event) => {
const data = JSON.parse(event.data);
if (data.token) {
appendToUI(data.token);
}
};
source.onerror = (error) => {
console.error('SSE connection error:', error);
// EventSource automatically attempts reconnection
};
// To stop the stream
source.close();For LLM streaming specifically, many implementations use the fetch API instead of EventSource because EventSource only supports GET requests, while LLM APIs typically require POST requests with a JSON body containing the prompt. The fetch-based approach reads the response body as a stream:
const response = await fetch('/api/chat', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ prompt: userMessage, stream: true })
});
const reader = response.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value, { stream: true });
// Parse SSE-formatted lines from the chunk
const lines = chunk.split('\n');
for (const line of lines) {
if (line.startsWith('data: ')) {
const payload = line.slice(6);
if (payload === '[DONE]') return;
const data = JSON.parse(payload);
appendToUI(data.choices[0].delta.content);
}
}
}On the server, implementing SSE requires setting the correct headers and writing events to the response stream as tokens are generated. Here is a Node.js/Express example that proxies an LLM API stream:
app.post('/api/chat/stream', async (req, res) => {
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
res.setHeader('X-Accel-Buffering', 'no'); // Disable nginx buffering
const llmStream = await openai.chat.completions.create({
model: 'gpt-4',
messages: req.body.messages,
stream: true,
});
for await (const chunk of llmStream) {
const content = chunk.choices[0]?.delta?.content;
if (content) {
res.write(`data: ${JSON.stringify({ content })}\n\n`);
}
}
res.write('data: [DONE]\n\n');
res.end();
});Reverse proxies like Nginx buffer responses by default, which defeats streaming. You must set X-Accel-Buffering: no (Nginx), proxy_buffering off in Nginx config, or equivalent settings for your proxy layer. Without this, tokens accumulate in the proxy buffer and arrive in bursts instead of streaming smoothly.
SSE has built-in reconnection semantics. When a connection drops, the EventSource API automatically reconnects after the interval specified by the retry: field (default is typically 3 seconds). On reconnection, the browser sends a Last-Event-ID header containing the id of the last received event, allowing the server to resume the stream from where it left off.
For LLM streaming, this is valuable for long-running generations. By assigning sequential IDs to token events, the server can buffer recently sent tokens and replay missed ones on reconnection, preventing partial responses from being lost during network hiccups.
Major LLM providers follow a consistent pattern. When the client sets stream: true in the API request, the response arrives as an SSE stream instead of a single JSON object. Each event contains a delta — the incremental content generated since the last event. OpenAI’s format sends choices[0].delta.content per chunk, while Anthropic uses content_block_delta events with a delta.text field. Both terminate with a [DONE] sentinel or a message_stop event.
The most common use of SSE in AI applications is the token-by-token chat experience. As the LLM generates each token, it is immediately pushed to the client and appended to the UI. This creates a responsive feel even when the full response takes 10+ seconds to generate. Time-to-first-token (TTFT) is typically 200-500ms, meaning users see output begin almost instantly.
SSE is also used to stream structured data like JSON responses progressively. Tools like streaming JSON parsers can begin processing partial JSON as it arrives, enabling features like progressive form filling, real-time data extraction, and incremental chart rendering as the LLM generates structured output.
In agentic applications, SSE delivers status updates during multi-step workflows. When an agent searches the web, calls tools, and synthesizes results, each step can emit SSE events with different event: types — event: thinking, event: tool_call, event: tool_result, event: content — allowing the client to render appropriate UI for each phase.
IDEs and coding assistants use SSE to stream code completions and explanations. As the model generates code, syntax highlighting can be applied progressively, and the editor can begin parsing the incoming code for structure before the full response is complete.
text/event-stream MIME type with a simple plain-text format: data: lines separated by \n\n.fetch API with ReadableStream instead of EventSource (which only supports GET).