Streaming Agent Responses in LangGraph: Tokens, Events, and Real-Time UI Integration

Scenario context

Stream agents token by token with astream_events; wire to FastAPI SSE for zero-spinner UX.

Your agent takes 25 seconds to respond. Users abandon after 8 seconds. You add streaming — same 25 seconds of total compute, but the first token appears in 0.4 seconds. Abandonment drops to near zero.

This is not a performance optimisation. It is a perception shift. The user's subjective experience of waiting for the…

This walkthrough focuses on The 25-Second Spinner: Why Streaming Is a UX Requirement, Not a Nice-to-Have.

Before making a design choice, clarify:

• Your agent takes 25 seconds to respond.
• Users abandon after 8 seconds.
• You add streaming — same 25 seconds of total compute, but the first token appears in 0.4 seconds.

Concepts in play: llm-agents-and-langgraph, graph-traversal-algorithms, system-design-hld-architectures, event-driven-architecture

Stage 1 — Token Streaming with astream_events: Filtering, Extracting, and Handling Events

• The raw event stream from astream events() is noisy — it includes start/end events for chains, tools, and the LLM itself.
• In practice you filter to the events that matter for your use case.
• Step 1 — Iterate the event stream Step 2 — Filter by event type LangGraph surfaces the following core event types: Step 3 — Extract token content Step 4 — Scope to a specific node In a multi-node graph you usually only…

Observable outcome: Step 1 — Iterate the event stream Step 2 — Filter by event type LangGraph surfaces the following core event types: Step 3 — Extract token content Step 4 — Scope to a specific node In a multi-node graph you usually only…

Stage 2 — Deep Dive: How LangGraph's Streaming Pipeline Works

• The Internals LangGraph builds on LangChain's Runnable abstraction.
• Every node in a LangGraph graph is a RunnableLambda or RunnableSequence under the hood.
• When you call astream events() , LangGraph: 1.
• Creates an AsyncCallbackManager and attaches it to the RunnableConfig passed to each node.

Observable outcome: Creates an AsyncCallbackManager and attaches it to the RunnableConfig passed to each node.

Stage 3 — The Streaming Event Flow: From Node Execution to Browser Pixel

• Each SSE frame is a UTF-8 string prefixed with data: and terminated by \n\n .
• The browser's EventSource API fires an onmessage callback for each frame, which appends the token to the DOM.
• No WebSocket handshake, no polling — a plain HTTP response that never closes until the agent finishes.

Observable outcome: No WebSocket handshake, no polling — a plain HTTP response that never closes until the agent finishes.

Stage 4 — Multi-Agent Streaming: Subgraphs, Parallel Fan-Out, and Per-Agent Filtering

• Single-agent streaming is straightforward.
• Multi-agent graphs introduce three complications: event multiplexing (tokens from multiple agents interleave), subgraph event bubbling (subgraph events propagate to the parent stream), and parallel branch synchronisatio…
• Subgraph event bubbling When you compile a subgraph and embed it as a node in a parent graph, events from inside the subgraph automatically appear in the parent's astream events() stream.
• The langgraph node metadata reflects the subgraph node name, not the parent node name.

Observable outcome: The langgraph node metadata reflects the subgraph node name, not the parent node name.

Stage 5 — Real-World Applications: How ChatGPT, Claude, and Copilot Stream Responses

• ChatGPT uses OpenAI's /v1/chat/completions endpoint with stream: true .
• The response is an SSE stream of JSON delta objects ( {"choices":[{"delta":{"content":"The"}}]} ).
• The frontend concatenates deltas into the visible message.
• Error recovery: if the stream drops mid-response, the UI shows a "regenerate" button — it does not attempt to resume from the partial output.

Observable outcome: Error recovery: if the stream drops mid-response, the UI shows a "regenerate" button — it does not attempt to resume from the partial output.

Stage 6 — Practical Example: FastAPI SSE Endpoint With Streaming Research Assistant

• This example builds a two-node LangGraph agent (search → synthesize) and exposes it as a streaming SSE endpoint, with a JavaScript EventSource client that renders tokens word by word as they arrive.
• The research-assistant scenario was chosen because it has two nodes with meaningfully different event profiles — the search node emits tool call events, and the synthesize node emits token-by-token text chunks — which l…
• As you read through the generate stream async generator, watch the event["event"] filter: every on chat model stream event that passes the check is one token yielded to the browser, and that is the complete streaming pi…
• Graph definition Important : pass streaming=True to ChatOpenAI .

Observable outcome: Graph definition Important : pass streaming=True to ChatOpenAI .

End-to-end outcome

After working through the stages, you should be able to explain:

• streaming=True on the LLM is not optional.
• The single most common reason developers find astream events() doesn't improve TTFT is forgetting to set streaming=True on the ChatOpenAI (or equivalent) constructor.

Alternatives considered

Three Streaming Modes: values, updates, and astream_events Compared

LangGraph exposes streaming through two sync and one async family of methods.

Decision Guide: stream() vs astream() vs astream_events() — When to Use Each

Decision Guide: stream() vs astream() vs astream_events() — When to Use Each

Failure and degraded modes

Trade-offs and Failure Modes: Partial Output, Error Mid-Stream, and Backpressure

• Detection: If your graph raises an exception inside a node after 200 tokens have already been sent, the user has already seen partial text.
• User or operator impact: If your graph raises an exception inside a node after 200 tokens have already been sent, the user has already seen partial text.