Multistep AI Agents: The Power of Planning

Scenario context

A simple ReAct agent reacts one tool call at a time. A multistep agent plans a complete task decomposition upfront, then executes each step sequentially — handling complex goals that require 5-10 interdependent actions without re-prompting the LLM for each step.

A line cook (simple ReAct agent): receives one ticket, cooks one dish, hands it over. Then the next ticket.

A head chef (multistep agent)…

This walkthrough focuses on Line Cook vs. Head Chef.

Before making a design choice, clarify:

• A line cook (simple ReAct agent): receives one ticket, cooks one dish, hands it over.
• A head chef (multistep agent): receives the full dinner party menu, plans the entire 5-course sequence, coordinates prep timing for all dishes, anticipates which items can be done in parallel, and manages the full execu…
• The difference: planning before acting .

Concepts in play: llm-agents-and-langgraph, graph-traversal-algorithms, stack-and-queue-data-structures, api-security-oauth2

Stage 1 — What Makes a Multistep Agent Different

• A simple ReAct agent operates with a single-step planning horizon : it looks at the current state, picks the next action, observes the result, and repeats.
• This works well for short, open-ended tasks where the required steps are unknown ahead of time.
• But when a task involves 5–10 interdependent actions, this approach becomes inefficient — the LLM must rediscover context at every step, decisions made in step 6 can conflict with assumptions from step 2, and there is n…
• A multistep agent solves this by introducing a distinct planning phase before any tool is called.

Observable outcome: A multistep agent solves this by introducing a distinct planning phase before any tool is called.

Stage 2 — The Plan-and-Execute Architecture

• Goal: "Research the top 3 AI papers from last month, summarize each, and draft a blog post." This flow highlights two key design decisions: plan validation happens before any tool call (catching hallucinated tool names…
• The re-plan path preserves completed step results — there is no need to restart from the beginning when a single step fails mid-execution.

flowchart TD
    Goal[User Goal] --> Planner["LLM Planner (produce JSON step list)"]
    Planner --> Validate["Validate Plan (tool names, schema)"]
    Validate -->|Valid| Decompose[Decompose into Parallel + Sequential Steps]
    Validate -->|Invalid| Planner
    Decompose --> Parallel["Run Parallel Steps (independent actions)"]
    Decompose --> Sequential["Run Sequential Steps (dependent actions)"]
    Parallel --> Aggregate[Aggregate Intermediate Results]
    Sequential --> Aggregate
    Aggregate --> Check{All steps complete?}
    Check -->|Yes| Final[Final Result Delivered]
    Check -->|Step Failed| Replan[Re-plan Remaining Steps from Current State]
    Replan --> Decompose

Observable outcome: The re-plan path preserves completed step results — there is no need to restart from the beginning when a single step fails mid-execution.

Stage 3 — Real-World Applications: Real-World Use Cases

• Multistep agents are not theoretical constructs — they power several categories of real production systems today.
• Automated Research Pipelines A research agent might receive a goal like "produce a competitive analysis of the top 5 CRM tools." The plan includes: search for the top 5 tools (step 1), fetch the pricing page for each to…
• No human re-prompts the system between steps — the plan drives the full execution.
• Coding Agents Agents like GitHub Copilot Workspace and Devin use multistep planning to handle tasks like "add OAuth login to this codebase." The plan might include: reading existing auth files, identifying integration p…

sequenceDiagram
    participant U as User
    participant P as LLM Planner
    participant E as Executor
    participant T as Tools

    U->>P: Submit goal
    P->>P: Decompose into step list
    P-->>E: Plan [step1, step2, step3...]
    E->>T: Execute step 1
    T-->>E: Result 1
    E->>P: Observe result 1
    P->>P: Update plan if needed
    E->>T: Execute step 2
    T-->>E: Result 2
    E->>T: Execute step 3
    T-->>E: Result 3
    E-->>U: Aggregated final answer

flowchart TD
    Goal[Receive Goal]
    Decompose[LLM: Decompose into step list]
    Queue["Queue Steps (DAG order)"]
    Execute[Execute Next Step with Tools]
    UpdateState[Update Shared State with Result]
    Done{All Steps Complete?}
    Replan{Step Failed?}
    Final[Return Final Result]
    ReplanStep[Re-plan Remaining Steps from State]

    Goal --> Decompose --> Queue --> Execute --> UpdateState
    UpdateState --> Done
    Done -->|Yes| Final
    Done -->|No| Replan
    Replan -->|Yes| ReplanStep --> Queue
    Replan -->|No| Execute

Observable outcome: Coding Agents Agents like GitHub Copilot Workspace and Devin use multistep planning to handle tasks like "add OAuth login to this codebase." The plan might include: reading existing auth files, identifying integration p…

Stage 4 — Practical: Building a Simple Plan-and-Execute Agent

• The fastest way to understand multistep agents is to trace how the planner and executor components interact in code.
• Below is a minimal Python example using LangChain's PlanAndExecute abstraction with custom tools.
• Step 1 — Define tools and the LLM: Step 2 — Wire up planner and executor: Step 3 — Invoke with a multi-step goal: When you run this with verbose=True , you will see the planner emit a JSON step list, then the executor c…
• This trace is invaluable for debugging — if a step produces an unexpected output, you can inspect exactly where the plan diverged from reality and adjust either the planner prompt or the tool implementation accordingly.

from langchain_openai import ChatOpenAI
from langchain.tools import tool

@tool
def search_web(query: str) -> str:
    """Search the web for the given query."""
    return f"Results for: {query}"

@tool
def summarize_text(text: str) -> str:
    """Summarize the provided text."""
    return f"Summary of: {text[:50]}..."

tools = [search_web, summarize_text]
llm = ChatOpenAI(model="gpt-4o", temperature=0)

from langchain_experimental.plan_and_execute import (
    PlanAndExecute, load_agent_executor, load_chat_planner
)

planner  = load_chat_planner(llm)                          # single LLM call that generates the JSON step list
executor = load_agent_executor(llm, tools, verbose=True)   # runs each step; verbose=True prints tool call traces
agent    = PlanAndExecute(planner=planner, executor=executor, verbose=True)

Observable outcome: This trace is invaluable for debugging — if a step produces an unexpected output, you can inspect exactly where the plan diverged from reality and adjust either the planner prompt or the tool implementation accordingly.

Stage 5 — LangGraph and AutoGen: How OSS Frameworks Implement Multistep Agent Orchestration

• Two open-source frameworks have emerged as the leading ways to build multistep agents with explicit state management, and they take very different architectural approaches.
• LangGraph is a graph-based agent orchestration library from LangChain that models an agent's execution as a directed graph of nodes (LLM calls, tool calls, or logic) and edges (conditional transitions).
• It gives you full control over state, supports cycles (loops), and is designed for durable, resumable workflows — making it the production-ready choice for complex multistep pipelines.
• AutoGen (Microsoft) takes a conversation-centric approach — agents are modeled as participants in a multi-agent dialogue, where each agent has a role (AssistantAgent, UserProxyAgent) and can call tools or other agents a…

from langgraph.graph import StateGraph, END
from typing import TypedDict

# Define the shared state for the agent
class AgentState(TypedDict):
    task: str
    plan: list
    results: list
    current_step: int

# Build the graph
graph = StateGraph(AgentState)

# Add nodes: each is a function that reads and writes state
graph.add_node("planner", run_planner)      # LLM call → produces plan list
graph.add_node("executor", run_executor)    # Tool call → appends to results
graph.add_node("checker", check_completion) # Logic → routes to next step or END

# Wire the execution flow
graph.set_entry_point("planner")
graph.add_edge("planner", "executor")
graph.add_conditional_edges(
    "executor",
    lambda state: "checker" if state["current_step"] < len(state["plan"]) else END,
)
graph.add_edge("checker", "executor")

agent = graph.compile()
result = agent.invoke({"task": "Research top 3 AI papers and draft a summary", "plan": [], "results": [], "current_step": 0})

from autogen import AssistantAgent, UserProxyAgent

planner = AssistantAgent(
    name="Planner",
    system_message="You decompose goals into numbered steps and produce a JSON plan.",
    llm_config={"model": "gpt-4o"},
)

executor = UserProxyAgent(
    name="Executor",
    human_input_mode="NEVER",
    code_execution_config={"work_dir": "./workspace"},
)

# The executor drives the conversation; the planner produces the plan
executor.initiate_chat(
    planner,
    message="Research the top 3 AI papers from last month and summarize each.",
    max_turns=10,
)

Observable outcome: AutoGen (Microsoft) takes a conversation-centric approach — agents are modeled as participants in a multi-agent dialogue, where each agent has a role (AssistantAgent, UserProxyAgent) and can call tools or other agents a…

Stage 6 — Key Lessons

• Plan validation is non-negotiable.
• Always validate the generated plan against the registered tool list and expected argument schema before execution begins.
• Catching hallucinated tool names at plan time costs one validation pass; catching them at execution time costs the tokens from all preceding steps plus a re-plan.
• Re-plan from failure state, not from scratch.

Observable outcome: Re-plan from failure state, not from scratch.

End-to-end outcome

A line cook (simple ReAct agent): receives one ticket, cooks one dish, hands it over.

Alternatives considered

Line Cook vs. Head Chef

A line cook (simple ReAct agent): receives one ticket, cooks one dish, hands it over.

Simple ReAct vs. Plan-and-Execute: Core Difference

Simple ReAct vs. Plan-and-Execute: Core Difference

Decision Guide: ReAct vs. Plan-and-Execute

Decision Guide: ReAct vs. Plan-and-Execute

Failure and degraded modes

Trade-offs & Failure Modes: When to Use Multistep Agents vs Simple Agents

• Detection: Critical failure modes for multistep agents: - Stale plan: If step 3 fails, steps 4-8 may be based on incorrect assumptions.
• User or operator impact: Critical failure modes for multistep agents: - Stale plan: If step 3 fails, steps 4-8 may be based on incorrect assumptions.