LangChain Memory: Conversation History and Summarization

Scenario context

LLMs are stateless — every API call starts fresh. LangChain memory classes (Buffer, Window, Summary, SummaryBuffer) explicitly inject history into each call, and RunnableWithMessageHistory is the modern LCEL replacement for the legacy ConversationChain.

You have built a customer support chatbot. The first message goes perfectly:

User: Hi, my account number is ACC-8837 and I cannot log in. **Bot…

This walkthrough focuses on The Amnesia Tax: Every Conversation Starts from Zero.

Before making a design choice, clarify:

• You have built a customer support chatbot.
• The first message goes perfectly: User: Hi, my account number is ACC-8837 and I cannot log in.
• Account ACC-8837 is locked after three failed attempts.

Concepts in play: llm-agents-and-langgraph, python-concurrency-nogil, caching, consistency-replication

Stage 1 — What the LLM API Actually Receives on Every Call

• Before choosing a memory strategy, it helps to see exactly what the API receives.
• Every call to llm.invoke(messages) sends a structured list of messages.
• Once the response is returned, the API discards everything.
• There is no session, no thread ID, no continuation token.

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

llm = ChatOpenAI(model="gpt-4o-mini")
response = llm.invoke([HumanMessage(content="My account is ACC-8837.")])
# The model processes this, returns a response, and forgets everything.

from langchain_core.messages import HumanMessage, AIMessage

messages = [
    HumanMessage(content="My account number is ACC-8837."),
    AIMessage(content="Got it. Account ACC-8837 is locked."),
    HumanMessage(content="Can you resend the reset link?"),
]
response = llm.invoke(messages)
# The model can now reference ACC-8837 because the history is in THIS call.

Observable outcome: There is no session, no thread ID, no continuation token.

Stage 2 — Memory Flow: From User Input to LLM Response

• Every memory-backed chain follows a consistent read-before-write cycle.
• The memory object wraps each LLM call like a decorator: it injects history before the model sees the prompt, and it persists the exchange after the response arrives.
• Figure: The memory read-before-write cycle on every conversation turn.
• The LLM itself remains stateless; the history management lives entirely in the LangChain chain layer above it.

graph TD
    A[User sends a message] --> B[Chain receives input]
    B --> C[Memory.load_memory_variables]
    C --> D[Retrieve stored history or running summary]
    D --> E[Assemble full prompt: history + new message]
    E --> F[LLM API call with full context]
    F --> G[LLM generates response]
    G --> H[Memory.save_context]
    H --> I[Write new input + output to memory store]
    I --> J[Return response to user]
    J --> A

Observable outcome: The LLM itself remains stateless; the history management lives entirely in the LangChain chain layer above it.

Stage 3 — Deep Dive: Inside LangChain's Memory Architecture

• Under the Internals: How Memory Classes Store and Retrieve State All four LangChain memory classes share the same two-method interface: load memory variables(inputs) and save context(inputs, outputs) .
• The ConversationChain (and RunnableWithMessageHistory ) calls them in sequence around every LLM invocation.
• Internally, every class wraps a ChatMessageHistory object — a simple ordered list of BaseMessage objects ( HumanMessage , AIMessage , SystemMessage ).
• The classes differ only in what they do before handing that list to the prompt: - BufferMemory returns the full list unchanged.

Progressively summarize the lines of conversation provided, adding onto
the previous summary and returning a new summary.

SUMMARY:
{summary}

NEW LINES OF CONVERSATION:
{new_lines}

NEW SUMMARY:

Observable outcome: The classes differ only in what they do before handing that list to the prompt: - BufferMemory returns the full list unchanged.

Stage 4 — Real-World Application: A Multi-Turn Code Review Assistant

• A Code Review Assistant is an ideal showcase for memory: the user submits code in the first turn and then asks follow-up questions that only make sense in the context of what was previously reviewed.
• Without memory, every follow-up requires the user to paste the code again.
• The following five-turn session uses ConversationSummaryBufferMemory .
• The first three turns fit comfortably within the 800-token verbatim buffer; by turn four, older context is summarized, yet the model retains the critical facts.

from langchain.memory import ConversationSummaryBufferMemory
from langchain.chains import ConversationChain
from langchain_openai import ChatOpenAI
from langchain_core.prompts import PromptTemplate

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=800)

template = """You are a senior code reviewer. Be concise and specific.

{history}
Human: {input}
Assistant:"""

prompt = PromptTemplate(input_variables=["history", "input"], template=template)
chain = ConversationChain(llm=llm, memory=memory, prompt=prompt)

# Turn 1 — user submits code for review
response = chain.predict(input="""
Review this Python function:
def fetch_user(user_id):
    conn = db.connect()
    result = conn.execute(f"SELECT * FROM users WHERE id = {user_id}")
    return result.fetchone()
""")
# → Critical issue: SQL injection vulnerability on the f-string query.
#   Use a parameterized query: conn.execute("SELECT * FROM users WHERE id = ?", (user_id,))
#   Also: the connection is never closed — wrap in a context manager.

# Turn 2 — follow-up: ask for corrected code
response = chain.predict(input="Show me the corrected version with both fixes.")
# → def fetch_user(user_id):
#       with db.connect() as conn:
#           result = conn.execute("SELECT * FROM users WHERE id = ?", (user_id,))
#           return result.fetchone()

# Turn 3 — ask about a related pattern
response = chain.predict(input="Is SQLAlchemy connection pooling better here?")
# → Yes — use create_engine() with pool_size and pool_recycle for production.
#   The context manager approach you have now is fine for scripts.

# Turn 4 — reference the corrected code from turn 2
response = chain.predict(input="Can the function you showed handle None as user_id?")
# → No. Add a guard: if user_id is None: raise ValueError("user_id must not be None")
#   The parameterized query would pass NULL to the database, returning unexpected results.

# Turn 5 — synthesize all findings
response = chain.predict(input="Summarize every issue found and the final recommended code.")
# → Three issues addressed:
#   1. SQL injection — fixed with parameterized query
#   2. Unclosed connection — fixed with context manager
#   3. None input — add a ValueError guard before the query
print(response)

Observable outcome: The first three turns fit comfortably within the 800-token verbatim buffer; by turn four, older context is summarized, yet the model retains the critical facts.

Stage 5 — Community Memory Backends: Redis, MongoDB, and DynamoDB

• The in-memory ChatMessageHistory is appropriate for single-process development and testing.
• In production, where multiple workers serve the same user concurrently or sessions must survive process restarts, you need a persistent backend.
• The langchain-community package ships ready-made history implementations for the most common stores.
• They all implement BaseChatMessageHistory and can be dropped in as the chat memory argument of any LangChain memory class.

# pip install langchain-community redis
from langchain_community.chat_message_histories import RedisChatMessageHistory
from langchain.memory import ConversationSummaryBufferMemory
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")

history = RedisChatMessageHistory(
    session_id="user-ACC-8837",
    url="redis://localhost:6379",
    ttl=3600,  # Expire inactive sessions after 1 hour
)
memory = ConversationSummaryBufferMemory(
    llm=llm,
    chat_memory=history,
    max_token_limit=1000,
    return_messages=True,
)

# MongoDB backend
from langchain_community.chat_message_histories import MongoDBChatMessageHistory

history = MongoDBChatMessageHistory(
    session_id="user-ACC-8837",
    connection_string="mongodb://localhost:27017",
    database_name="chat_sessions",
    collection_name="message_history",
)

Observable outcome: They all implement BaseChatMessageHistory and can be dropped in as the chat memory argument of any LangChain memory class.

Stage 6 — The Modern LCEL Memory Pattern: RunnableWithMessageHistory

• ConversationChain is the legacy API.
• For any new project, use RunnableWithMessageHistory — LangChain's LCEL-native memory wrapper.
• It is composable with the pipe operator, supports async streaming natively, and cleanly separates session management (your responsibility) from chain logic (LangChain's responsibility).
• To persist sessions across restarts, replace ChatMessageHistory() with RedisChatMessageHistory(session id=session id, url=REDIS URL) inside get session history .

from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful coding assistant."),
    MessagesPlaceholder(variable_name="history"),
    ("human", "{input}"),
])

chain = prompt | llm  # Clean LCEL pipe — no ConversationChain wrapper

# In-memory store: swap for RedisChatMessageHistory in production
store: dict[str, BaseChatMessageHistory] = {}

def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]

chain_with_history = RunnableWithMessageHistory(
    chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="history",
)

# Pass session identity via config on every call
config = {"configurable": {"session_id": "user-ACC-8837"}}

r1 = chain_with_history.invoke({"input": "My account is ACC-8837."}, config=config)
r2 = chain_with_history.invoke({"input": "What account did I just mention?"}, config=config)
print(r2.content)
# → "You mentioned account ACC-8837."

Observable outcome: To persist sessions across restarts, replace ChatMessageHistory() with RedisChatMessageHistory(session id=session id, url=REDIS URL) inside get session history .

End-to-end outcome

After working through the stages, you should be able to explain:

• ConversationBufferMemory — Append Everything The simplest approach: store every message in a list and send the complete list on every call.
• The memory object implements two key methods on every turn: load memory variables({}) reads the history before the LLM call, and save context({"input": ...}, {"output": ...}) writes the new exchange back after.
• Set a token budget before your first deploy, not after your first incident.
• ConversationBufferMemory is tempting because it requires zero configuration.

Alternatives considered

Decision Guide: Choosing the Right Memory Strategy

Decision Guide: Choosing the Right Memory Strategy

Failure and degraded modes

Trade-offs & Failure Modes in Conversational Memory

• Detection: The most expensive lesson teams learn is deploying ConversationBufferMemory to production and discovering that power users with 150-turn sessions start hitting context window limits at 2 AM.
• User or operator impact: The most expensive lesson teams learn is deploying ConversationBufferMemory to production and discovering that power users with 150-turn sessions start hitting context window limits at 2 AM.