Advanced AI: Agents, RAG, and the Future of Intelligence

Scenario context

Large Language Models are brilliant "brains in a jar." Retrieval-Augmented Generation (RAG) hands them a constantly refreshed memory, while AI Agents give them tools to act in the world. Combined, they turn static knowledge into dynamic, goal-directed intelligence — the next frontier of practical AI.

Klarna's AI support bot answered 2.3 million customer queries in its first month — but hallucinated refund…

This walkthrough focuses on Decision Guide: From Smart Chatbots to Systems That Actually Act.

Before making a design choice, clarify:

• Klarna's AI support bot answered 2.3 million customer queries in its first month — but hallucinated refund policies roughly 15% of the time, because it had no access to Klarna's current policy documents.
• The model was brilliant at generating fluent, confident text; it simply didn't know what the actual rules were on any given day.
• RAG (Retrieval-Augmented Generation) fixes this by fetching relevant documents from a live knowledge base before the LLM generates its response — so it answers with actual policy, not a statistical guess.

Concepts in play: vector-databases-rag, vector-embeddings-and-similarity, database-indexing, event-driven-architecture

Stage 1 — The Two Pillars: RAG and Agents Explained

• Retrieval-Augmented Generation (RAG) Problem: LLMs are trained on a static snapshot of the internet.
• Anything beyond that cutoff — stock prices, internal docs, breaking news — is invisible to them.
• Idea: Before generating a response, a retriever fetches the most relevant passages from an external corpus (vector store, database, search engine).
• The LLM then conditions on those passages, effectively reading fresh material.

Observable outcome: The LLM then conditions on those passages, effectively reading fresh material.

Stage 2 — What You Need to Know First

• RAG (Retrieval-Augmented Generation) is a technique that grounds LLM responses in external knowledge.
• Instead of relying solely on weights baked in during training, a RAG pipeline fetches relevant documents at query time and injects them into the model's context.
• This eliminates hallucination on factual questions and keeps answers current without retraining.
• Agents are LLM-powered systems that can plan, make decisions, and use tools to accomplish multi-step goals.

Observable outcome: Agents are LLM-powered systems that can plan, make decisions, and use tools to accomplish multi-step goals.

Stage 3 — How RAG and Agents Work Under the Hood

• Retriever — a bi-encoder (e.g., Sentence-Transformers) mapping queries and documents into the same embedding space, queried via FAISS or HNSW for sub-millisecond ANN lookup.
• LLM Engine — decoder-only transformer receiving a concatenated [SYSTEM][CONTEXT][USER] prompt where context is the retrieved passages.
• Agent Controller — an orchestrator (LLM or rule-based) that decides which registered tool to fire.
• Tool outputs are fed back into the next generation step.

graph LR
    A[User Input] --> B{RAG needed?}
    B -- Yes --> C[Retriever  Top-k Passages]
    C --> D[LLM contextual generation]
    B -- No --> D
    D --> E{Agent needed?}
    E -- Yes --> F[Tool Selector]
    F --> G[Execute Tool]
    G --> D
    E -- No --> H[Final Answer]

Observable outcome: Tool outputs are fed back into the next generation step.

Stage 4 — Deep Dive: RAG and Agent System Internals

• Internals A production RAG pipeline has four core modules working in concert: 1.
• Embedding Model — encodes both query and documents into a shared vector space.
• Bi-encoders (e.g., all-MiniLM-L6-v2 ) process documents at index time and queries at runtime, keeping online latency low.
• Vector Index — stores document embeddings for approximate nearest neighbor (ANN) lookup.

Observable outcome: Vector Index — stores document embeddings for approximate nearest neighbor (ANN) lookup.

Stage 5 — Advanced Architecture: Multi-Agent Orchestration

• Production RAG+Agent systems rarely rely on a single agent.
• Common advanced patterns: The most successful production deployments combine a tight retrieval corpus (well-scoped and frequently refreshed) with conservative agent permissions (read-only by default; write actions requi…

Observable outcome: Common advanced patterns: The most successful production deployments combine a tight retrieval corpus (well-scoped and frequently refreshed) with conservative agent permissions (read-only by default; write actions requi…

Stage 6 — Building a RAG+Agent Handler in Python

• This example implements the full RAG + Agent architecture described in this post: cosine similarity retrieval, an LLM conditioned on the retrieved passage, and a ReAct agent governed by a hard step budget.
• It was chosen because each of the three code sections maps directly to a concept covered above — retrieval scoring formula, grounded generation, and the runaway-agent failure mode — so you can see exactly where each mec…
• As you read it, focus on how the three stages are independent: retrieval fires before generation, and tool execution happens only when the LLM explicitly decides a call is needed.
• 📊 ReAct Agent Loop This sequence diagram traces the ReAct loop from the code example above: the LLM alternates between internal Thought steps (reasoning) and outbound Tool calls (acting), incorporating each Observation…

# pip install sentence-transformers langchain langchain-openai langchainhub numpy

import numpy as np
from sentence_transformers import SentenceTransformer
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langchain.agents import create_react_agent, AgentExecutor
from langchain import hub

# ── Step 1: RAG — retrieve the most relevant doc via cosine similarity ─────────
# This implements the retrieval scoring formula: s_i = (q · d_i) / (||q|| · ||d_i||)
embed_model    = SentenceTransformer("all-MiniLM-L6-v2")

docs           = [
    "Refund policy: 30-day returns on all items.",
    "Shipping: free on orders over $50.",
    "Account: password reset via settings > security.",
]
doc_embeddings = embed_model.encode(docs)   # shape: (3, 384) — indexed at startup

def retrieve(query: str) -> str:
    """Cosine similarity retrieval — higher score = more relevant passage."""
    q_vec = embed_model.encode([query])                           # (1, 384)
    sims  = (q_vec @ doc_embeddings.T)[0]                        # dot products
    sims /= (np.linalg.norm(q_vec) * np.linalg.norm(doc_embeddings, axis=1))
    return docs[int(sims.argmax())]   # top-1 retrieved passage

# ── Step 2: LLM conditioned on retrieved context ───────────────────────────────
# The LLM never guesses policy — it reasons only over the retrieved passage
llm     = ChatOpenAI(model="gpt-4o-mini", temperature=0)
query   = "What is the return policy?"
context = retrieve(query)            # → "Refund policy: 30-day returns on all items."
answer  = llm.invoke(f"Context: {context}\nQuestion: {query}")
print("RAG answer:", answer.content) # grounded in retrieved fact, not hallucinated

# ── Step 3: Agent — LLM + tool + hard step budget ──────────────────────────────
# The @tool decorator registers check_order_status in the agent's tool registry
@tool
def check_order_status(order_id: str) -> str:
    """Look up the current status of a customer order by ID."""
    return f"Order {order_id}: shipped, arrives 2025-08-10."  # mock DB lookup

tools    = [check_order_status]
prompt   = hub.pull("hwchase17/react")    # ReAct template: Thought → Action → Observation
agent    = create_react_agent(llm, tools, prompt)
executor = AgentExecutor(
    agent=agent, tools=tools,
    max_iterations=5,   # hard step budget — prevents the runaway-agent failure mode
    verbose=True        # prints the full reasoning scratchpad at each iteration
)

result = executor.invoke({"input": "What is the status of order ORD-42?"})
print("Agent answer:", result["output"])
# verbose=True reveals the ReAct loop:
#   Thought: I need to look up order ORD-42.
#   Action: check_order_status("ORD-42")
#   Observation: Order ORD-42: shipped, arrives 2025-08-10.
#   Final Answer: Your order ships on 2025-08-10.

sequenceDiagram
    participant U as User
    participant L as LLM
    participant T as Tool
    U->>L: Task / Question
    L-->>L: Thought: plan action
    L->>T: Tool call
    T-->>L: Observation
    L-->>L: Next thought
    L-->>U: Final Answer

Observable outcome: 📊 ReAct Agent Loop This sequence diagram traces the ReAct loop from the code example above: the LLM alternates between internal Thought steps (reasoning) and outbound Tool calls (acting), incorporating each Observation…

End-to-end outcome

Klarna's AI support bot answered 2.3 million customer queries in its first month — but hallucinated refund policies roughly 15% of the time, because it had no access to Klarna's current policy documents.

Alternatives considered

Decision Guide: From Smart Chatbots to Systems That Actually Act

Klarna's AI support bot answered 2.3 million customer queries in its first month — but hallucinated refund policies roughly 15% of the time, because it had no access to Klarna's current policy documents.

Decision Guide: Choosing the Right Mix

Decision Guide: Choosing the Right Mix

Failure and degraded modes

Trade-offs & Failure Modes: Trade-offs and Operational Failure Modes

• Detection: Performance at scale: Cache hot queries and parallelize independent tool calls to reduce tail latency.
• User or operator impact: Performance at scale: Cache hot queries and parallelize independent tool calls to reduce tail latency.