Prompt Engineering Guide: From Zero-Shot to Chain-of-Thought

Scenario context

Prompt Engineering is the art of writing instructions that guide an LLM toward the answer you want. Zero-Shot, Few-Shot, and Chain-of-Thought are systematic techniques — not guesswork — that can dramatically improve accuracy without changing a single model weight.

Research shows LLMs pay the most attention to content at the beginning and end of the prompt. Information in the middle is often skipped.

`…

This walkthrough focuses on The "Lost in the Middle" Problem.

Before making a design choice, clarify:

• Research shows LLMs pay the most attention to content at the beginning and end of the prompt.
• Information in the middle is often skipped.
• Practical rule: Put your most important instructions or the specific text to analyze at either the very start or the very end.

Concepts in play: llm-prompt-engineering, consistency-replication, tree-data-structures, deep-learning-foundations

Stage 1 — The Super-Smart Intern With Zero Initiative

• Imagine a brilliant intern who has read every book ever written but has no common sense.
• Give them a vague instruction ("Fix this") and they stare blankly.
• Give them a structured instruction and they execute perfectly.
• Prompt Engineering is the art of writing the structured instruction.

Observable outcome: Prompt Engineering is the art of writing the structured instruction.

Stage 2 — How Prompts Work: The Fundamentals

• Every time you send text to an LLM, the entire input — system message, instructions, examples, and your question — is packed into a context window .
• The model reads it all at once and predicts the next most likely token.
• That prediction is directly shaped by every word you included.
• Three core ingredients of a well-formed prompt: Missing any one of these is the most common source of vague or off-topic responses.

Observable outcome: Three core ingredients of a well-formed prompt: Missing any one of these is the most common source of vague or off-topic responses.

Stage 3 — The Four Core Techniques

• Zero-Shot: Ask Directly Give the task with no examples: Best when: The task is well-defined and the model likely saw similar tasks in pre-training.
• Few-Shot: Teach by Example Provide 2–5 examples before the actual question: Why it works: The examples prime the model's "in-context learning." It extracts the pattern from the examples rather than relying on training d…
• Tip: Select diverse examples that cover edge cases.
• Avoid examples biased toward one label.

Classify this movie review as Positive or Negative:
"The acting was incredible and the plot kept me on the edge of my seat."

Classification:

Review: "Disappointing. Slow pacing, weak script." → Negative
Review: "A masterpiece. Emotional and visually stunning." → Positive
Review: "Predictable but fun summer blockbuster." → ?

Observable outcome: Avoid examples biased toward one label.

Stage 4 — Deep Dive: Self-Consistency and Tree-of-Thought

• Self-Consistency Instead of one CoT sample, generate 3–5 different reasoning chains and vote on the most common answer.
• Reduces reasoning variance on math problems.
• Tree-of-Thought (ToT) For planning problems, generate multiple "thought branches" at each step, evaluate which branch is most promising, and expand only that branch.
• Outperforms linear CoT on reasoning tasks that require backtracking.

answers = [generate_cot_response(prompt) for _ in range(5)]
from collections import Counter
final = Counter(answers).most_common(1)[0][0]

Observable outcome: Outperforms linear CoT on reasoning tasks that require backtracking.

Stage 5 — Real-World Applications: Prompt Engineering in Production

• Prompt engineering is not just a prototyping trick — it underpins real product features across industries.
• The table below shows common industry use cases and which technique typically drives the best results.
• Case Study — SaaS Customer Triage: A mid-size SaaS company was spending 8 engineer-hours per week manually labeling incoming support tickets before routing them to the correct team.
• After building a Few-Shot classifier prompt with 4 labeled examples per category (billing, bug, feature request, account access), the model achieved 91% routing accuracy on a held-out test set.

flowchart TD
    Task[Define Generation Task]
    WellDefined{Task clearly defined?}
    Examples{Have labeled examples?}
    MultiStep{Requires multi-step reasoning?}
    Variance{High-stakes, reduce variance?}

    ZeroShot["Zero-Shot (simple, direct)"]
    FewShot["Few-Shot (2-5 examples)"]
    CoT["Chain-of-Thought (step-by-step reasoning)"]
    SelfConsist["Self-Consistency (vote on 5 CoT runs)"]
    ReAct["ReAct (reason + act + observe)"]

    Task --> WellDefined
    WellDefined -->|No: rephrase| Task
    WellDefined -->|Yes| Examples
    Examples -->|No| ZeroShot
    Examples -->|Yes| FewShot
    FewShot --> MultiStep
    MultiStep -->|Yes| CoT
    MultiStep -->|No| FewShot
    CoT --> Variance
    Variance -->|Yes| SelfConsist
    Variance -->|No + tools| ReAct

Observable outcome: After building a Few-Shot classifier prompt with 4 labeled examples per category (billing, bug, feature request, account access), the model achieved 91% routing accuracy on a held-out test set.

Stage 6 — Hands-On Exercises: Putting the Techniques to Work

• The fastest way to internalize prompt engineering is to rewrite bad prompts.
• These three exercises cover the three most common failure modes.
• Exercise 1 — Zero-Shot to Few-Shot upgrade You have this Zero-Shot prompt that struggles with mixed-sentiment reviews: Rewrite it as a Few-Shot prompt with 3 examples that cover each label class.
• Make sure one example is genuinely mixed-sentiment so the model learns the boundary.

Classify this review as Positive, Negative, or Mixed:
"The camera is stunning but battery life is terrible."

A jacket costs $120. It's 30% off. What is the final price?

Observable outcome: Make sure one example is genuinely mixed-sentiment so the model learns the boundary.

End-to-end outcome

Research shows LLMs pay the most attention to content at the beginning and end of the prompt.

Alternatives considered

Decision Guide: Prompting Technique Selection

Not sure which technique to reach for?

Failure and degraded modes

Trade-offs & Failure Modes: Common Anti-Patterns

• Detection: Trade-offs & Failure Modes: Common Anti-Patterns
• User or operator impact: Trade-offs & Failure Modes: Common Anti-Patterns