'The Developer''s Guide: When to Use Code, ML, LLMs, or Agents'

Scenario context

AI is a tool, not a religion. Use Code for deterministic logic (banking, math). Use Traditional ML for structured predictions (fraud, recommendations). Use LLMs for unstructured text (summarization, chat). Use Agents only when a task genuinely requires multi-step planning and external tool calls.

---

Choosing the right approach starts by analyzing your problem's input structure, output requirements…

This walkthrough focuses on The Decision Flow: From Problem to Paradigm.

Before making a design choice, clarify:

• Choosing the right approach starts by analyzing your problem's input structure, output requirements, and tolerance for non-determinism.

Concepts in play: llm-agents-and-langgraph, deep-learning-foundations, event-driven-architecture, hash-tables-and-bloom-filters

Stage 1 — One Codebase, Four Paradigms: Know Before You Reach for the LLM

• The most expensive mistake in modern software is using an LLM for a problem deterministic code solves in 5 lines.
• Before adding an AI component, ask two questions: 1.
• Does the input have known structure?
• If both answers are no and the input is natural language, then LLMs are the right tool.

Observable outcome: If both answers are no and the input is natural language, then LLMs are the right tool.

Stage 2 — Deep Dive: The Four Paradigms at a Glance

• A quick reference before diving deeper.
• Each paradigm has a distinct role — picking the wrong one costs time, money, and unnecessary complexity.
• Think of it as a filter: start at Code and only move down the list when the previous paradigm genuinely cannot handle the problem.

Observable outcome: Think of it as a filter: start at Code and only move down the list when the previous paradigm genuinely cannot handle the problem.

Stage 3 — Pure Code: When Determinism Is Non-Negotiable

• Any operation where you can write an explicit rule belongs here.
• When code beats AI: Banking transactions, data migrations, format validation, mathematical computations, protocol parsing.
• The rule: if a junior developer could write a test that covers every case, write code.

Observable outcome: The rule: if a junior developer could write a test that covers every case, write code.

Stage 4 — Traditional ML: Patterns in Structured Tabular Data

• Use ML when the rule is too complex to write by hand, but the input is structured (rows and columns with known features).
• ML requires: labeled training data, feature engineering, model evaluation, and a retraining pipeline.
• If you don't have those, use rules instead.

flowchart LR
    Features[Structured Features age, amount, location] --> Model[ML Model XGBoost / Random Forest]
    Model --> Prediction[Score or Label fraud probability]

Observable outcome: If you don't have those, use rules instead.

Stage 5 — Deep Dive: LLMs: When the Input Is Unstructured Text

• LLMs excel at tasks where the input is free-form text and the output is also text (or a structured schema derived from text).
• Cost reminder: Every LLM call costs money and adds latency.
• Never use an LLM for tasks that code or a simple ML model can solve.
• 🔬 Internals Traditional software follows deterministic control flow: the same input always produces the same output via explicit conditional branches.

Observable outcome: 🔬 Internals Traditional software follows deterministic control flow: the same input always produces the same output via explicit conditional branches.

Stage 6 — Agents: For Multi-Step Goals That Require External Tools

• Use agents when completing the task requires: 1.
• Multiple actions (not just one generation) 2.
• Calling external APIs or tools (not just text transformation) 3.
• Adapting plans based on intermediate results Red flag: If you're describing your agent as "it just generates text and returns it," you needed a plain LLM call, not an agent.

Observable outcome: Adapting plans based on intermediate results Red flag: If you're describing your agent as "it just generates text and returns it," you needed a plain LLM call, not an agent.

End-to-end outcome

Choosing the right approach starts by analyzing your problem's input structure, output requirements, and tolerance for non-determinism.

Alternatives considered

Decision Guide: Choosing the Right Tool

The flowchart below turns the two intro questions into a repeatable process you can run mentally for any new backlog item.

HuggingFace Transformers & scikit-learn: LLM Inference vs Traditional ML Pipeline Side by Side

HuggingFace Transformers is the go-to Python library for loading and running open-source LLMs (GPT-2, LLaMA, Mistral).

Failure and degraded modes

Trade-offs & Failure Modes: When Each Paradigm Breaks Down

• Detection: This flowchart maps any new requirement through a four-question decision tree that routes it to the right paradigm.
• User or operator impact: This flowchart maps any new requirement through a four-question decision tree that routes it to the right paradigm.