What are Logits in Machine Learning and Why They Matter

Scenario context

Logits are the raw, unnormalized scores produced by the final layer of a neural network — before any probability transformation. Softmax converts them to probabilities. Temperature scales them before Softmax to control output randomness.

1. Training stability: Working with logits and applying log-Softmax is numerically more stable than computing probabilities first, then taking their logarithm for the **cr…

This walkthrough focuses on Why Not Output Probabilities Directly?.

Before making a design choice, clarify:

• Training stability: Working with logits and applying log-Softmax is numerically more stable than computing probabilities first, then taking their logarithm for the cross-entropy loss — the standard training objective th…
• Flexibility: The same logits can be processed differently (Softmax for classification, sigmoid for multi-label, raw for regression).
• Temperature scaling: You can reshape the distribution from the logits before applying Softmax, which you couldn't do if the network directly output probabilities.

Concepts in play: vector-embeddings-and-similarity, llm-prompt-engineering, deep-learning-foundations, quantization-and-llm-inference

Stage 1 — The Confidence Meter Before Calibration

• A doctor looks at an X-ray and assigns a raw "gut score": Cancer likelihood = 8.2, Normal = 1.4.
• These numbers are not percentages — they haven't been normalized yet.
• To get percentages, she runs them through a calibration formula.
• Logits are those raw gut scores.

Observable outcome: Logits are those raw gut scores.

Stage 2 — What Exactly Is a Logit?

• The word logit comes from statistics — it's short for "log-odds unit." In modern deep learning, the term is used more loosely to mean the raw, unnormalized score produced by the final linear layer of a neural network be…
• Think of a neural network as a pipeline: raw inputs pass through multiple layers of transformations, and at the very end, a linear layer projects everything into a vector of numbers — one number per possible output clas…
• Key properties of logits: - They can be any real number — positive, negative, or zero.
• • A higher logit means the network is more confident in that class relative to others.

Observable outcome: - A higher logit means the network is more confident in that class relative to others.

Stage 3 — From Network Output to Prediction

• A classification network predicting image labels outputs raw scores: These numbers have no fixed scale.
• A "4.5" just means "more confident Cat than the others." Softmax converts logits to probabilities: $$P i = \frac{e^{z i}}{\sum j e^{z j}}$$ Applied to the example — computing e^4.5 = 90.02, e^−1.2 = 0.30, e^0.8 = 2.23,…
• 📊 Logits to Token Pipeline This flowchart traces the journey of a raw logit vector from model output to a sampled output token.
• Starting with the raw scores produced by the final linear layer, Softmax converts them into a probability distribution, Top-K filtering narrows the candidates to the most likely tokens, and sampling draws the final outp…

Raw output (logits):
  Cat:  4.5
  Dog: -1.2
  Bird: 0.8

flowchart LR
    LG[Raw Logits] --> SM[Softmax]
    SM --> PR[Probabilities]
    PR --> TK[Top-K Filter]
    TK --> SP[Sample Token]
    SP --> OUT[Output Token]

Observable outcome: Starting with the raw scores produced by the final linear layer, Softmax converts them into a probability distribution, Top-K filtering narrows the candidates to the most likely tokens, and sampling draws the final outp…

Stage 4 — The Logit Pipeline: From Input to Prediction

• Here is how logits flow through the full inference pipeline in a classification or language model: Step-by-step: 1.
• Input — raw data (tokens, pixels) enters the network.
• Hidden layers — extract abstract features through multiple transformations.
• Linear layer — maps features to a logit per class (no activation, just weights × input + bias).

flowchart TD
    Input["Input Data (text, image, etc.)"] --> Encoder["Encoder / Hidden Layers (feature extraction)"]
    Encoder --> Linear["Final Linear Layer (produces raw logits z_i)"]
    Linear --> Temp[" Temperature T (optional scaling)"]
    Temp --> Softmax[Softmax  probability distribution P_i]
    Softmax --> Decision[Argmax or Sampling  predicted class / next token]

Observable outcome: Linear layer — maps features to a logit per class (no activation, just weights × input + bias).

Stage 5 — Deep Dive: Temperature: Reshaping the Logit Distribution

• In language models, the vocabulary logits are scaled by a temperature $T$ before Softmax: $$P i = \frac{\exp(z i / T)}{\sum j \exp(z j / T)}$$ Effect of temperature on the cat/dog/bird example (logits: 4.5, -1.2, 0.8):…
• Repetitive in language models.

flowchart LR
    Hidden[Hidden Layer Output] --> Logits["Logit Layer (raw z_i scores)"]
    Logits --> Temp[ Temperature T]
    Temp --> Softmax[Softmax  probabilities P_i]
    Softmax --> Sample[Sample or Argmax  predicted class / next token]

Observable outcome: Repetitive in language models.

Stage 6 — Real-World Applications of Logits

• Logits appear in almost every modern ML system.
• Here are concrete examples of where they matter: 1.
• Large Language Models (ChatGPT, Gemini, Claude) Every time an LLM generates the next word, it produces a logit for every token in its vocabulary (often 50,000+ tokens).
• Temperature and top-k/top-p sampling are applied to these logits before selecting the next token.

sequenceDiagram
    participant C as Context Tokens
    participant M as LLM
    participant L as Logits
    participant S as Sampler
    C->>M: input sequence
    M->>L: compute logits
    L->>S: apply temperature
    S-->>C: append new token
    C->>M: next iteration

Observable outcome: Temperature and top-k/top-p sampling are applied to these logits before selecting the next token.

End-to-end outcome

Training stability: Working with logits and applying log-Softmax is numerically more stable than computing probabilities first, then taking their logarithm for the cross-entropy loss — the standard training objective that penalizes the model proportionally to how wrong its predicted probability distribution is compared to the true label.

Alternatives considered

Decision Guide: When to Use Logits vs. Probabilities

Decision Guide: When to Use Logits vs. Probabilities

Failure and degraded modes

Trade-offs & Failure Modes: Logits in Different Contexts

• Detection: Trade-offs & Failure Modes: Logits in Different Contexts
• User or operator impact: Trade-offs & Failure Modes: Logits in Different Contexts

Key Lessons and Common Pitfalls

• Detection: Lesson 1 — Don't double-apply Softmax.
• User or operator impact: Lesson 1 — Don't double-apply Softmax.