Logits: change the scores before choosing a token

Logits are unnormalized model scores. A language model produces one score for each vocabulary token at a position; softmax converts those scores into probabilities.

Only score differences matter to softmax. Adding the same constant to every logit leaves the distribution unchanged, while multiplying their differences changes its concentration.

Mask a token at the score level

This NumPy example suppresses one candidate before normalization and sampling. The numbers are illustrative model outputs.

import numpy as np

logits = np.array([2.4, 1.2, -0.5, 0.8])
logits[1] = -np.inf
probabilities = np.exp(logits - logits.max())
probabilities /= probabilities.sum()
next_id = np.random.default_rng(7).choice(len(logits), p=probabilities)

The blocked token receives zero probability, and the remaining probabilities sum to one. Subtracting the largest score before exponentiation improves numerical stability without changing the distribution.

A logit is not a probability, and neither token probability nor a large score establishes factual confidence. Training losses often accept logits directly and perform stable normalization internally. Keep that contract clear to avoid applying softmax twice.