Classification: separate the score from the decision

Classification predicts a category: spam or ordinary mail, one image class, or several labels attached to the same document. A model often produces scores or probabilities first; a decision rule turns them into labels.

These are separate operations. Training changes the scoring function. Moving a threshold changes how its existing scores are used.

Fit once, choose the threshold twice

This scikit-learn example fits a small logistic classifier to two-feature observations, then applies two policies to new points. Install scikit-learn and numpy.

import numpy as np
from sklearn.linear_model import LogisticRegression

features = np.array([[-2, -1], [-1, -2], [-1, 0], [1, 0], [1, 2], [2, 1]])
labels = [0, 0, 0, 1, 1, 1]
model = LogisticRegression().fit(features, labels)

probabilities = model.predict_proba([[0.1, 0.1], [2, 2]])[:, 1]
decisions = {t: probabilities >= t for t in (0.5, 0.8)}

The probabilities stay fixed while the decisions can change. The 0.8 threshold asks for stronger evidence before choosing the positive class; it does not retrain the classifier.

Multiclass classification chooses among exclusive alternatives. Multilabel classification can select several simultaneously. A softmax over exclusive classes and independent sigmoid outputs therefore express different problems, even when both return a vector of numbers.