Fine-tuning: change the model's learned behaviour

Fine-tuning continues the training of an existing model on selected data. A prompt changes what a model receives for one request; fine-tuning changes trainable parameters and carries that change into future requests.

For a language model, examples can teach a response format, a specialised writing style or a task such as extracting fields from documents. The training objective decides which predictions receive feedback. In instruction tuning, that often means learning desired responses to instructions rather than treating every part of a conversation as a target.

What actually changes

Full fine-tuning updates the selected base-model weights. Parameter-efficient methods such as LoRA freeze the base and train a smaller set of added parameters. Either way, a forward pass produces predictions, a loss compares them with the targets, and backpropagation supplies gradients for the update.

The result is an adapted model or adapter. The training examples do not need to be supplied again as prompt demonstrations every time the model runs.

Teach the behaviour you can measure

Consistent examples matter more than simply accumulating rows. If examples disagree about format or omit difficult cases, the model receives conflicting or incomplete feedback. Evaluate on held-out cases resembling the real task, and compare with the original model.

Fine-tuning can also overfit or weaken other capabilities. A lower training loss alone does not establish a useful improvement.

For changing facts, retrieval often offers a more direct update path: replace a document and retrieve it. Fine-tuning is particularly useful when the desired change concerns how the model performs a task.