In-context learning: examples change this request

In-context learning lets a model adapt its response to instructions or examples supplied in the current context. Its weights stay unchanged. What changes is the information available when it computes the next token.

A few examples can specify a format, a label convention or a task. They can establish an arbitrary mapping that was chosen for this request rather than a fact the model had to memorize during pretraining.

Give a rule through examples

This Transformers/PyTorch example gives a small instruction model a private label convention: dax means approving sentiment and wug means disapproving sentiment. The prompt supplies that convention; the model is not fine-tuned. The first run downloads the model.

from transformers import pipeline

generate = pipeline("text-generation", model="HuggingFaceTB/SmolLM2-135M-Instruct")
prompt = """Use the labels demonstrated below. Return only the label.
Review: A thoughtful, engaging film. Label: dax
Review: Dull and badly acted. Label: wug
Review: I loved every minute. Label:"""
result = generate([{"role": "user", "content": prompt}],
                  max_new_tokens=8, do_sample=False)
label = result[0]["generated_text"][-1]["content"]

Try changing both example labels consistently while keeping the final review unchanged. A model using the examples should change its answer accordingly. Compare the responses to see whether the model follows the new convention.

The adaptation lives in the context

The prompt's representations and attention computations affect the output. Starting a new request without those examples removes that particular source of task information unless an application supplies it again.

Research has found connections between in-context adaptation and learning algorithms in controlled settings. That does not mean ordinary inference secretly updates the stored model weights.

Few-shot prompting supplies examples; fine-tuning changes trainable parameters. Prompt caching can reuse computation for repeated examples, but the examples still occupy context.