Prompt: the model sees a formatted sequence

A prompt is the input used to condition a model's response. In a chat application, it can include instructions, conversation history, tool results and retrieved material—not only the latest sentence typed by the user.

Role-labelled messages are an application-level structure. A chat template converts them into the token sequence the model was trained to interpret.

Use the checkpoint's template

This Transformers helper accepts an already-loaded chat model, its matching tokenizer and a message list. It generates only the new assistant text.

def reply(model, tokenizer, messages):
    inputs = tokenizer.apply_chat_template(
        messages, tokenize=True, add_generation_prompt=True,
        return_tensors="pt", return_dict=True,
    ).to(model.device)
    generated = model.generate(**inputs, max_new_tokens=64, do_sample=False)
    start = inputs["input_ids"].shape[1]
    return tokenizer.decode(generated[0, start:], skip_special_tokens=True)

add_generation_prompt supplies the template's assistant-start marker where required. The slice removes the input sequence from the returned tokens.

Different chat models can use different control tokens. Manually writing a familiar-looking “User:” prefix is not necessarily equivalent to the proper template.

Prompt changes influence the computation performed with existing weights. They do not themselves update those weights or guarantee that every included instruction will be followed.