Chain of thought: use intermediate steps as context

Chain of thought (CoT) is a sequence of intermediate reasoning steps produced before an answer. The model can use those steps as part of the context for what it generates next. An arithmetic problem, for example, can be split into intermediate quantities rather than jumping directly to a final number.

Early work showed that examples containing worked steps could improve performance on arithmetic, symbolic and commonsense tasks. Related work elicited stepwise answers with a brief instruction instead of worked examples. The benefit depends on the model and task.

More generation changes the computation

An autoregressive model processes generated intermediate tokens just as it processes other continuation tokens. Those steps cost additional decoding work and occupy context. A longer scratchpad gives the model more sequential computation to work with; it also creates more opportunities for an incorrect intermediate result to influence later steps.

This is one way reasoning can shift the balance between fast prompt processing and repeated generation. See prefill and autoregressive generation for those phases.

A useful derivation still needs checking

A written explanation is generated text, not a direct readout of every internal cause of a prediction. Research has demonstrated cases where a model's explanation omits factors that influenced its answer.

For a verifiable task, inspect the result and the consequential steps: substitute a calculated value, run the code or check the cited evidence. Reasoning steps can help a model solve a problem and help a reader follow a derivation. Their length or fluency alone does not establish correctness.