Chain of thought: thinking out loud

Why writing the steps helps

Chain-of-thought reasoning is the practice of having the model generate the intermediate steps of its working as text, before committing to a final answer. Mechanically the reason it helps is simple: each token the model writes becomes part of the context it reads to write the next token. By spelling out 'first I find the subtotal, then I add tax,' the model effectively gives its later computation the partial results it needs — the written steps are working memory it can lean on.

This is the concrete form of the deliberate-versus-intuitive idea from guide one. A direct answer forces all the computation into the single forward pass that produces one token. A chain of thought spreads that computation across dozens of tokens, each able to build on the last.

p\!\left(a \mid q\right)=\sum_{r} p\!\left(a \mid r,\,q\right)\,p\!\left(r \mid q\right)

Why writing the steps helps: chain of thought makes the reasoning r explicit instead of marginalizing it away, so the answer is conditioned on the model's own working.

How to ask for it

There are two classic ways to elicit it, both forms of chain-of-thought prompting. The zero-shot version simply appends an instruction like 'Let's think step by step.' The few-shot version shows one or two fully worked examples first, so the model imitates that step-by-step style on your real question.

Question: A cafe sells muffins at $3 and lattes at $5.
A customer buys 4 muffins and 3 lattes. What is the total?

Let's think step by step.
- 4 muffins at $3 each = 12
- 3 lattes at $5 each = 15
- total = 12 + 15 = 27
Answer: $27

Zero-shot chain of thought: the 'step by step' cue makes the model lay out its working before the answer.

Sampling many paths

A single chain of thought can take a wrong turn early and confidently march to a wrong answer. Self-consistency fixes a lot of that: sample several independent chains for the same question (using a non-zero temperature so they differ), then take a majority vote on the final answers. Different valid routes tend to converge on the same correct result, while mistakes scatter — so the vote favours the truth.

Send the same problem several times with a moderate sampling temperature.
Let each run produce its own full chain of thought and final answer.
Extract just the final answers and ignore the differing reasoning text.
Return the answer that appears most often as the consensus result.

\hat{a}=\arg\max_{a}\sum_{i=1}^{N}\mathbb{1}\!\left[a_i=a\right]

Self-consistency marginalizes over reasoning paths: sample N independent chains and return the final answer the most chains agree on.

Branching and breaking down

Two richer structures go beyond a single straight line. Tree of thought explores several candidate next steps at each stage, evaluates them, and expands the most promising — a search over a tree of partial solutions instead of one greedy path. Least-to-most prompting takes a different angle: first ask the model to break a hard problem into a sequence of easier sub-problems, then solve them in order, feeding each answer into the next.