Thinking Out Loud: Chain-of-Thought & Self-Consistency

Why a model fails at 'easy' multi-step problems

Ask a model for the final answer to a multi-step word problem and it may blurt out a wrong number with total confidence. The reason traces back to next-token prediction: when you demand the answer immediately, the model must compress several reasoning steps into one leap. Each token it generates is also a place to 'think', and a one-token answer gives it nowhere to work.

P(x_{1:T}) = \prod_{t=1}^{T} P(x_t \mid x_{<t})

Next-token prediction factorizes the whole answer left-to-right, so each token — including the final one — is only as good as the context before it.

The fix is delightfully simple: let it write the steps before the answer.

Chain-of-thought: ask for the steps

Chain-of-thought (CoT) prompting asks the model to reason step by step before committing to an answer. The intermediate steps become extra context the final answer can build on. Often you trigger it with nothing more than a line like 'Let's think step by step' — that small nudge alone lifts accuracy on arithmetic and logic tasks.

Q: A shop sells pens at 3 for $5. How much for 12 pens?
Let's think step by step.
A:

Zero-shot CoT: the single 'think step by step' line invites visible reasoning before the answer.

You can also teach the style of reasoning with examples — a few-shot CoT prompt where each demonstration shows the worked steps, not just the answer. This is in-context learning applied to reasoning: the model imitates how your examples think, not only what they conclude.

Break it down: least-to-most prompting

For problems too big for one chain, least-to-most prompting first asks the model to break the problem into simpler sub-problems, then solves them in order, feeding each solved piece into the next. Where plain CoT reasons through one problem, least-to-most decomposes a hard problem into a staircase of easier ones.

Stage one: prompt the model to list the sub-questions needed to answer the main question.
Stage two: solve the first sub-question on its own.
Stage three: append that answer and solve the next sub-question, repeating until the final one is reached.

Because each stage stays small and well-grounded, this scales to problems where a single long chain would lose the thread — a first taste of composing prompts rather than writing one.

Self-consistency: vote across several tries

A single chain of thought can take a wrong turn early and ride it to a wrong answer. Self-consistency hedges against that: run the same CoT prompt several times with some randomness, get several independent reasoning paths, then take the answer the majority agree on.

The randomness comes from sampling — you raise the temperature so the paths differ instead of repeating. Different routes that converge on the same answer are good evidence it's right; if they scatter, you've learned the model is unsure. The cost is real: you pay for several generations per question, so reserve it for high-value answers.

p_i = \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)}

Raising the sampling temperature T flattens the next-token distribution, so repeated runs explore different reasoning paths to vote over.

Putting reasoning prompts together

These three stack naturally. Use chain-of-thought to make the model show its work, least-to-most to decompose a problem that's too big for one chain, and self-consistency to vote away unlucky single runs. Each trades extra tokens and time for reliability, so apply them where correctness is worth the cost.