Buying Time To Think

The pause

You ask a question and nothing happens for twenty seconds. Maybe you watch a stream of the model working through the problem. Then an answer arrives.

The instinct is that this model is bigger, and bigger is slower.

Usually not. The difference is that it is doing more before it answers, and you are watching it happen.

Where the quality comes from

Remember how generation works: one token at a time, each conditioned on everything before it.

That has a consequence. A model that answers immediately has to commit to its first token with no opportunity to work anything out. All of its reasoning has to be implicit, compressed into a single forward pass.

Give it room to generate intermediate text first, and the situation changes. It can lay out the problem, try an approach, notice a contradiction, and revise. That intermediate text becomes context for the final answer.

So the improvement is not mysterious. It is thinking out loud, and thinking out loud helps for the same reason it helps people.

You could already do some of this

If you have ever added "think step by step" to a prompt and gotten a better answer, you have used a manual version of this.

What reasoning models change is that it is trained in rather than requested, applied more thoroughly, and often includes something you cannot do by asking: trying an approach, evaluating it, and backing out if it fails.

That backtracking is the part that most distinguishes them from a step-by-step prompt.

Where it pays

Extra thinking helps when the problem has multiple steps and a right answer:

• Maths beyond the trivial
• Debugging, where the cause is not where the symptom is
• Planning with dependencies and constraints
• Reconciling sources that disagree
• Careful analysis of a document you supplied
• Anything where one wrong step invalidates everything after

The common thread: you could check whether the answer is right. That is the tell.