Slow Thinking for Models

The Need for Thinking Time

Why can a model answer a simple fact instantly but struggle with a math problem like $$12345 \times 56789$$?

Complex tasks require more than just pattern matching. They require test-time computation.

This is computation spent after training but before the final answer is shown. It allows the model to process steps that a direct answer would skip.

Fast versus Slow Reasoning

Models often use fast reasoning for easy prompts. This relies on immediate intuition and pattern completion.

Slow reasoning is deliberate. It allocates extra steps to inspect assumptions and split tasks into smaller parts.

Think of it like a human. You answer "What is your name?" instantly, but you pause and use a scratchpad for long division.

How Models Think at Inference

A model can "think" by generating intermediate tokens. This might look like a chain of thought or a scratchpad of partial products.

It can also use search, revision, or tool calls to verify its work.

For example, a coding model might run a test before finalizing a script. This extra work helps the model recover from mistakes before you see the result.

When to Spend Compute

Extra tokens do not always mean better quality. More work must lead to a better decision to be useful.

Use slow reasoning for tasks requiring decomposition or error correction.

Skip it for simple greetings or basic facts. Spending extra computation on easy tasks is wasteful and does not change the outcome.

Test-time computation refers to extra work a model does _____.

• during inference
• during training

Answer: during inference

Test-time computation happens during inference (when the model is generating an answer), not during the initial training phase.

Models use _____ for immediate pattern completion, while _____ is used for deliberate tasks like error correction.

Answer: Models use fast reasoning for immediate pattern completion, while slow reasoning is used for deliberate tasks like error correction.

Fast reasoning is like intuition; slow reasoning involves deliberate steps to ensure accuracy on complex problems.

Which scenario justifies the use of extra computation during inference?

• A user says 'Hello' to the chatbot
• A model solves a complex multi-step math problem
• A user asks for the current date
• A model repeats a single word 100 times

Answer: A model solves a complex multi-step math problem

Extra computation is useful only when it can change the quality of the decision. Simple greetings are handled efficiently by fast reasoning.