Thinking Models: Effort, Cost and When They Pay Off
How thinking and reasoning modes work, when to use more effort, and how to prompt them.
Thinking models write hidden steps before they answer. That helps on hard, multi-step problems. It also costs more and takes longer. Give them a goal and limits, not every step. Test low and high effort, and keep the cheapest that passes.
- Reasoning tokens
- The hidden thinking steps, billed like output.
- Effort
- A setting for how much the model thinks.
- Length limit
- The cap on total tokens, including thinking.
- Fast and cheap
- Good for simple questions
- Good for chat and high volume
- Slower and costlier
- Good for hard multi-step problems
- Needs room in the length limit
Some models can think before they answer. They work through a problem in a hidden scratchpad, then reply. Anthropic calls this thinking. OpenAI calls these reasoning models. The idea is the same. More thinking can give better answers on hard problems. It also costs more and takes longer. This lesson shows when it pays off.
What actually happens
- The model writes private thinking steps first. These steps are made of tokens.
- You are billed for those tokens, even when you cannot see all of them.
- Then the model writes the final answer you read.
The knob you control
Both companies give you a way to set how much the model thinks. OpenAI uses an effort level, such as low, medium or high. Claude has an effort setting, and older modes used a thinking budget, which is a cap on thinking tokens. The details change between versions. Check the current docs for your model. The idea stays the same: more effort means more time and cost.
When to turn thinking up
- Maths, logic and puzzles with many steps.
- Hard code problems and bug hunts.
- Plans that must fit many rules at once.
- Long research tasks where the model checks its own work.
When to leave it low
- Simple questions, such as rewriting a sentence.
- Chat that must feel instant.
- High volume jobs where cost matters more than a small gain in quality.
How to prompt a thinking model
Do not write every step for it. The OpenAI docs advise giving a clear goal, the limits and the output format. Then let the model plan. Also say what done means, and how it should check its work.
Goal: choose the cheapest delivery plan for our shop. Limits: orders must arrive in 3 days; budget is fixed at the numbers below. Material: <rates> ...paste the rate table... </rates> Done means: one plan, with a short reason, and a check that it meets the 3 day rule. Output: a table of the plan, then two sentences of explanation.
Watch the cost
- Thinking tokens count toward the length limit. Leave enough room, or the answer can be cut off before it appears.
- Higher effort is not the first fix for a weak prompt. Improve the prompt first.
- Test a low setting and a higher one on the same cases. Keep the cheaper one that passes.
A simple test you can run today
Pick one hard question from your own work. Ask it first with low effort. Then ask again with a higher setting. Compare three things: the quality, the time and the cost. Write down what you see. Often the gain is small, and the cheaper setting is enough. Sometimes the gain is large, and then you have a clear reason to pay for it.
- ✓Thinking models write hidden steps first, then the answer. You pay for those steps.
- ✓More effort helps with hard, multi-step problems, and wastes money on easy ones.
- ✓Give a thinking model the goal, limits and format, not every step.
- ✓Leave room in the length limit for thinking and the answer.
- ✓Test low and high settings, and keep the cheapest one that passes.
Compare thinking levels
- Pick three questions: one easy, one medium and one hard.
- Ask a thinking-capable tool with low effort, then with high effort if it lets you choose.
- Note the speed and quality of each answer.
- Write one rule for yourself: when you will use higher effort.
Q1Why does a thinking model cost more?show answer
It writes extra hidden steps first. Those tokens are billed as output.
Q2Should you list every step for a thinking model?show answer
No. Give the goal, the limits and the format, and let it plan. Add how to check the work.
Q3Your answer was cut off with no text. What might be wrong?show answer
Thinking used the length limit up. Raise the limit or lower the effort.