QWE AI Academy Your model just spent 40 seconds (and a pile of tokens) on a two-line question
You’ve seen it: a simple ask, a long pause, then a correct but expensive answer. Or worse – a long pause and a worse answer because the model second-guessed itself into a loop. That’s the reasoning-model tradeoff in one screenshot.
A reasoning model (thinking model, LRM) is an LLM trained to burn extra inference compute on intermediate steps before it commits to a final reply. OpenAI popularized the commercial category with o1-preview in September 2024. DeepSeek-R1 landed in January 2025 with an open technical blueprint built around large-scale reinforcement learning. As of 2026, most major labs ship a dedicated reasoning line or a thinking toggle on the main model – check current product names, because IDs change fast.
Standard chat models still win on speed, cost, and everyday writing. Reasoning mode wins when multi-step logic, math, or code correctness matter more than latency. The skill isn’t “always turn thinking on.” It’s knowing when extra tokens buy accuracy – and when they buy analysis paralysis.
Why plain LLMs stall on multi-step work
Fast next-token prediction is System-1 for language models: great at summaries and simple Q&A, shaky when the job needs decomposition, backtracking, or checking intermediate results.
Chain-of-thought prompting helped. It was still a hack. Reasoning models bake deliberation into training – often RL on verifiable math and code rewards – and into inference via dedicated reasoning tokens or a thinking budget. What you actually get, per IBM’s overview: longer internal traces, stronger logic-heavy scores, higher latency, and higher token spend.
DeepSeek’s report (arXiv:2501.12948) showed pure RL could pull out self-reflection and strategy shifts without hand-written reasoning demos for every path. That openness is why smaller distilled reasoners exist at all.
How to run a reasoning model without burning budget
Three controls matter: pick a reasoning-capable model or mode, set effort/budget on purpose, and write a short prompt that states the goal – not the thought process.
1. Turn on thinking the way your provider expects
Lower effort → fewer reasoning tokens and less waiting. Higher effort → hard planning and coding. Miss a supported value and some stacks answer with HTTP 400.
OpenAI’s current guidance (as documented in their reasoning guide) favors the Responses API with a reasoning object. Effort strings are model-dependent: none, minimal, low, medium, high, xhigh, max. Some models reject none outright – read the model page before you ship defaults (OpenAI reasoning guide).
from openai import OpenAI
client = OpenAI()
response = client.responses.create(
model="your-reasoning-model-id", # use the current provider id
reasoning={"effort": "medium"},
input=[{
"role": "user",
"content": (
"Given this CSV schema and three failing unit tests, "
"propose the smallest code change that makes all tests pass. "
"Return only: (1) root cause in one sentence, "
"(2) patched function, (3) why the old tests failed."
),
}],
)
print(response.output_text)Claude extended thinking is a budget problem, not a vibes problem. Manual paths use budget_tokens with a documented minimum of 1024; other generations expose adaptive/effort controls. Thinking tokens bill as output – see Claude’s extended thinking docs. Start near the minimum on light tasks. Raise the budget only when your own evals show a real gain. Prior thinking blocks are often stripped from later context automatically; you still paid to generate them once.
2. Prompt for outcomes, not for “think harder”
Native reasoners already allocate internal steps. Official guidance from OpenAI, DeepSeek, and Claude lands in the same place: task, constraints, output shape. Skip “think step by step,” long few-shot CoT demos, and nested process scripts – they can fight the trained policy.
- Good: “Find the cheapest shipping combination under 5 kg. List options tried, then the final cart total.”
- Bad: “First brainstorm, then reason carefully in five detailed paragraphs, then double-check each assumption…”
Math-only format hints like “put the final answer in boxed{}” are fine on some DeepSeek-style stacks. For agents, hand tools early so the model stops meta-reasoning and starts calling.
Pro tip: Run the same hard prompt at
lowandhigheffort once. If the answers match and only the token count changes, lock the lower setting for that task class. Effort is a dial, not a badge of seriousness.
Real workflow: debug a flaky data pipeline
Nightly job drops ~3% of rows after a schema change. A standard model often guesses a plausible fix. Medium-effort reasoning tends to walk failure modes, compare timestamps, and propose a minimal patch.
Feed it sample bad rows, the transform function, and the three failing assertions. Ask for root cause + smallest diff + a regression test. Stream when the API separates reasoning from answer fields so you can log cost without waiting on the full monologue.
If the job is “rewrite this email,” switch thinking off. You’re paying for deliberation you don’t need.
The overthinking trap most tutorials skip
Easy, underspecified, or missing-premise questions are where reasoning models blow the budget. Field reports and research coverage (including IEEE Spectrum’s software-engineering agent writeups and missing-premise overthinking work) show longer traces, more “wait/actually” loops, and stretches where lower effort – or a non-reasoning model – wins on both cost and success rate. More overthinking correlated with fewer resolved issues in that agent setup.
Missing premises are ugly in practice: the model may spiral on guessed user intent instead of saying “this is underspecified.” If your product accepts messy input, detect thin prompts and either refuse early or force low effort.
Billing mechanics matter here. Reasoning tokens are real output spend even when the UI collapses them. Claude documents thinking charged as output; OpenAI counts reasoning tokens separately from the final answer tokens you read.
Is a longer visible trace always “better thinking”? Not reliably. Traces are shaped under reward pressure. They can look thorough while only partly matching the internal path. Treat them as a debugging aid, not courtroom evidence.
Quick decision table
| Situation | Prefer |
|---|---|
| Math proofs, contest coding, multi-constraint planning | Reasoning on, medium→high effort |
| Agent loops with tools and ambiguous plans | Reasoning on; A/B high vs medium on your eval set |
| Chat, rewrite, classify, simple extract | Standard model or effort none/low |
| Ill-posed or incomplete user questions | Clarify first; avoid max effort |
| Hard latency or unit-cost caps | Standard model, or low effort + cache |
Related next steps: chain-of-thought for non-reasoning models, agent tool use, production token-cost monitoring.
FAQ
Do I still need “think step by step” prompts?
Usually no. Clear goals beat process theater.
When should I raise reasoning effort?
When offline evals show a clear accuracy jump on that task family – brittle coding refactors, multi-document reconciliation, science-style derivation. Start medium for planning/coding. Reserve xhigh/max for cases you’ve measured. If quality plateaus and only latency climbs, drop back.
Are open reasoning models “good enough” vs closed ones?
Often yes for math and code-shaped work, especially if you can self-host distilled weights. DeepSeek-R1 (Jan 2025) made the training recipe inspectable and spawned smaller distillations. Closed APIs still tend to win on tool ecosystems, structured outputs, and SLAs. The honest gap isn’t one leaderboard number – it’s ops: rate limits, function-calling maturity, and whether visible traces matter for compliance. Benchmark on your tasks; public contest-math scores transfer imperfectly to messy business logic.
Pick one recurring hard task this week. Run it three ways – standard model, reasoning at low effort, reasoning at high effort – with the same prompt. Log tokens, latency, and correctness. Keep the cheapest setting that stays inside your error budget. That single experiment beats another abstract model roundup.