Reasoning models — LLMs that generate extended internal chains of thought before producing an answer — represent a genuine capability advance. OpenAI's o3 and o3-mini, Anthropic's extended-thinking mode in Claude 4, and Google's thinking-enabled Gemini 2.5 Pro all demonstrate measurable improvements on tasks that standard models handle poorly: multi-step mathematical reasoning, logical deduction chains, and complex code debugging.
The problem is cost. A reasoning model token is not equivalent to a standard model token. The extended internal computation — which users do not see but which is billed — multiplies effective token cost by a factor of 4 to 8 depending on task complexity and the model's "thinking budget." OpenAI has implemented budget controls in the o3 API that let developers cap thinking tokens, but the default settings for high-accuracy tasks land in territory that makes per-query economics difficult for most applications.
The use cases where the math works are narrow but real. Tax and legal reasoning — where the cost of an error vastly exceeds the inference cost — is the clearest fit. Scientific literature synthesis for pharmaceutical research is another, where a single useful insight can justify hundreds of dollars in inference cost. Complex code bug localisation in large codebases, where a developer's hourly rate makes a $0.50 inference call economically trivial, is a third.
The use cases where the math does not work: customer support, content generation, data extraction at volume, and anything where throughput matters. For these workloads, a standard GPT-4o or Claude Sonnet call at a tenth the cost produces acceptable output.
What this creates is a tiered inference stack: fast, cheap models (Flash, Sonnet, GPT-4o-mini) for high-volume tasks; standard models for mid-complexity tasks; reasoning models reserved for decision-critical queries where accuracy difference is worth a 10x cost premium. Most enterprise AI architectures that have deployed at scale are implementing exactly this routing logic. The challenge is building classifiers that reliably route queries to the right tier — which is itself an inference task, creating a routing meta-problem that teams are still working through.