Reasoning Models Explained: How AI Thinks

by

Key Takeaways

Reasoning models are AI systems that generate an internal chain of thought before producing a final answer, rather than replying instantly. Spending extra computation at answer time — known as test-time compute — was popularized by OpenAI’s o1 in September 2024 and made open by DeepSeek-R1 in January 2025. Thinking longer raises accuracy on hard math, coding, and science problems, but it also raises cost and latency, so more thinking is not always the better answer.

  • Chain-of-thought started with a 2022 Google paper — eight worked examples let a 540B model hit state-of-the-art on the GSM8K math benchmark.
  • OpenAI’s o1 (September 2024) and DeepSeek-R1 (January 2025) put “thinking before answering” into the mainstream.
  • DeepSeek-R1 scored 79.8% on AIME 2024 versus o1’s 79.2% — matching a closed flagship with open weights.
  • The intelligence is not free: cost and latency rise, and on simple tasks models can “overthink” and lose.

Ask a modern AI chatbot a hard question and you may notice something new: instead of answering instantly, it pauses, flashing a “thinking” indicator for a few seconds. Ask the same question to two different systems and one fires back immediately while the other deliberates — and often returns a far better answer. That gap is the whole story of how AI reasoning models think before they answer.

A reasoning model does not blurt out a reply. It first works through the problem the way a person scribbles steps on scratch paper before writing down a final answer. OpenAI’s o1 popularized the approach in 2024, and DeepSeek-R1 followed in 2025 as an open model anyone could download — together creating the impression that AI suddenly got smarter. This guide explains what reasoning models are, how they work, how they differ from ordinary AI, and why they are not always the right tool.

AI-generated illustration: a glowing translucent brain with chain-of-thought nodes flowing step by step toward a single bright answer
Reasoning models work through several steps of “thought” before answering — AI-generated illustration (OpenAI gpt-image-1-mini)

What Are Reasoning Models? AI That Thinks Before Answering

A reasoning model is a language model that, instead of emitting an answer the instant it receives a prompt, first generates the intermediate steps needed to reach that answer. The core idea is simple: the harder the problem, the more computation you let the model spend “thinking,” and the higher the accuracy climbs. Spending that extra computation at answer time — during inference rather than training — is what the field calls test-time compute.

This marked a genuine shift in direction. For years, the path to a smarter model ran through “bigger model, more data.” Reasoning models opened a second axis: think harder at the moment of answering. Lifting performance on hard problems without inflating model size is not a tweak — it is a change in approach.

In simpler terms: If a normal AI is the student who writes down an answer the moment they read the question, a reasoning model is the student who works the problem on scratch paper and checks it before committing. It takes longer, but it gets more of the hard questions right.

How They Work: Chain of Thought and “Thinking Tokens”

Two ideas unlock how reasoning models work: the chain of thought and the thinking token. Before delivering a final answer, a reasoning model generates a long stream of internal “thinking tokens” that you usually never see. Those tokens are the intermediate work — the model’s scratch paper.

That internal work tends to break into three moves. The model decomposes a complex problem into smaller steps, catches and corrects its own errors as it goes, and sometimes drafts several candidate answers before selecting the most plausible one — a strategy known as best-of-N. Best-of-N matters because solving a problem several times and scoring the attempts raises the odds of landing on the right answer even when the first try misses. According to Towards Data Science, this hidden work can balloon to roughly 2,000 tokens just to produce a single-number answer.

Training has evolved too. Rather than rewarding only whether the final answer was correct, newer methods reward whether each step of the reasoning was sound — so-called process reward models. As Hugging Face has documented, the 2025 ReTool framework went further, training a model to interleave natural-language reasoning with tool use and posting gains on multi-hop question answering and math.

a dark blue background with lots of lines
A reasoning model chains many internal steps before it speaks — Photo by Sandip Kalal on Unsplash
In simpler terms: Thinking tokens are the model muttering to itself under its breath. You never hear the muttering, but the longer and sharper it is, the more likely the spoken answer is correct.

Where It Came From: From the Chain-of-Thought Paper to o1 and DeepSeek-R1

The roots reach back to 2022. A paper from Google’s Jason Wei and colleagues introduced “chain-of-thought prompting.” It showed that giving a 540-billion-parameter model just eight worked examples was enough to reach state-of-the-art accuracy on the GSM8K grade-school math benchmark, surpassing even a fine-tuned GPT-3 paired with a verifier.

An even simpler discovery followed soon after — appending the single phrase “Let’s think step by step” to a prompt lifted reasoning accuracy with no examples at all (Kojima et al., 2022). This “zero-shot chain of thought” hinted that eliciting a thought process could be far cheaper than anyone assumed.

The decisive turn was building the reasoning ability into the model itself. On September 12, 2024, OpenAI released a preview of o1, the first widely available model that spent compute on an internal “reasoning token” budget before answering, beating GPT-4o on math, science, and coding. DeepSeek then released R1 on January 20, 2025 — trained with reinforcement learning and shipped with open weights — putting a high-end reasoning model into anyone’s hands.

white robot near brown wall
o1 and DeepSeek-R1 handed “thinking” AI to the public — Photo by Alex Knight on Unsplash

How They Differ From an Ordinary LLM

The biggest difference is how compute gets spent. An ordinary large language model (LLM) uses roughly the same amount of computation per token whether the question is trivial or fiendish. There is no structural room to pause and reconsider. The result is fast and consistent, but the limits show whenever an answer does not arrive in a single pass.

A reasoning model, by contrast, allocates its thinking — and its compute — according to difficulty. Easy questions get short answers; hard ones get long deliberation. In effect, a reasoning model trades away some speed and cost to buy accuracy on the problems that matter.

That trade-off contrasts neatly with the push to make models smaller. Where a small language model (SLM) maximizes efficiency to run on your phone, prioritizing cost and speed, a reasoning model does the opposite, pouring in extra computation to raise its intelligence. One camp aims for “light and everywhere,” the other for “heavier but more accurate.” The two answer the question of where and how to deploy AI in opposite ways.

a person's head with a circuit board in front of it
Ordinary models answer instantly; reasoning models vary compute by difficulty — Photo by Steve A Johnson on Unsplash
Key point: Ordinary LLMs fix compute per token; reasoning models vary it by difficulty. You buy smarter answers by paying in speed and cost.

Performance: What the Benchmarks Say

Reasoning models shine where answers are objectively right or wrong — math, coding, and science. The headline comparison is DeepSeek-R1 against OpenAI o1. By DeepSeek’s official technical report, R1 scored 79.8% on the AIME 2024 math competition benchmark, edging out o1-1217 at 79.2%, and posted 97.3% on MATH-500 versus o1’s 96.4%.

Coding was a dead heat — on Codeforces, R1 hit 96.3% to o1’s 96.6%, and on SWE-bench Verified, which tasks models with fixing real GitHub issues, R1 scored 49.2% against o1’s 48.9%. The performance gap between a top closed model and an open one had effectively vanished.

The lineage behind those numbers traces straight to chain of thought. The 2022 result — beating a fine-tuned GPT-3 on GSM8K with a handful of examples — became, a few years later, the move to bake reasoning into the model itself. The technical genealogy is unusually clean.

Not Free: Cost, Latency, and the Shadow of “Overthinking”

Reasoning models are no silver bullet. The first wall is cost. Because the invisible “thinking tokens” are all billable, the same question can cost far more to run on a reasoning model than on an ordinary one. The industry broadly agrees that the compute spent on inference — generating the answer — is growing faster than the compute spent on training.

Speed pays a price too. The longer a model thinks, the slower it responds, with p95 latency (the slowest 5% of responses) stretching past 30 seconds. For real-time chat or anything that needs an instant reaction, that delay is itself a liability.

The most paradoxical problem is overthinking. Apple’s machine-learning researchers reported that reasoning models fall into a “thinking trap,” burning thousands of tokens on something as trivial as adding 1 to 9,900 — and sometimes talking themselves out of the right answer. On low-complexity problems, ordinary models were often more accurate at no extra cost. More thinking does not always mean a better answer.

When Should You Use a Reasoning Model?

So the real question is when to reach for one. Reasoning models earn their keep on multi-step problems — competition math, coding, scientific reasoning, complex planning, debugging a large codebase — anything that does not yield to a single pass. For simple summaries, classification, and everyday chat, a faster and cheaper ordinary model is the better fit.

In practice, the right answer is usually to mix the two and route between them. Towards Data Science described one coding-assistant team that routed the 70% of traffic made up of simple tasks to an ordinary model, cutting daily cost from $3,000 to $970 — an annual saving north of $740,000, or about 68%. UC Berkeley research likewise found that allocating compute adaptively by difficulty improved efficiency more than fourfold over a fixed-budget approach.

The practical test is refreshingly simple. Ask whether a wrong answer is expensive, or whether speed and low cost matter more. When accuracy is the whole value — a code review, a mathematical proof — the extra cost of a reasoning model is easy to justify. When you need to process a flood of simple queries quickly, the ordinary model wins.

In short, a reasoning model is not “a smarter AI” so much as “an AI that thinks harder when it needs to.” Alongside alternative bets like the world models proposed by critics who argue LLMs alone fall short, reasoning models have become one of the most powerful current answers to the question of how to make AI think better. The next time a chatbot flashes “thinking,” you will know what those few seconds are buying.

Key point: Use reasoning models for hard reasoning, math, and code; use ordinary models for simple work. Routing between them captures accuracy and cost at once.

Frequently Asked Questions (FAQ)

Q1. What is the difference between a reasoning model and an ordinary LLM?

An ordinary LLM generates an answer immediately, spending roughly the same compute per token. A reasoning model first builds an intermediate chain of thought using invisible “thinking tokens,” and it allocates more compute as the problem gets harder. The payoff is higher accuracy on complex math and coding, at the cost of more money and more waiting.

Q2. What exactly is test-time compute?

It means spending extra computation during inference — when the model is answering your question — rather than during training. The idea grew out of the finding that letting a model “think” longer raises accuracy on hard problems. It is a new axis for improving performance, separate from simply making the model bigger.

Q3. Are reasoning models always more accurate?

No. Apple researchers documented a “thinking trap” in which reasoning models waste thousands of tokens on simple arithmetic and can talk themselves out of a correct answer. On low-complexity problems, ordinary models are frequently more accurate at no added cost. Thinking more does not guarantee a better result.

Q4. Why was DeepSeek-R1 such a big deal?

Because it shipped with open weights in January 2025, letting anyone download a high-end reasoning model that had previously been closed. It also drew attention for matching or slightly beating OpenAI o1 on major benchmarks such as AIME 2024 (79.8%) and MATH-500 (97.3%).

Q5. Are reasoning models more expensive to run?

Generally, yes. The invisible “thinking tokens” are all billable, so the same question can cost noticeably more. With the industry pointing to inference as a fast-growing share of compute, routing simple tasks to an ordinary model is an important cost-control move.

Q6. Which tasks should use a reasoning model?

They fit multi-step problems — competition math, coding, scientific reasoning, complex planning, and debugging large codebases — where the answer does not come in one pass. Simple summaries, classification, and casual chat run faster and cheaper on ordinary models. One team cut costs by 68% simply by routing simple work to an ordinary model.

💡 Related Posts
AI Agent vs Chatbot vs LLM: The 2026 Complete Guide

References

Stay Ahead of the Curve

Get the latest insights on AI, tech, and markets — delivered in both Korean and English.

Explore More Posts