Reasoning Models vs Fast Models: When Should Developers Use Each?

The problem: you picked the wrong model and now your app is slow or dumb

You shipped a feature that calls an LLM. It works, but the response takes 40 seconds and costs you a fortune. Or you swapped to a smaller model and now the output is full of mistakes. The issue is not the model itself. It is that you used one model for every request, when you should have routed between a reasoning model and a fast model.

This article gives you a practical decision framework, a latency measurement script, and a routing config you can copy into your project today. You will learn when to pay for deep reasoning and when a fast model is the right engineering choice.

• Understand the tradeoff between reasoning and fast models
• Measure latency and cost in your own environment
• Build a simple router that picks the model per request
• Get a recommended setup for common developer tasks

What is a reasoning model? What is a fast model?

Reasoning models (like OpenAI o1 or o3, Anthropic Claude with extended thinking, or DeepSeek R1) spend extra tokens to think through the problem before answering. They are better at math, logic, and multi-step planning, but they take longer and cost more.

Fast models (like GPT-4o mini, Claude Haiku, or Llama 3.1 8B) respond in a fraction of a second. They are fine for classification, extraction, and straightforward generation, but they struggle with complex reasoning and are more likely to make errors when the task has many constraints.

curl -s https://api.openai.com/v1/models \
  -H "Authorization: Bearer $OPENAI_API_KEY" | jq '.data[].id'

• Reasoning models: o1, o3, DeepSeek R1, Claude with extended thinking
• Fast models: GPT-4o mini, Claude Haiku, Llama 3.1 8B
• The key difference is token generation: reasoning models emit hidden chain-of-thought tokens before the answer

The decision framework: five questions to ask

Before you write any code, decide which model to use for each request. Use this checklist. If you answer yes to any of the first three, choose a reasoning model. If you answer no to all five, a fast model is fine.

• Does the task require multi-step logic or math?
• Is a wrong answer expensive (financial, legal, medical)?
• Does the prompt have many constraints that must all be satisfied?
• Can the user wait 30+ seconds for a response?
• Is your budget enough to cover 10-20x the cost per request?

Step 1: Measure your actual latency and cost

Do not rely on vendor marketing. Run a quick script that calls both a reasoning model and a fast model with the same prompt, and print the latency and token usage. This gives you real numbers for your use case.

Save this script as measure_models.py and run it with your API key. The script uses the OpenAI SDK, but the same approach works for Anthropic or any provider.

import time
from openai import OpenAI

client = OpenAI()

prompt = """
A farmer has 17 sheep. All but 9 die. How many are left?
Think step by step.
"""

models = ["gpt-4o-mini", "o1-mini"]

for model in models:
    start = time.time()
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    elapsed = time.time() - start
    usage = response.usage
    print(f"{model}:")
    print(f"  Time: {elapsed:.2f}s")
    print(f"  Prompt tokens: {usage.prompt_tokens}")
    print(f"  Completion tokens: {usage.completion_tokens}")
    print(f"  Total tokens: {usage.total_tokens}")
    print(f"  Answer: {response.choices[0].message.content[:50]}")
    print()

• Run this with a representative prompt from your app, not a toy example
• Record the 90th percentile latency, not just the average
• Cost per request = (prompt tokens * input price) + (completion tokens * output price)

Step 2: Build a simple router

Once you have the numbers, create a router that sends each request to the right model. A simple rule-based router works for most apps. You can use keywords, a classifier, or a combination.

Here is a Python router that checks for math or complex instructions and sends those to a reasoning model, everything else to a fast model.

import re
from openai import OpenAI

client = OpenAI()

REASONING_MODEL = "o1-mini"
FAST_MODEL = "gpt-4o-mini"

MATH_PATTERN = re.compile(r"\b(calculate|compute|math|solve|how many|equation)\b", re.I)
COMPLEX_PATTERN = re.compile(r"\b(plan|analyze|compare|evaluate|multi-step|step-by-step)\b", re.I)

def route_prompt(prompt: str) -> str:
    if MATH_PATTERN.search(prompt) or COMPLEX_PATTERN.search(prompt):
        return REASONING_MODEL
    return FAST_MODEL

def ask(prompt: str):
    model = route_prompt(prompt)
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return model, response.choices[0].message.content

# Example usage
print(ask("What is 17*23?"))
print(ask("Write a haiku about the ocean."))

• Start with keyword rules; they are transparent and easy to debug
• If you need smarter routing, use a tiny classifier model (like a fast LLM) to decide the route
• Log the model used for each request so you can audit quality and cost

Step 3: Route with a fast model as a classifier

Keyword rules fail when the prompt is ambiguous. A better approach is to use a fast model to classify the request into 'reasoning' or 'fast' before calling the main model. The overhead is a few hundred milliseconds and a fraction of a cent.

Here is a function that uses gpt-4o-mini to decide the route, then calls the appropriate model.

from openai import OpenAI

client = OpenAI()

def route_with_classifier(prompt: str) -> str:
    classification = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a router. Reply with either 'reasoning' or 'fast'."},
            {"role": "user", "content": prompt}
        ],
        max_tokens=1,
    )
    label = classification.choices[0].message.content.strip().lower()
    return "o1-mini" if label == "reasoning" else "gpt-4o-mini"

def ask(prompt: str):
    model = route_with_classifier(prompt)
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return model, response.choices[0].message.content

# Test it
print(ask("Explain quantum entanglement in simple terms."))
print(ask("What is the derivative of x^2?"))

• Use max_tokens=1 to force a single-token output, minimizing cost
• Cache the classification result for identical prompts
• Fall back to the fast model if the classifier returns an unexpected label

Step 4: Set up a timeout and fallback

Even with routing, a reasoning model can hang or take too long. Always set a timeout and a fallback to a fast model. This keeps your app responsive and prevents user frustration.

Here is a wrapper that uses asyncio to enforce a timeout and fall back to the fast model if the reasoning model exceeds the limit.

import asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI()

async def call_with_timeout(prompt: str, model: str, timeout: float = 10.0):
    try:
        response = await asyncio.wait_for(
            client.chat.completions.create(
                model=model,
                messages=[{"role": "user", "content": prompt}],
            ),
            timeout=timeout,
        )
        return response.choices[0].message.content
    except asyncio.TimeoutError:
        return None

async def ask_with_fallback(prompt: str):
    reasoning = await call_with_timeout(prompt, "o1-mini", timeout=10.0)
    if reasoning:
        return "o1-mini", reasoning
    fast = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
    )
    return "gpt-4o-mini", fast.choices[0].message.content

# Run it
async def main():
    model, answer = await ask_with_fallback("Solve this: 2x + 5 = 15")
    print(f"Model: {model}")
    print(answer)

asyncio.run(main())

• Choose a timeout that is 2-3x the average latency of the reasoning model
• Log every timeout so you can tune the threshold
• Consider using a queue if you have high concurrency

Recommended setup for a typical developer tool

For most developer tools, I recommend this default: use a fast model for 80-90% of requests, and a reasoning model only for explicit user requests or when the task is clearly complex. This keeps costs low and latency acceptable.

Here is a copy-paste starter config for an environment file that sets model IDs and timeouts.

# .env
FAST_MODEL=gpt-4o-mini
REASONING_MODEL=o1-mini
ROUTER_MODEL=gpt-4o-mini
REASONING_TIMEOUT=15
FAST_TIMEOUT=5

• Set FAST_MODEL to a cheap, low-latency model like gpt-4o-mini or claude-3-5-haiku
• Set REASONING_MODEL to o1-mini or a similar reasoning model for hard tasks
• Use ROUTER_MODEL for the classifier if you need smart routing
• Start with REASONING_TIMEOUT=15 and FAST_TIMEOUT=5, then adjust based on your measurements

FAQ

Answers to the questions that come up most often on this topic.

• Q: Can I use a reasoning model for every request? A: Technically yes, but it will be slow and expensive. Most requests do not need deep reasoning, so you waste time and money.
• Q: How much slower is a reasoning model in practice? A: It depends on the task, but typically 3-10x slower. Run the measurement script in Step 1 to get real numbers for your prompts.
• Q: What about open-source models? A: You can apply the same routing logic with models like Llama 3.1 8B for fast and DeepSeek R1 for reasoning, using a local server like Ollama or vLLM.
• Q: How do I know if my router is working? A: Log the model used and the latency for each request. Monitor the error rate and user feedback to catch regressions.
• Q: Is there a risk that the classifier router makes mistakes? A: Yes, but you can mitigate it by setting a confidence threshold and falling back to the reasoning model when uncertain.

Next action: run the measurement script

Do not guess which model to use. Run the measurement script from Step 1 with a few real prompts from your app. Compare the latency, cost, and output quality, then set up the router that fits your numbers.

Create a file called measure_models.py, paste the script, and run it with your API key. You will have the data you need to make the right call.

python measure_models.py