> The Cheap Model Does the Reading. The Expensive Model Does the Thinking.

The Wake-Up Call: When AI Bills Become Cloud Bills

For the first two years of the AI boom, the playbook was embarrassingly simple: pick the best model you could afford, send it everything, route the invoice to accounting. That era is ending fast.

Uber’s experience illustrates the problem. After expanding access to AI coding tools across its engineering organization, the company reportedly exhausted its annual budget for those tools within four months. Adoption rose from 32% of engineers in February to 84% in March, and Uber later introduced a $1,500 monthly limit per employee, per agentic coding tool, with exceptions requiring approval.

Meta faced a similar incentive problem. Reporting on an internal employee dashboard found that staff consumed more than 60 trillion AI tokens during one 30-day period. Later reports placed usage at 73.7 trillion tokens before the leaderboard was removed. These figures illustrate how quickly internal AI consumption can scale, although token usage alone does not demonstrate productivity or business value.

The pattern is increasingly common: as soon as agents start making requests instead of humans, token spend stops looking like a SaaS line item and starts looking like a cloud compute bill that nobody sized correctly.

The Fix Isn't "Use Worse AI." It's Routing.

The solution converging across engineering orgs right now isn't abandoning frontier models. It's building a routing layer that decides, per task, whether frontier intelligence is actually needed.

Example: DoorDash’s Staged Code-Review Pipeline

DoorDash's AI research team built an internal benchmark called DashBench by curating 1,000 historical pull requests down to 105 stress-test cases:

• Tricky diffs that trick reviewers
• Noisy review histories
• Clean PRs (to catch false positives)
• PRs later reverted (to catch real regressions)

They tested staged scout-and-reviewer combinations:

1. Scout model analyzes the pull request broadly and flags suspicious areas
2. Reviewer model investigates those flags, confirms whether each concern holds up, and drops claims that don't survive scrutiny

The strongest result overall: a Kimi K2.6 scout paired with a Claude Fable 5 reviewer — 65.2% weighted recall and 75.3% weighted F1, at $3.81 per pull request.

The more revealing comparison, though, is what that setup replaced. DoorDash's own previous production harness — Claude Sonnet 4.6 as scout, Claude Opus 4.8 as reviewer, both frontier models — cost $3.91 per PR (essentially the same price) but only reached 53.6% weighted recall and 66.3% weighted F1. Swapping in the open-weight scout didn't just cut cost against a cheap baseline; at flat cost, it moved recall from 53.6% to 65.2% and F1 from 66.3% to 75.3%. That's the comparison co-founder Andy Fang was pointing to when he called it "Better quality, cheaper cost."

The single-pass baselines (no scout at all) were cheaper still — $0.75/PR for GPT-5.5 and $0.65/PR for Claude Opus — but the coverage gap was substantial, not marginal: those baselines found 164 and 115 real issues respectively, versus 537 for the winning staged setup, roughly a 2-3x difference in what got caught. DoorDash's own conclusion: no single configuration dominated every axis. Staging bought coverage at a cost, and the Kimi/Fable pairing was the one staged combination that improved on DoorDash's existing staged setup for free.

The Math: Why Routing Works at Scale

Using API prices checked in July 2026:

DeepSeek V4 Pro: $0.435 per million cache-miss input tokens and $0.87 per million output tokens. Claude Opus 4.8: $5 per million input tokens and $25 per million output tokens.

A fair comparison must separate input and output tokens because providers price them differently. Consider a team processing 100 million tokens per month, consisting of 80 million input tokens and 20 million output tokens. Assume that the routed workloads have the same input-output ratio and that prompt caching, batch discounts, tool charges, and infrastructure expenses are excluded.

All-Claude Opus approach

Input: 80 million × $5 = $400 Output: 20 million × $25 = $500 Total: $900 per month

70/30 routing split

If 70% of traffic goes to DeepSeek V4 Pro and 30% goes to Claude Opus 4.8:

Estimated total: approximately $306.54 per month Estimated reduction: approximately 65.9%

80/20 routing split

If 80% of traffic goes to DeepSeek V4 Pro and 20% goes to Claude Opus 4.8:

Estimated total: approximately $221.76 per month Estimated reduction: approximately 75.4% Estimated annual saving compared with the all-Opus approach: approximately $8,139

These figures are illustrations, not production guarantees. Actual savings depend on the input-output ratio, prompt caching, model selection, escalation rate, retries, tool calls, and differences in tokenization.

The Cost-Aware Model Routing Architecture

Here's what the two-stage pattern looks like in practice:

The Two Traps: Why Routing Fails in the Wild

Routing isn't free, and it isn't a set and forget config. Two real failure modes cost companies more than they save.

Trap #1: The Router Can Quietly Betray You

The cautionary tale: An engineer built a cascade where a cheap model's output got schema-checked, with failures escalating to the frontier model. A minor provider side formatting change broke the schema check so the router silently started escalating 90% of "easy" traffic to the most expensive model.

Nothing errored. Nothing alerted. The bill tripled.

The lesson: A routing layer needs the same monitoring rigor as the spend it's supposed to control. You need:

• Real-time routing decision logs
• Escalation rate monitoring
• Cost attribution per routing decision
• Alerts for unexpected pattern shifts

Trap #2: Benchmarks Decay, and Generic Ones Lie

DoorDash's writeup is explicit, everyday signals—acceptance rate, thumbs-up feedback, aggregate scores can make a reviewer look useful while hiding exactly where it fails.

A benchmark built on someone else's tasks tells you how a model performs on someone else's tasks.

Companies getting real value are re-testing against their own traffic, not trusting leaderboards. That means:

• Curating your own eval set (the way DoorDash did with 105 real-world cases)
• Testing regularly against your specific codebase, support tickets, or data
• Measuring what matters to your users, not generic benchmarks

Building the Moat: Benchmark > Router

Anyone can wire up an if/else that sends "easy" tasks to a cheap model. The hard, valuable part is knowing what's actually easy for your business.

Evidence	Evaluation asset	Observed result
DoorDash	A custom 105-case pull-request benchmark	Staging increased weighted recall and coverage, while single-pass baselines remained cheaper and faster.
RouteLLM	MT-Bench, MMLU, and GSM8K evaluations	Savings varied significantly by benchmark, ranging from approximately 35% to more than 85%, while targeting 95% of GPT-4 performance.

The benchmark is the competitive advantage. The router is the delivery mechanism.

Who Benefits Most (And When)

Routing generally becomes more valuable as request volume, model-price differences, and task repetition increase:

• At very high volume, even a small improvement in model allocation can create meaningful savings.
• At moderate volume, routing is most useful when a few repeatable tasks account for a large share of total AI spending.
• Growing teams can use model gateways to test routing without immediately building a complete internal platform.
• At low volume, evaluation and routing complexity may cost more than the inference savings they produce.

The break-even point depends on current spending, evaluation costs, error costs, workload stability, and the difference in price between the candidate models.

A company running 100 AI calls a day doesn't have a bill problem worth solving with an internal benchmark team. This is a scale story.

A Note on Open-Weight Models and Data Sovereignty

Important distinction: Self-hosted open-weight models genuinely keep data in-house. But calling an open-weight model through someone else's hosted API does not—that's still a third party seeing your traffic.

Be precise: "open-weight" ≠ "on my infrastructure."

If data sovereignty is the goal, you need:

1. Self-hosted open-weight model (on your infra)
2. Private routing layer (no third-party visibility)
3. Monitoring that doesn't leak request data

If you just need cost reduction, routing to hosted open-weight via an API gateway is often good enough and much easier to operate.

What Routing Actually Saves You

Beyond the 60-80% cost reduction, the real wins are subtler.

1. Latency flexibility

• Cheap models are often faster
• For 70% of traffic (the easy stuff), you can shave milliseconds
• Complex requests still get the frontier model's accuracy

2. Rate limit resilience

• Frontier model rate limits become a bottleneck, not a ceiling
• Route cheap tasks to an always available model
• Preserve frontier capacity for what actually needs it

3. Compliance and privacy

• Route sensitive data to self-hosted or private models
• Route routine tasks to faster, cheaper shared infrastructure
• Fine-grained control over where different data goes

4. Multi-model ecosystem

• Cheap models are often better at specific domains (code, translation, structured output)
• Frontier models are generalists
• Route to the specialist for each job type

The Likely Shape of 2027

This is a snapshot of mid-2026, not a law of nature. The gap between open and closed models has been narrow and roughly stable. It could re-widen if a frontier lab pulls ahead sharply, or narrow further if open releases keep pace.

The durable bet isn't "open-weight models win" or "frontier models win." It's that routing as a discipline is now table stakes the same way caching or CDN strategy became table stakes once web traffic got expensive enough to matter.

The companies that figure out routing first are going to run circles on cost and capability around the ones still routing every request through their most expensive model.

Where to Start

If you're building or evaluating a routing strategy:

1. Inventory your traffic. What tasks are you actually running? Where do they cluster?
2. Pick one high-volume task. Pick code review, customer support triage, or data extraction—something that's 30%+ of your AI spend.
3. Build a micro-benchmark. Pick 50-100 real examples. Test cheap vs. frontier. Measure what matters to your users.
4. Wire up the router. Use an existing gateway (OpenRouter, LiteLLM, Portkey) or build a thin layer.
5. Monitor obsessively. Log every routing decision. Alert on unexpected escalation rates.
6. Iterate. Re-test your benchmark quarterly. Pricing and model quality change.