Claude tier bake-off: which requests actually need Opus?

Switchyard can send each type of request to a different Claude model: a quick question to Haiku, a hard analysis to Opus. That only saves money if the map from request type to model is right, and the first map was written by hand. So we measured it: every request type, every model, the same problems, and the cheapest model that clears the bar wins the type. The costs on this page are metered token costs, the way an API key is billed. A cheaper model does not lower the monthly fee of a Pro or Max plan. It can lower the paid extra usage those plans offer beyond their limits, which is billed at API rates.

A test tenant, not a customer result

acme-support is our own test tenant, and “Acme Outfitters” is a fictional company. This run proves the method (measure each request type, then route) on realistic support work. It does not set any real client's quality bar, and none of these figures is a customer's.

Run October 13, 2026 · 360 graded answers · judge claude-opus-5-5 · total run cost $3.09

The hand-written map

Wrong in both directions

It sent order status to Haiku 5.5, which passed 12 of 15. It paid for a bigger model on refund request (Sonnet 5.5) and deep analysis (Opus 5.5), where a cheaper one cleared the bar.

The measured map

$1.07 per 1,000 requests

80% below Opus everywhere and 55% below the hand-written map, with every type at or above the bar.

The map running today

$2.28 per 1,000 requests

Only 5% below the hand-written map, and 12% above Sonnet everywhere, because deep analysis is held on Opus until harder problems are measured.

Pass rate by request type

Each cell is 15 graded answers (5 problems, 3 runs each). An answer passes at 4 or more out of 5. A model clears a type when at least 95% of its answers pass; the cheapest model that clears it wins the type.

Request type	Haiku 5.5	Sonnet 5.5	Opus 5.5	Cheapest that clears	Hand map	Live map
Quick questionquick-question	15/15100%	15/15100%	15/15100%	Haiku 5.5	Haiku 5.5	Haiku 5.5
Product FAQproduct-faq	15/15100%	15/15100%	15/15100%	Haiku 5.5	Haiku 5.5	Haiku 5.5
Refund requestrefund-request	15/15100%	15/15100%	15/15100%	Haiku 5.5	Sonnet 5.5 · overpaid	Haiku 5.5
Deep analysisdeep-analysis	15/15100%	15/15100%	15/15100%	Haiku 5.5	Opus 5.5 · overpaid	Opus 5.5 · held
Order statusorder-status	12/1580%	15/15100%	15/15100%	Sonnet 5.5	Haiku 5.5 · below bar	Sonnet 5.5
Summarize a ticketsummarize	12/1580%	15/15100%	15/15100%	Sonnet 5.5	Sonnet 5.5	Sonnet 5.5
Writingwriting	14/1593.3%	15/15100%	15/15100%	Sonnet 5.5	Sonnet 5.5	Sonnet 5.5
Codecode	14/1593.3%	15/15100%	15/15100%	Sonnet 5.5	Sonnet 5.5	Sonnet 5.5

Average score out of 5 across all 120 answers each: Haiku 5.5 4.59 · Sonnet 5.5 4.74 · Opus 5.5 4.89.

What each map costs

Answer cost per 1,000 requests, with the 8 types equally mixed. The judge's cost is not included: it is the cost of measuring, not of serving. Per type, Opus cost 33× to 195× what Haiku did.

Map	Per 1,000 requests	Quality
Opus everywhere	$5.26	Clears the bar on every type.
Hand-written map	$2.40	Below the bar on 1 type.
Live map (today)	$2.28	The measured map, with 1 type held on Opus. Clears the bar on every type.
Sonnet everywhere	$2.03	Clears the bar on every type.
Measured map	$1.07	Clears the bar on every type.
Haiku everywhere	$0.10	Below the bar on 4 types. Shown for scale, not as an option.

Why the live map saves so little: deep analysis on Opus costs $9.85 per 1,000 requests of that type, more than the other 7 types' picks combined ($8.41). Most of the measured saving is waiting on that one decision.

Where Haiku missed

Haiku missed 8 answers, on 5 of the 40 problems. Sonnet and Opus missed none. The reasons below are the judge's, summarized. Most are plain errors: a wrong delivery date, a broken word limit. One cell rests on a single problem: every “summarize a ticket” miss was on acme-summarize-2, made the same way in each run.

acme-code-5· Code· missed 1 of 3 runs
The final values were right, but it first showed wrong values in bold before correcting itself.
acme-order-status-1· Order status· missed 1 of 3 runs
Miscounted business days across a weekend, giving a delivery window of Oct 12–14 instead of Oct 13–15, and added prose where JSON only was asked for.
acme-order-status-4· Order status· missed 2 of 3 runs
The content was right, but the reply ran past the 80-word limit (the judge counted about 90 to 103 words).
acme-summarize-2· Summarize a ticket· missed 3 of 3 runs
Listed the agent’s instruction (a 4-hour charge and a 3-second hold) as something the customer had already done. The customer reported charging overnight and holding the button, without confirming the hold duration.
acme-writing-5· Writing· missed 1 of 3 runs
About 64 words against a 60-word limit, plus a “recharges quickly” claim the product specs do not support.

What this does not prove

• A test tenant. The tenant and the company are ours and fictional. This proves the method, not a real client's bar.
• Small cells. 15 graded answers per cell, so one miss is 93.3%, already below the 95% bar. Several cells were decided by a single problem.
• Thinking was not equal. Haiku and Sonnet ran with thinking off. Opus 5.5 could not turn thinking off, so it ran with adaptive thinking at low effort and extra room for output.
• Deep analysis was well-defined. Its problems are calculations with one right answer. Haiku passing them does not show it can handle open-ended analysis, which is why the live map keeps that type on Opus.
• One judge. Claude Opus 5.5 graded every answer, including its own. The problems went through four rounds of adversarial review before the run, and each was graded against a written expected answer.

How this was measured

• 40 hand-written support problems: 5 for each of 8 request types, with ground truths checked by script.
• Each model answered every problem 3 times: 360 answers in all.
• claude-opus-5-5 graded each answer from 1 to 5 against the problem's written expected answer. 4 or more passes.
• Costs are each answer's tokens at list price. Answers cost $0.89 and grading $2.20. 0 errors, 0 ungraded.
• The cost table assumes the 8 types arrive equally often. A real mix will move the totals, not the per-type picks.
• Run id run_1791907256408. This is a manual, reproducible run, repeated when the models or the problems change.

Next to measure

Open-ended deep analysis problems. If Haiku or Sonnet clears the bar on those too, the live map can drop Opus and take most of the measured saving. If not, the hold was right, and this page will say so.

Other bake-offs

The judge bake-off asks whether a cheaper model can grade as well as a premier one, and the router comparison sets our measured routing against OpenRouter's automatic one. See them all, with what each found.

All bake-offs Switchyard

Measured bake-off