Measured routing vs. spend-share routing

Spend-share routing answers “what is everyone else using for tasks shaped like this?” For a chatbot, that means popularity decides — here, turns routed to gemini-2.5-flash, deepseek-v4-pro, and deepseek-v4-flash. Measured routing knows, from a benchmark on the client's own tasks, which model already passes — so it routes there and keeps the difference. Quality holds because the cheaper model was measured to clear the bar first. Popularity can pick among options; only measurement tells you which options are good.

How big the cost gap is depends on the bot. On dlr it nearly vanishes — auto-beta happened to pick cheap models that also worked, and the two arms cost within 3% of each other. The consistent differences are quality (we win 8 divergent turns to 1 across 65) and latency (we answer 3–4× faster). Cost is the headline only when popularity happens to favour an expensive model.

Method: each chatbot's own turns, run through both routers, both answers graded against the known-correct answer by the same reference judge. The rival arm is openrouter/auto-beta, OpenRouter's task-aware router — a different model id from the older openrouter/auto, which routes on a different basis and gives a much larger gap. A small, dated sample — directional, not a guarantee. Generated 2026-07-28.