The cheapest model that's good enough, for every request.
Merit Order is a self-hosted LLM gateway. Your tools keep their own API. For each request it picks the cheapest model and backend that meets the work's quality bar, and records what it cost.
- level
- from the task's wording; tools only say it's coding
- chance
- each model's, from published benchmarks
- pick
- lowest expected cost to finish the task
- auto, expected total cost 97.7% finished$0.40
- auto, threshold 97.5% finished$1.01
- Claude Sonnet 5.5, always 97.4% finished$0.73
- Claude Opus 5.5, always 97.7% finished$1.43
The idea
Line up the options by cost. Skip what isn't good enough. Take the first.
Power grids dispatch generators in merit order: cheapest first, expensive peakers only when needed. Merit Order does the same with models. Your own GPUs, when steady load pays for them, are the cheap baseload. Provider APIs are always there for everything else.
The merit order for coding level 4 · input price per million tokens, each model's cheapest route
- DeepSeek V4.1 Flash coding 2.4, below level 4$0.02
- Claude Haiku 5.5 coding 3.0, below level 4$0.10
- GPT-6 Luna coding 2.3, below level 4$0.10
- Kimi K3.1 Pro coding 2.5, below level 4$0.24
- GPT-6.1 Terra coding 4.2$1.00
- Gemini 4 Carbon coding 4.4$2.00
- Claude Sonnet 5.5 coding 3.9, below level 4$2.00
- Claude Opus 5.5 coding 4.1$4.00
- Claude Fable 5.2 coding 5.0$8.00
How it saves money
Six mechanisms, each with a worked example.
Routing's numbers come from the routing simulator; the others are list-price arithmetic or illustrations on the prices of 12 November 2026, and each says which. Every mechanism in detail.
Routing by expected total cost
routing simulatorEvery model has a chance of finishing each level of work, from published benchmarks. auto picks the model with the lowest expected cost to finish the task: the try, plus the chance it fails times a person's time and the next attempt. Priced for the rest of the session, including rebuilding the prompt cache elsewhere.
2,000 simulated tasks · 97.7% against 97.4% finished
Sonnet 5.5 for everything → auto, per finished task
Prompt caching
list-price arithmeticThe gateway adds Claude's cache markers, sets OpenAI's prompt_cache_key, and never changes the early part of the prompt. On Claude it asks for a 1-hour cache from a session's second turn, for $0.0030 more per turn.
100,000-token conversation · person steps away 20 minutes
write again → read from cache
Sticky sessions
list-price arithmeticA session's cache lives on one provider. It moves only if that saves at least 10% plus $0.01 over the rest of the session, and within a task it keeps its model outright.
80,000-token Claude Opus 5.5 session, past its first turn
rebuild on a cold backend → a turn where it is
Tool-output compression
illustration · opt-inDeterministic rules strip terminal codes, collapse repeated lines, and keep the head and tail of anything over 24,000 characters. Same bytes every turn, so the cache stays valid. About $0.10 less on Sonnet 5.5 the turn it arrives.
A 100,000-character test log
roughly 25,000 fewer tokens
Batch prices
illustrationOpenAI's Batch API and Anthropic's Message Batches are routed like live requests, among providers with a batch API and running GPUs with room. Ten providers' batch APIs are supported.
10,000 summaries · 5,000 tokens in, 500 out · Sonnet 5.5
live → batch price
Your GPUs at break-even
shipped configurationA replica starts when the work it could serve would cost more than its hourly price plus 20% at API prices for 10 minutes, and drains when demand falls below break-even. While it runs, it is priced at 2% of the API price up to 60% load.
Qwen3 Coder Next on four H100s, $16 an hour
GPU price → API-priced demand that starts it
The seven causes are computed in a fixed order, each against the result of the previous one, so they add up to the total. Prompt caching the harness set up itself isn't counted as a Merit Order saving.
How it routes
A level for the work. A score for each model.
Send model: "auto" and Merit Order chooses. Name a model and you always get it; Merit Order only picks the cheapest place to run it. Follow one request through.
Levels from the task's wording
Tools say the work is coding; the task's wording sets the level, with no model call. Claude Code and Codex send dozens of tools on every request, so a tool count can't tell a typo from a redesign. Length never raises a level. The word lists are settings, and every request records why it got its level.
Expected total cost, the default
A failed try is still paid for, and a person has to notice it and ask again. So auto compares try + (1 − chance) × (person's time + next attempt) for every model that may serve the task. Cost per success and the old threshold remain as settings.
Learns from your own traffic
Outcome signals, such as a resent request, a raised floor, failing tests or a fix loop, train a router that corrects the rules' level. It proves itself in shadow mode first. Opt-in exploration tries about 4% of new tasks one level up or down.
Model scores come from Epoch AI's benchmark data by default, refreshed daily, against a fixed bar. Run meritorder eval to score models on your own tasks instead.
What you'd pay vs. what you pay
Ten developers on Claude Code and Codex, simulated.
A month of model spend for 10 developers, 8 tasks a day each, 21 working days: 1,680 coding tasks, scaled from the routing simulator's 2,000-task coding-only run. Every model call is mocked; this is not yet measured on live traffic. The full results, or scale them to your team.
The mark is the simulated month; the band is its 90% interval (a paired bootstrap over the tasks). The total counts failed attempts too: it adds $15 of someone's time for each one (noticing it, asking again, waiting, cleaning up).
What the numbers say
Against Claude Opus 5.5 for everything: $773 against $2,947 of model spend, 74% less, with 98.1% of tasks finished against 98.3%.
Against Claude Sonnet 5.5 for everything (or a fixed-model proxy set to it, which sends the same tokens): $773 against $1,518, 49% less, at 98.1% against 97.9%. In total, counting failed attempts: $4,830 against $5,752.
Where it doesn't win. On coding-only work, cost per success spends less on models ($759) and comes out about even once failed attempts are counted: $3.00 against $2.93 per finished task. The opt-in escalation inside a task finished slightly more (98.3%) but cost more ($980). The oracle, which knows in advance which cheapest model would succeed, needs $273: no router gets there, and the gap is how much better routing could still do.
A hosted auto-router isn't simulated: its decision model isn't public, so we can't say what it would pick, and its platform fee would come on top.
| Assumption | Value |
|---|---|
| Developers, tasks a day, working days | 10, 8, 21 |
| Clientswho sends the coding tasks | Claude Code 75%, a small agent such as Codex 15%, chat with code 10% |
| Task difficultySWE-bench Verified's time-to-fix mix | easy 38%, medium 53%, hard 8%, very-hard 1% |
| Outcomesno model is called | drawn from each model's published benchmark ability against each task's difficulty |
| Attempts per task | up to 3 |
| Later turns after a 5 to 60 minute pause | 10% |
| Price per failed attemptsomeone's time; adjustable in the calculator | $15 |
| Prices and scores | the main repo's price list of 12 November 2026 |
From scripts/routing_sim.py in the main repository, run on 16 November 2026. Not counted: compression, batch, the response cache and GPUs, and the cost of running the gateway. Claude Code on non-Claude models works through translation, but Anthropic doesn't support it, so expect rough edges.
Direct, one model
Nothing to run. Claude Code on Claude is what Anthropic supports. Predictable quality, if the model is strong enough for your hardest work.
Fixed-model proxy
One place for keys, budgets and logs. LiteLLM covers over 100 providers and is used in production by many organizations.
Hosted auto-router
Nothing to deploy, very large scale, hundreds of models. OpenRouter's Auto Router weighs the cost of losing a prompt cache and picks reasoning effort per request.
Merit Order
Runs in your network on your provider accounts. Budgets checked before a request is sent. Savings split by cause and checked against bills. Your own GPUs at break-even.
Works with
Point your tools at it. Nothing else changes.
Merit Order speaks OpenAI Chat Completions, OpenAI Responses (HTTP and WebSocket) and Anthropic Messages. When a tool and the chosen model speak different formats, it translates requests, replies and streams both ways.
meritorder setup claude-codemeritorder setup codexmeritorder setup pibase_url=<gateway>/v1ANTHROPIC_BASE_URLControl and trust
Your network, your keys, your records.
Merit Order is software you run, not a service you send prompts to.
Self-hosted
Docker Compose or a Helm chart. Your keys, your network, your Postgres and Redis. Signed multi-arch images when a version is tagged.
AGPL-3.0, every line
Read, change and run all of it. If you change it and offer it to others over a network, you share your changes under the same license.
No content stored by default
The ledger and session table hold metadata: hashes, timestamps, token counts, costs. The response cache, stored Responses and batch files are opt-in or explicit, and sealed.
Every decision recorded
Each request keeps its routing reason, for example coding: 32 tools; simple task wording (write (unit )?tests): level 2, and the chances and costs it compared.
Pre-production: version 0.0.1
Most of Merit Order is tested only against mocks of each provider's API. Checked live so far: OpenRouter, Anthropic and OpenAI, Claude Code sessions through the gateway, the Docker Compose stack, and GPUs on a local kind cluster with a stand-in vLLM.
Not yet checked live: the other providers, real GPUs, a real Codex CLI session, Entra and Google sign-in, batch APIs and bills. Set an organization budget before you send it real traffic. What's verified.
Quickstart
No keys needed to try it.
Without provider keys, requests go to built-in mock backends, and the reply says [mock-small] You said: hello.
- Needs Python 3.12+ and uv, or Docker.
- Set
MERITORDER_DEV_KEYin.env, plus any provider keys you have. - Run
meritorder checkto see which backends and models are live.
git clone https://github.com/Merit-Order/meritorder && cd meritorder
uv sync
cp .env.example .env # set MERITORDER_DEV_KEY
uv run meritorder check # which backends are live
uv run meritorder serve # http://localhost:8080
# in a second terminal
curl localhost:8080/v1/chat/completions \
-H "authorization: Bearer $MERITORDER_DEV_KEY" \
-H "content-type: application/json" \
-d '{"model": "auto", "messages": [{"role": "user", "content": "hello"}]}'cp .env.example .env # provider keys optional
docker compose up -d # gateway, worker, Postgres, Redis
docker compose exec gateway meritorder keys create \
--user you@example.com --role admin
# then open http://localhost:8080/ui/ and sign in with that key
meritorder setup claude-code