What a router is supposed to fix
Send the easy work to something cheap and fast, the hard work to something strong, and stop paying frontier prices for a rename. That part is well understood.
Open source · AGPL-3.0 · .NET · Windows
Most routers pick.
This one learns.
They pick a model using a rule someone wrote down once, and never find out whether it was the right call. They’re as good on day ninety as they were on day one. That’s not stability, that’s a ceiling.
Arc Router is a local proxy. Point your existing coding agent at it, send
"model": "auto", and each task goes to the model that has actually earned
that kind of work. Then
the answer gets graded, and the result changes what happens next
time.
Pre-release v1.0.0 · Windows 10 1809+ · x64 · not code-signed yet
- 9,999
- In-distribution tasks
- 176
- Out-of-distribution tasks
- 8
- Backend models scored
> TotallyHotArcRouter --sync-benchmark-data
benchmark_id_results ..... 9,999 tasks x 8 models
benchmark_ood_results .... 176 tasks x 8 models
benchmark_models ......... 8 models, USD pricing
verified ................. ok
# routing a request
task ....... refactor: extract service layer
class ...... design · mid difficulty
vote ....... 4 of 5 voters, llm_router abstained
routed ..... backend-3 est. $0.004
score ...... blank until graded
01 The problem
Every task costs the same. Not every task is the same.
Coding agents send a stream of wildly uneven work to a single model. Renaming a variable and redesigning a module both arrive as “a request”, and both get billed at frontier prices.
What almost nobody does
Check afterwards. Most routers decide before an answer exists, never grade what came back, and so never learn that the cheap model got good at your codebase.
02 How it works
Three steps. Everyone does the first two.
Two of these you would expect from any router. The third is the one that makes the difference, and it happens after you already have your answer.
It sizes up the job
Every request is read for what kind of work it is, how hard it looks and what it’s written in — plus a fingerprint of what it means, so two descriptions of the same problem land in the same place even when they share no words.
That read is provisional, and treated that way. Nothing downstream takes it as settled.
It decides who should take it
The router weighs what it already knows — how models have scored on this kind of work, on jobs like this one, on your codebase specifically — and routes accordingly. Anything that doesn’t recognise the job is set aside rather than forced into a low-confidence answer.
This is where most routers stop. The decision is made, the request is served, and nothing that happens next will change what it does tomorrow.
It grades the answer, and the next one is better for it
You get your answer immediately; the grading happens afterwards, so nothing ever waits on it. Every response is scored on whether it was any good, never on what it cost. And it learns from the work you were already doing — so it gets better at what your team actually builds, rather than at a benchmark somebody else ran.
This is step three, and it is the entire product.
The same three steps, in detail
-
A job arrives
A job comes in, and the router sizes it up.
heuristic classifier → dimension · difficulty · language
-
It takes a fingerprint
It works out what the job means, so “fix this null crash” and “this throws NullReferenceException” land in the same place — even though they share almost no words.
task embedding, on a stopwatch — the request proceeds without one rather than making you wait
-
Five helpers vote
Five helpers each read the notebook a different way and vote on who should do the job. Any of them is allowed to say “I don’t know.”
dim_best · memory_kNN · cluster_best · logreg · llm_router → weighted sum, argmax. Abstention is a designed answer, not a failure
-
Sometimes it runs a trial
If you always pick your favourite, you never find out the cheap helper got good. So a fixed 1 in 20 jobs goes to someone else on purpose — and it records those odds, so the trial can’t skew what it learns.
ε-exploration; propensity (1−ε)+ε/K logged per request so the statistics can be de-biased
-
It writes a receipt
It writes down what happened straight away — but leaves the grade blank, because nobody has marked the work yet. You never wait for the marking.
transcript row written at serve time, score column null. Grading runs off the hot path
-
Later, it grades the answer
A parser and a judge grade the answer, their two opinions become one score, and the blank gets filled in. Then it checks whether it actually beat its old self.
two graders joined into exactly one score → RouterMemory + EmbeddingMemory → scored against a frozen baseline, leave-one-out
The router does not get to grade its own work.
That is the whole design, and it is why the numbers can be trusted: the router logs the odds it had of making each pick, and grades itself against a frozen baseline that was never shown the answer. Read the full walkthrough →
03 Why it's different
Most routers are a config file with ambition.
Two of the rows below are the ones that matter. They are the questions where the honest answer for most of the field is simply “no”.
In plain terms: most routers guess who should do a job and never find out whether they guessed right. Arc Router grades the answer, sometimes tries someone new on purpose, and keeps a report card against its old self — so “it got better” is something it can prove instead of something it says.
| Capability | Typical router today | Arc Router |
|---|---|---|
| Picks by 1 ,2 ,3 | A strategy you configure — weighted shuffle, lowest latency, lowest cost, least busy — or one learned classifier over the prompt. | Five independent voters hold a weighted election, and any of them may abstain. |
| Where the learning lives 4 ,5 | A hosted meta-model, trained centrally on evaluation data. | Your machine — a local SQLite tally of your own traffic. |
| Checks the answer? 5 | No. The decision is scored before an answer exists. | Yes. Every response is graded by a parser and a rubric judge, joined into exactly one score. |
| Tries new models? 5 | Exploits what it already believes. | About 1 in 20 requests explore on purpose — and it records the odds, so the statistics stay honest. |
| Can it prove it helped? 5 | A dashboard of spend. | Scored against a frozen baseline, computed leave-one-out. |
| Task categories 3 | Fixed, hand-written. | Re-clustered from your own history — spherical k-means, k swept 6 to 24. |
| Runs where 5 | A hosted API. | Your machine. AGPL-3.0, source published. |
- LiteLLM — Router load balancing strategies
- LiteLLM — Auto Routing (beta)
- Arc Router — the orchestrator ensemble
- Not Diamond — approaches to AI model routing
- Arc Router — how it learns
Vendor claims verified 2026-09-07 against the documentation linked above.
04 The evidence
It grades every answer, then checks the grade.
Two independent graders score every response, and neither one looks at price. Their opinions are joined into exactly one score before anything is written down.
Does it hold up when you read it?
Code in the response is parsed, never run. C# goes through Roslyn and JavaScript through Acornima — real parsers, real verdicts. Python and shell get a bracket-balance heuristic, which is flagged as non-authoritative and weighted at half, because a bracket count is not a compiler’s opinion.
Four analyzers look for the specific way a weaker model fails: placeholders, truncation, diagnostics, and runaway complexity. A stub parses perfectly — catching that is the most valuable check of the four.
Is it any good?
Parsing is not the same as being right, and perfect nonsense compiles fine. So the response also goes to a judge model with a rubric written for that task type.
It returns a digit from 1 to 5 — and then the interesting part: rather than trust the digit it happened to sample, the router reads the token probabilities and takes a probability-weighted mean. A judge torn between 3 and 4 scores 3.5, not a coin flip.
Where cost actually enters
Worth being precise, because it is a real design decision. The live vote is
cost-blind. dim_best, memory_kNN and
cluster_best all rank purely on quality score; none of them looks at price.
Cost appears in exactly two places, both of them accounting rather than selection: the
reward function used by the offline harness, and the live report card against the frozen
baseline. The router
chases quality, then measures whether that was worth the money.
Measured against published data
Arc Router evaluates against CodeRouterBench, a task-by-model benchmark published by Zhou et al. alongside arXiv:2606.22902. The dataset is their work, released on Hugging Face under the MIT license, and this project consumes it as an external dependency.
- benchmark_id_results
- 9,999 in-distribution tasks scored across 8 models (the probing and id_test splits combined).
- benchmark_ood_results
- 176 out-of-distribution tasks across the same 8 models, to test behaviour off the training distribution.
- benchmark_id_tasks / benchmark_ood_tasks
- Task metadata for both splits.
- benchmark_models
- Canonical backend model identities with USD pricing, so cost is compared on the same footing as quality.
05 In your critical path
Built for real routing, not a demo.
The .NET implementation ships the plumbing a router needs once it sits between your agents and your providers.
System proxy
Point an existing coding agent at the router and it routes what already flows through your tooling, with no client rewrite.
MCP endpoint
Router functions are exposed as MCP tools, including sync_benchmark_data, so agents can drive it directly.
Cost and token telemetry
Per-request token accounting reconciled against provider-reported usage, charted in the dashboard.
Honest response headers
Every response reports the model you asked for, the model that served it, and why they differ.
Secrets at rest
Provider keys are handled through a dedicated secrets-at-rest design rather than left in plain configuration.
Local benchmark database
Benchmark tables sync on demand into a local SQLite file. Nothing about your traffic leaves the machine to make a routing call.
06 What it doesn't know
Here is what we haven’t measured.
A system that grades itself should be candid about what it cannot yet prove.
- Nothing here proves correctness. The verifier can tell you code parses, is not a stub, and is not truncated. The judge can tell you one model’s opinion of it. Neither is a test suite, and for live traffic there is no ground truth to check against.
- Python and shell are graded on a bracket count. No in-process parser exists for them on .NET, so their syntax verdict is a heuristic. It is flagged and weighted at half rather than hidden, but it is still the weakest link.
- “The ensemble beats every single voter” is not yet measured. That half of the original exit criterion carries forward to the offline harness.
- The graders still cannot see the question. The prompt travels with the request as far as the grader, but nothing reads it yet — so a well-formed answer to a different question still scores as well as one that answers this one.
This list is maintained in the repository, not written for this page. Read it at source →
07 Get it
Windows x64. Pre-release. Open source.
Download
The 1.0.0 installer, what Windows will say about it, and how to verify the download.
Get Arc Router →Bootstrap
A short path from empty folder to a running router with benchmark data loaded.
Bootstrap guide →Tutorials
Guided walkthroughs of setup, proxy routing, telemetry, and the MCP endpoint.
Watch the walkthroughs →Documentation
The handbook plus every design document for the router, the GUI, and the data layer.
Read the docs →Standing on published work
The routing approach and the CodeRouterBench dataset are the work of Zhou et al. (arXiv:2606.22902). The quality-scoring judge implements G-Eval, by Liu et al. (arXiv:2303.16634). If you use either, cite their papers — the repository carries the BibTeX.