LLMRouter

Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

Tao Feng^1,*  ·  Fangxu Yu^2,*  ·  Haozhen Zhang^3,*  ·  Zhongjie Dai¹  ·  Liangqi Yuan⁴  ·  Zijie Lei¹  ·  Weizhi Zhang⁵  ·  Kunlun Zhu¹  ·  Haodong Yue¹  ·  Keyang Xuan¹  ·  Ge Liu¹  ·  Jiaxuan You¹

^* Equal contribution

¹ University of Illinois Urbana-Champaign ² University of Maryland, College Park ³ Nanyang Technological University ⁴ Purdue University ⁵ University of Illinois Chicago

▤ Paper ⌘ Code 🤗 Hugging Face Blog ▤ Leaderboard Tutorials

Overview

No single large language model is optimal across all queries and budget constraints. LLMRouter is an open-source foundation for selecting the right model for each request, making routing methods easier to develop, compare, and deploy under a shared quality–cost objective.

It unifies routers that were previously implemented in incompatible stacks—from simple quality predictors and cost-aware cascades to graph-based, multi-turn, and personalized policies—and provides the shared infrastructure needed to evaluate them fairly.

LLMRouter joins data construction, router training, inference, and evaluation in one reusable workflow.

A Unified Routing Formulation

Every router is represented as a sequential decision process. A shared formulation turns different routing ideas into comparable design choices.

01

Describe the state

Represent the query, user context, interaction history, and candidate-model information.

02

Make a routing decision

Score candidate compatibility, then dispatch, escalate, or stop under the operating budget.

03

Learn and evaluate

Optimize task quality and inference cost with a common protocol across routing settings.

xRouteBench

LLMRouter constructs routing supervision by running a candidate pool across benchmarks, scoring each response with its task metric, and recording token-level cost. Every router then faces the same queries, models, metrics, and quality–cost protocol.

The resulting benchmark spans generic LLM tasks, memory-augmented reasoning, image and video understanding, time-series, and personalized routing. This makes it possible to compare both performance and cost instead of optimizing one in isolation.

Tracks: Generic LLM tasks · Memory · Vision & video · Time-series · Personalization

Explore xRouteBench results →

Key Findings

Learned routing improves over fixed-model baselines. The empirical study finds a 14.6% relative improvement over the strongest fixed-model baseline.

No router dominates every deployment. Rankings change across tasks and reverse as cost constraints become tighter.

User context changes the right answer. Personalized routing gains from preference and interaction history when it is available.

BibTeX

@article{feng2026llmrouter,
  title={LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers},
  author={Feng, Tao and Yu, Fangxu and Zhang, Haozhen and Dai, Zhongjie and Yuan, Liangqi and Lei, Zijie and Zhang, Weizhi and Zhu, Kunlun and Yue, Haodong and Xuan, Keyang and others},
  journal={arXiv preprint arXiv:2608.06867},
  year={2026}
}