Arch-Router：将大语言模型路由与人类偏好对齐

摘要

随着大型语言模型（LLMs）的迅速普及——每种模型都针对不同的优势、风格或延迟/成本特性进行了优化——路由技术已成为有效利用不同模型的关键手段。然而，现有的LLM路由方法存在两个主要局限：它们通过基准测试评估性能，而这些基准往往无法捕捉由主观评价标准驱动的人类偏好；并且它们通常仅从有限的模型池中进行选择。在本研究中，我们提出了一种偏好对齐的路由框架，该框架通过将查询与用户定义的领域（如旅游）或操作类型（如图像编辑）相匹配来指导模型选择，为在路由决策中编码偏好提供了一种实用机制。具体而言，我们引入了Arch-Router，这是一个紧凑的1.5B模型，它学习将查询映射到领域-操作偏好，以支持模型路由决策。我们的方法还支持无缝添加新模型进行路由，而无需重新训练或修改架构。在对话数据集上的实验表明，我们的方法在将查询与人类偏好匹配方面达到了最先进（SOTA）水平，超越了顶级专有模型。我们的方法能够捕捉主观评价标准，使路由决策更加透明和灵活。我们的模型可在以下网址获取：https://huggingface.co/katanemo/Arch-Router-1.5B。

English

With the rapid proliferation of large language models (LLMs) -- each optimized for different strengths, style, or latency/cost profile -- routing has become an essential technique to operationalize the use of different models. However, existing LLM routing approaches are limited in two key ways: they evaluate performance using benchmarks that often fail to capture human preferences driven by subjective evaluation criteria, and they typically select from a limited pool of models. In this work, we propose a preference-aligned routing framework that guides model selection by matching queries to user-defined domains (e.g., travel) or action types (e.g., image editing) -- offering a practical mechanism to encode preferences in routing decisions. Specifically, we introduce Arch-Router, a compact 1.5B model that learns to map queries to domain-action preferences for model routing decisions. Our approach also supports seamlessly adding new models for routing without requiring retraining or architectural modifications. Experiments on conversational datasets demonstrate that our approach achieves state-of-the-art (SOTA) results in matching queries with human preferences, outperforming top proprietary models. Our approach captures subjective evaluation criteria and makes routing decisions more transparent and flexible. Our model is available at: https://huggingface.co/katanemo/Arch-Router-1.5B.

Arch-Router：将大语言模型路由与人类偏好对齐

Arch-Router: Aligning LLM Routing with Human Preferences

摘要

Support