SEAR：基於模式評估與路由的LLM網關方案

摘要

在评估生产级大语言模型（LLM）响应并实现多供应商LLM网关的请求路由时，需要细粒度的质量信号和基于运营实际的决策机制。为填补这一空白，我们提出SEAR系统——一种面向多模型、多供应商LLM网关的基于模式化架构的评估与路由方案。SEAR定义了可扩展的关系型数据模式，涵盖LLM评估信号（上下文、意图、响应特征、问题归因和质量评分）和网关运营指标（延迟、成本、吞吐量），通过约百个可SQL查询的类型化字段实现跨表一致性关联。为可靠生成评估信号，SEAR提出自包含的信号指令、模式内推理和多阶段生成技术，直接输出数据库就绪的结构化结果。由于信号源自LLM推理而非浅层分类器，SEAR能捕捉复杂请求语义、生成人类可理解的路由解释，并在统一查询层实现评估与路由的协同。经过数千次生产环境验证，SEAR在人工标注数据上展现出强劲的信号准确性，支持实际路由决策（包括在保持同等质量前提下实现显著成本降低）。

English

Evaluating production LLM responses and routing requests across providers in LLM gateways requires fine-grained quality signals and operationally grounded decisions. To address this gap, we present SEAR, a schema-based evaluation and routing system for multi-model, multi-provider LLM gateways. SEAR defines an extensible relational schema covering both LLM evaluation signals (context, intent, response characteristics, issue attribution, and quality scores) and gateway operational metrics (latency, cost, throughput), with cross-table consistency links across around one hundred typed, SQL-queryable columns. To populate the evaluation signals reliably, SEAR proposes self-contained signal instructions, in-schema reasoning, and multi-stage generation that produces database-ready structured outputs. Because signals are derived through LLM reasoning rather than shallow classifiers, SEAR captures complex request semantics, enables human-interpretable routing explanations, and unifies evaluation and routing in a single query layer. Across thousands of production sessions, SEAR achieves strong signal accuracy on human-labeled data and supports practical routing decisions, including large cost reductions with comparable quality.

SEAR：基於模式評估與路由的LLM網關方案

SEAR: Schema-Based Evaluation and Routing for LLM Gateways

摘要

Support