SEAR：基于模式评估与路由的大语言模型网关方案

摘要

评估生产级大语言模型（LLM）响应并在LLM网关中实现多供应商请求路由，需要细粒度的质量信号和基于运营实际的决策。为填补这一空白，我们提出SEAR——一种面向多模型、多供应商LLM网关的基于模式的评估路由系统。SEAR定义了可扩展的关系模式，涵盖LLM评估信号（上下文、意图、响应特征、问题归因和质量评分）与网关运营指标（延迟、成本、吞吐量），通过约百个可SQL查询的类型化字段实现跨表一致性关联。为可靠生成评估信号，SEAR提出自包含的信号指令、模式内推理和多阶段生成技术，直接输出数据库就绪的结构化结果。由于信号源自LLM推理而非浅层分类器，SEAR能捕捉复杂请求语义，提供人类可理解的路由解释，并在统一查询层整合评估与路由功能。经过数千次生产环境验证，SEAR在人工标注数据上实现高精度信号识别，支持实际路由决策（包括在保持同等质量的同时显著降低成本）。

English

Evaluating production LLM responses and routing requests across providers in LLM gateways requires fine-grained quality signals and operationally grounded decisions. To address this gap, we present SEAR, a schema-based evaluation and routing system for multi-model, multi-provider LLM gateways. SEAR defines an extensible relational schema covering both LLM evaluation signals (context, intent, response characteristics, issue attribution, and quality scores) and gateway operational metrics (latency, cost, throughput), with cross-table consistency links across around one hundred typed, SQL-queryable columns. To populate the evaluation signals reliably, SEAR proposes self-contained signal instructions, in-schema reasoning, and multi-stage generation that produces database-ready structured outputs. Because signals are derived through LLM reasoning rather than shallow classifiers, SEAR captures complex request semantics, enables human-interpretable routing explanations, and unifies evaluation and routing in a single query layer. Across thousands of production sessions, SEAR achieves strong signal accuracy on human-labeled data and supports practical routing decisions, including large cost reductions with comparable quality.

SEAR：基于模式评估与路由的大语言模型网关方案

SEAR: Schema-Based Evaluation and Routing for LLM Gateways

摘要

Support