TRACER：基于追踪的自适应成本优化路由策略在LLM分类中的应用

TRACER: Trace-Based Adaptive Cost-Efficient Routing for LLM Classification

April 16, 2026

作者: Adam Rida

cs.AI

摘要

每次调用LLM分类终端都会产生已保留在生产日志中的标注输入-输出对。这些数据对构成了一个免费且持续增长的训练集：基于此训练的轻量级替代模型能以近乎零边际推理成本承担未来大量请求。核心问题在于何时部署替代模型才足够可靠、其处理与转交任务的边界如何划分，以及该边界如何随数据积累动态演化。 我们推出TRACER（基于追踪的自适应成本效益路由系统），这一开源系统利用LLM自身生产轨迹训练机器学习替代模型，并通过一致性校验门控部署策略：仅当替代模型与LLM的预测一致率超过用户设定阈值α时才激活使用。为实现路由边界的透明化，TRACER生成可解释性分析报告，清晰展示替代模型的有效处理范围、性能瓶颈区域及转交决策依据。 在77类意图识别基准测试中（以Sonnet 4.6作为教师模型），TRACER实现的替代模型覆盖率可达83-100%（具体取决于质量目标α）；在150类基准测试中，替代模型可完全取代教师模型。对于自然语言推理任务，系统通过一致性校验门正确拒绝部署，因为嵌入表示无法支撑可靠的决策分离。本系统已作为开源软件发布。

English

Every call to an LLM classification endpoint produces a labeled input-output pair already retained in production logs. These pairs constitute a free, growing training set: a lightweight surrogate trained on them can absorb a significant portion of future traffic at near-zero marginal inference cost. The open questions are when the surrogate is reliable enough to deploy, what it handles versus defers, and how that boundary evolves as data accumulates. We introduce TRACER (Trace-based Adaptive Cost-Efficient Routing), an open-source system that trains ML surrogates on an LLM's own production traces and governs deployment through a parity gate: the surrogate is activated only when its agreement with the LLM exceeds a user-specified threshold α. To make the routing boundary transparent, TRACER generates interpretability artifacts describing which input regions the surrogate handles, where it plateaus, and why it defers. On a 77-class intent benchmark with a Sonnet 4.6 teacher, TRACER achieves 83-100% surrogate coverage depending on the quality target α; on a 150-class benchmark, the surrogate fully replaces the teacher. On a natural language inference task, the parity gate correctly refuses deployment because the embedding representation cannot support reliable separation. The system is available as open-source software.