模型还是框架？一种以交互为中心的智能体故障定位分类法

摘要

现有的评估通常将智能体故障简化为系统级结果，从而掩盖了故障的根源所在以及哪项干预措施能够改进智能体系统。这引出了修复分配问题：相同的可见故障可能因其来源不同，而分别要求模型后训练、编排框架工程、环境重新设计或基准修复。由于智能体行为源于模型、编排框架、用户、工具、记忆和环境之间的交互，结果级标签往往不足以指导改进。现有的大多数故障分类体系对此助益有限，因为它们具有基准特异性且缺乏共享结构。我们提出了一种以交互为中心的故障分类体系，将故障定位到其产生的交互环节，并识别出责任组件。该体系通过将41种故障模式分配到两个组件之间的边上，并标注指示修复归属的故障侧，从而实现了可操作性：模型侧故障指明后训练目标，编排框架侧故障指向脚手架与工具集成修复，而环境或评分器故障则揭示需要重新设计的评估条件。该分类体系适用于各类智能体架构，从编码助手到长时程个人助手以及多智能体系统。我们在公开基准、模型系统卡、已发表报告和智能体轨迹日志的实例分析中对该分类体系进行了验证，并使用独立推理智能体作为评判器评估其可复现性。在四个前沿模型中，最强的评判器与人工类别标签的Cohen's κ达到0.76，表明这些类别捕捉到的是共享结构而非标注者特有的偏好。

English

Existing evaluations often reduce agent failures to system-level outcomes, obscuring where the fault originated and which intervention would improve the agent system. This creates a repair-assignment problem: the same visible failure may call for model post-training, harness engineering, environment redesign, or benchmark repair depending on its source. Because agent behavior emerges from interactions among models, harnesses, users, tools, memory, and environments, outcome-level labels are often insufficient for improvement. Most failure taxonomies do little to resolve this problem because they are benchmark-specific and lack a shared structure. We introduce an interaction-centric taxonomy that localizes failures to the interactions in which they originate and identifies the responsible component. It organizes 41 failure modes by assigning each to an edge between two components and a fault side indicating where the repair belongs. This makes the taxonomy actionable: model-side failures identify targets for post-training, harness-side failures point to scaffolding and tool-integration fixes, and environment or grader failures reveal evaluation conditions requiring redesign. The schema applies across agent architectures, from coding assistants to long-horizon personal assistants and multi-agent systems. We ground the taxonomy in worked examples from public benchmarks, model system cards, published reports, and logged agent trajectories, and evaluate its reproducibility using independent reasoning agents as judges. Across four frontier models, the strongest judge reaches Cohen's κ=0.76 against human category labels, suggesting that the categories capture shared structure rather than annotator-specific preferences.