超越最终答案：多智能体工业工作流中轨迹级幻觉的审计

摘要

大语言模型（LLMs）正被越来越多地部署为能够推理、使用工具并执行多步操作的自主智能体。然而，大多数幻觉基准测试仍仅评估最终输出，忽略了源自中间“思考-行动-观察”步骤的失败。我们提出Trajel——一个用于审计多智能体工业工作流中轨迹级幻觉的数据集与评估框架。Trajel基于来自AssetOpsBench的专家标注智能体轨迹，引入了五类幻觉分类法（事实性、指代性、逻辑性、程序性和范围性）。我们在子任务、轨迹和长上下文三个层面基准测试了有监督检测模型。结果表明，最常见的失败模式被现有基准测试遗漏，近一半的幻觉轨迹同时涉及多种类型，且具有高二元准确率的自动检测器仍无法正确分类最微妙的类型。轨迹感知检测显著优于标准的后验验证，这使得基于分类法的评估成为更安全智能体部署的必要条件。

English

Large Language Models (LLMs) are increasingly deployed as autonomous agents that reason, use tools, and act over multiple steps. Yet most hallucination benchmarks still evaluate only the final output, missing failures that originate in intermediate Thought-Action-Observation steps. We present Trajel, a dataset and evaluation framework for auditing trajectory-level hallucinations in multi-agent industrial workflows. Trajel introduces a five-type hallucination taxonomy (factual, referential, logical, procedural, and scope-based) over expert-annotated agent traces from AssetOpsBench. We benchmark supervised detection models at the subtask, trajectory, and long-context levels. Our results show that the most common failure modes are missed by existing benchmarks, that nearly half of hallucinated trajectories involve multiple types at once, and that automated detectors with high binary accuracy still misclassify the subtlest types. Trajectory-aware detection significantly outperforms standard post-hoc verification, making taxonomy-grounded evaluation necessary for safer agentic deployment.