从智能体痕迹构建自动机：失败与下一步预测

摘要

基于大语言模型的智能体能够执行多步骤任务，但其行为结构仍然不透明：冗长的非结构化轨迹难以满足部署所需的安全审计与运行时监控要求。现有方法仅在单条轨迹层面或仅针对成功案例进行操作，因而遗漏了连接下一步预测与失败预测的跨运行拓扑结构。为恢复这种共享结构，我们将整个轨迹语料库压缩为单个紧凑的有限状态机（FSM），作为原本不可预测的大语言模型智能体行为的结构基底。在十二个公开数据集上，该有限状态机具有紧凑性（7–43个状态），在保留数据上的回放拟合度≥0.997，且各划分间的拓扑结构几乎一致，构建时间仅为毫秒级。该基底可同时满足两个预测目标。在下一步预测方面，FSM状态上下文在所有与真值匹配的数据集上均优于Agent Workflow Memory方法。在失败预测方面，基于状态的行为特征在保留数据上达到了最高0.94的AUROC，且在线监控器可仅凭部分轨迹即将失败运行排序在成功运行之前，从而在任务完成前很早触发提前停止。行为拓扑结构似乎更多地由部署框架而非大语言模型本身所塑造，这为安全审计和运行时监控提供了一种模型无关的结构性原语。

English

LLM-based agents execute multi-step tasks, but their behavioral structure remains opaque: long unstructured traces resist the safety auditing and runtime monitoring that deployment requires. Existing approaches operate per-trace or success-only, so they miss the cross-run topology that links next-step and failure prediction. To recover that shared structure, we collapse an entire trace corpus into a single, compact finite-state machine (FSM) that serves as a structural substrate for the otherwise unpredictable behavior of LLM agents. Across twelve public datasets, the FSMs are compact (7-43 states), replay held-out data at >=0.997 fitness with near-identical topology across splits, and build in milliseconds. This substrate addresses both prediction goals. For next-step prediction, FSM-state context outperforms Agent Workflow Memory on every ground-truth-matched dataset. For failure prediction, per-state behavioral features reach held-out AUROC up to 0.94, and an online monitor ranks failing runs above passing ones from a partial trace, triggering early stopping well before completion. Behavioral topology thus appears shaped more by the deployment harness than by the LLM, providing a model-agnostic structural primitive for safety auditing and runtime monitoring.