從智能體軌跡建構自動機：失敗與下一步預測

摘要

基於LLM的智能體執行多步驟任務，但其行為結構仍然不透明：冗長的非結構化軌跡阻礙了部署所需的安全審計與運行時監控。現有方法以逐軌跡或僅限成功案例的方式運作，因而錯失連結下一步預測與失敗預測的跨運行拓撲結構。為還原此一共用結構，我們將整個軌跡語料庫壓縮為單一緊湊的有限狀態機（FSM），作為LLM智能體原本不可預測行為的結構基底。在十二個公開資料集上，該FSM保持緊湊（7至43個狀態），以≥0.997的適應度重放留出資料，各分割間呈現近乎相同的拓撲結構，且建構僅需毫秒。此基底同時達成兩項預測目標：就下一步預測而言，FSM狀態上下文在所有真實標註配對的資料集上均優於智能體工作流記憶（Agent Workflow Memory）；就失敗預測而言，逐狀態行為特徵在留出資料上可達0.94的AUROC，且在線監控器僅憑部分軌跡即將失敗運行排序於成功運行之上，遠在任務完成前觸發提前停止。由此可見，行為拓撲受部署框架的形塑程度更甚於LLM本身，為安全審計與運行時監控提供了與模型無關的結構原語。

English

LLM-based agents execute multi-step tasks, but their behavioral structure remains opaque: long unstructured traces resist the safety auditing and runtime monitoring that deployment requires. Existing approaches operate per-trace or success-only, so they miss the cross-run topology that links next-step and failure prediction. To recover that shared structure, we collapse an entire trace corpus into a single, compact finite-state machine (FSM) that serves as a structural substrate for the otherwise unpredictable behavior of LLM agents. Across twelve public datasets, the FSMs are compact (7-43 states), replay held-out data at >=0.997 fitness with near-identical topology across splits, and build in milliseconds. This substrate addresses both prediction goals. For next-step prediction, FSM-state context outperforms Agent Workflow Memory on every ground-truth-matched dataset. For failure prediction, per-state behavioral features reach held-out AUROC up to 0.94, and an online monitor ranks failing runs above passing ones from a partial trace, triggering early stopping well before completion. Behavioral topology thus appears shaped more by the deployment harness than by the LLM, providing a model-agnostic structural primitive for safety auditing and runtime monitoring.