AutoSaddler：基於代理執行軌跡之自動框架最佳化與持久更新

摘要

LLM 智能體在長時程任務上仍不可靠，其中微小的局部失敗可能因長時間的互動而累積，最終導致整體任務失敗。儘管外部控制框架能大幅提升穩健性，但控制框架的設計仍是一項耗費人力且成本高昂的過程，需要在龐大的提示詞、工具配置及控制邏輯空間中進行搜尋。我們提出 AutoSaddler——一個自動化控制框架優化系統——將控制框架改進形式化為離線學習問題，並利用小批次中獲得的失敗訊號迭代更新控制框架。AutoSaddler 結合了失敗軌跡診斷、將控制框架視為程式碼的結構化補丁生成，以及基於驗證的更新選擇。在 GAIA2、SWE-Bench Pro 和 Terminal-Bench 2.0 上的實驗表明，AutoSaddler 相較於對應的基礎控制框架大幅提升了智能體效能，分別取得了 9.0、9.6 和 10.0 個百分點的增益。消融研究進一步指出，有效的控制框架優化得益於三個要素：深度除錯而非淺層反思、針對性修改而非無約束編輯，以及具有泛化意識的選擇而非針對個別軌跡的修復。綜合上述結果，自動化控制框架優化是通往更高效能、更可靠智能體系統的一條有前景的途徑。

English

LLM agents remain unreliable on long-horizon tasks, where small local failures can compound over extended interactions and lead to overall task failure. Although external harnesses can substantially improve robustness, harness design remains a manual and expensive process that requires searching over a large space of prompts, tool configurations, and control logic. We propose AutoSaddler, an automatic harness optimization framework that formulates harness improvement as an offline learning problem and iteratively updates the harness using failure signals from mini-batches. AutoSaddler combines failure-trace diagnosis, structured patch generation that treats the harness as code, and validation-based update selection. Experiments on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0 show that AutoSaddler substantially improves agent performance over the corresponding base harnesses, achieving gains of 9.0, 9.6, and 10.0 percentage points, respectively. Ablation studies further suggest that effective harness optimization benefits from three ingredients: deep debugging rather than shallow reflection, targeted modifications rather than unconstrained editing, and generalization-aware selection rather than trajectory-specific repair. Together, these results suggest that automatic harness optimization is a promising path toward more performant and reliable agent systems.