程序圖：LLM 代理的自演化執行結構

摘要

大型語言模型日益被部署為能夠進行長遠規劃並透過外部工具行動的代理。大多數代理透過在累積歷史上進行無約束生成來選擇動作，使得關於做什麼、以何種順序以及在何種條件下做的程序性知識保持隱含。隨著軌跡變長，代理可能迷失目標、不按順序調用工具，並重複無效的動作。我們引入程序圖譜：正如知識圖譜將事實性知識組織為 (實體, 關係, 實體) 三元組以回答「什麼是」的問題，程序圖譜將程序性知識組織為 (程序, 關係, 程序) 三元組以回答「如何做」的問題。在每個決策步驟，該框架定位代理的活躍節點，並由引導模型將周圍子圖轉換為步驟級情境引導，從而偏向求解器的下一個動作而不強制決定它。該圖譜是自演化的：一個大型語言模型精煉器將失敗軌跡與成功軌跡進行對比，並編輯圖譜的拓撲與屬性，提交那些能保持或提升留出驗證性能的編輯，同時保留被拒絕的編輯以阻止重複。從最小骨架開始，該迴圈構建的圖譜能夠匹配或超越手工設計的圖譜。它還能修復有缺陷的專家先驗。在多個資料集、任務類型和大型語言模型上，程序圖譜相較於基於記憶的基線帶來一致的增益，且自演化無需人工工程即可進一步提升性能。

English

Large language models are increasingly deployed as agents that plan over long horizons and act through external tools. Most agents select actions through unconstrained generation over an accumulating history, leaving implicit the procedural knowledge of what to do, in what order, and under which conditions. As trajectories lengthen, agents can lose track of their objectives, invoke tools out of order, and repeat unproductive actions. We introduce the Procedural Graph: just as a knowledge graph organizes factual knowledge into (entity, relation, entity) triplets for what-is questions, a Procedural Graph organizes procedural knowledge into (procedure, relation, procedure) triplets for what-to-do questions. At each decision step, the framework localizes the agent's active node, and a guidance model translates the surrounding subgraph into step-level situational guidance that biases the solver's next action without dictating it. The graph is self-evolving: an LLM refiner contrasts failed trajectories with successful ones and edits the graph's topology and attributes, committing edits that preserve or improve held-out validation performance while retaining rejected ones to discourage repetition. Starting from a minimal skeleton, the loop builds graphs that match or surpass hand-designed ones. It can also repair a flawed expert prior. Across multiple datasets, task types, and LLMs, the Procedural Graph delivers consistent gains over memory-based baselines, and self-evolution further improves performance without manual engineering.