StepGuard：以可擴展監督與安全-效用平衡學習步驟級防護機制

摘要

基於LLM的智能體可以透過工具調用與外部環境互動，但此能力也引入了安全風險，例如檔案修改、資訊洩露與未授權操作。現有的護欄機制通常評估已完成的軌跡，導致步驟級動作的執行前監控仍未獲充分探索。我們提出StepGuard，一種步驟級護欄模型，既能審計已完成的智能體軌跡，也能在工具動作執行前進行檢查。為訓練StepGuard，我們引入StepGen，一個自動化資料引擎，可生成具有相同上下文但在風險步驟採取不同動作的安全與不安全軌跡。為進一步減少過度防禦與防禦不足，我們提出Balance-GRPO，該方法根據安全與不安全動作的觀測準確率，動態平衡對兩者的學習。實驗結果顯示，StepGuard在開放權重護欄模型中達到最高平均準確率，效能可與GPT-5.4相匹敵。在AgentDojo與AgentDyn上守護智能體時，相較於無護欄設定，StepGuard將平均攻擊成功率降低了77.3%，而平均效用僅下降2.8個百分點。

English

LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage, and unauthorized actions. Existing guardrails often evaluate completed trajectories, leaving pre-execution monitoring of step-level actions underexplored. We propose StepGuard, a step-level guard model that can audit completed agent trajectories and check tool actions before they are executed. To train StepGuard, we introduce StepGen, an automatic data engine that generates safe and unsafe trajectories with the same context but different actions at the risky step. To further reduce over-defense and under-defense, we propose Balance-GRPO, which dynamically balances learning between safe and unsafe actions based on their observed accuracy. Experiments show that StepGuard achieves the highest average accuracy among open-weight guard models, with performance comparable to GPT-5.4. When used to guard agents on AgentDojo and AgentDyn, StepGuard reduces mean attack success rate by 77.3% relative to the no-guard setting, while mean utility drops by only 2.8 percentage points.