LLM智能体的冷启动安全鸿沟

摘要

具备工具调用能力的大语言模型智能体在对话全程是否同样安全？我们发现并非如此：智能体在会话启动阶段最为脆弱，而在完成若干常规智能体任务后安全性显著提升——这种现象我们称之为“冷启动安全缺口”。为系统研究这一现象，我们提出了面向智能体的安全性深度评估基准（SODA），该基准可控制智能体在遭遇安全威胁前完成的常规智能体任务数量，最多可设置20个前置任务。通过对4个模型家族7个模型的评估发现，随着前置常规任务数从0增至20，安全性提升了9%-52%。表征分析证实，随着前置任务增加，模型隐藏状态逐渐向安全对齐区域迁移。通过系统研究前置对话中哪些部分最影响安全性，我们发现：常规智能体任务本身是提升安全性的主要驱动力，而智能体自身的先前响应虽对安全性影响较小，却是维持后续实用性的关键。该结论在开源安全基准（AgentHarm、Agent Safety Bench）与实用性基准（BFCL、API-Bank）上的评估中进一步得到验证，证实部署前让智能体通过常规任务进行预热，既能提升安全性又能保持完整能力。基于这些发现，我们推荐一种简单的部署策略：在可能接触安全关键请求前，让智能体先完成若干常规智能体任务，以缓解冷启动安全缺口。我们的代码已开源：https://github.com/Trustworthy-ML-Lab/Agent-Cold-Start-Safety-Gap

English

Are tool-calling LLM agents equally safe throughout a conversation? We discover they are not: agents are most vulnerable at the very start of a session and become substantially safer after a few regular agentic tasks -- a phenomenon we term the cold-start safety gap. To study this systematically, we introduce Safety Over Depth for Agents (SODA), a benchmark that controls how many regular agentic tasks the agent completes before encountering a safety threat, supporting up to 20 preceding tasks. Evaluating 7 models from 4 families, safety improves by 9--52% as the number of preceding regular agentic tasks increases from zero to twenty. Representation analysis confirms that model hidden states gradually shift toward a safety-aligned region as more preceding tasks are present. By systematically studying which part of the preceding conversation matters most, we find that the regular agentic tasks themselves are the primary driver of safety, while the agent's own prior responses have less effect on safety but are essential for preserving later utility. This conclusion is further supported by evaluation on open-source safety benchmarks (AgentHarm, Agent Safety Bench) and utility benchmarks (BFCL, API-Bank), confirming that warming up the agent with regular agentic tasks before deployment makes it safer and preserves full capability. Based on these findings, we recommend a simple deployment strategy: having the agent complete a few regular agentic tasks before possible exposure to safety-critical requests mitigates the cold-start safety gap. Our code is available at https://github.com/Trustworthy-ML-Lab/Agent-Cold-Start-Safety-Gap