StealthBench：衡量自主进攻性安全代理的操作隐蔽性

摘要

隐蔽性——即在实现目标时不暴露自身存在、能力或已获取情报的技术——正是区分资深操作者与易被察觉者的关键。顶尖安全研究员与高级持续性威胁能在无人察觉的情况下达成目标；自主智能体正越来越多地承担相同的进攻性任务，但它们是否继承了这种战术素养？我们推出StealthBench，一个从操作安全（OPSEC）六个维度衡量自主进攻性安全智能体操作隐蔽性的基准测试。我们从真实的漏洞赏金和红队行动轨迹中提取了11个经人工验证的OPSEC事件，并将其扩展为14个Docker化任务场景。在这些场景中，智能体虽能发现真实漏洞，却在隐蔽性上出现不符合标准操作战术的失误：将凭据嵌入公开上传内容、为证明访问权限而删除生产资源、强行添加无关用户以演示竞态条件。我们使用由三个大语言模型（LLM）组成的评审小组，通过多数投票聚合的方式评估智能体的行动轨迹，测量安全成功率（即任务完成且保持隐蔽）、隐蔽性@求解（成功求解中的战术质量）以及鲁莽求解率（任务完成但暴露行踪）。结果表明，没有任何模型的安全成功率超过54%（这一复合指标要求同时完成任务和保持隐蔽），证实了OPSEC失败在各模型家族中具有系统性。我们以公开基准测试的形式发布StealthBench，旨在支持对隐蔽性敏感的智能体开发，以及为自主进攻性安全部署提供自动化的OPSEC监控。交互式排行榜、评估框架和数据集可在https://stealthbench.com获取。

English

Stealth, the discipline of achieving an objective without revealing your presence, capabilities, or collected intelligence, is what separates sophisticated operators from detectable ones. Elite security researchers and advanced persistent threats achieve their objectives unnoticed; autonomous agents increasingly inherit the same offensive tasks, but do they inherit the tradecraft? We introduce StealthBench,a benchmark that measures operational stealth in autonomous offensive-security agents across six operational security (OPSEC) dimensions. We extract 11 hand-verified OPSEC incidents from real bug-bounty and red-team trajectories, expanded into 14 dockerized task scenarios, where agents, despite finding real vulnerabilities, committed stealth failures inconsistent with standard operational tradecraft: embedding credentials in public uploads, deleting production resources to prove access, force-adding uninvolved users to demonstrate a race condition. We evaluate agent trajectories using a 3-model large language model (LLM) judge panel with majority-vote aggregation, measuring safe success rate (solved and stealthy), Stealth@Solve (tradecraft quality among successful solves), and reckless solve rate (solved but cover blown). Our results show that no model exceeds 54% safe success rate (the compound metric requiring both task completion and stealth), confirming that OPSEC failures are systematic across model families. We release StealthBench as a public benchmark to support both the development of stealth-aware agents and automated OPSEC monitoring for autonomous offensive-security deployments. The interactive leaderboard, evaluation harness, and dataset are available at https://stealthbench.com.