SafePyramid: 一种用于上下文策略护栏的分层基准

摘要

在实际应用中，护栏通常需要依据具体应用的安全策略来识别用户与模型之间的不安全交互，而非依赖预定义的风险分类体系。本研究在上下文策略护栏范式下探讨这一场景，即护栏根据上下文提供的策略规范预测安全违规行为。为系统评估这一能力，我们提出SafePyramid安全基准测试，包含覆盖10个领域的1,000轮多轮对话及3,000条对应的应用特定策略，共计61,699条自然语言规则。SafePyramid将评估划分为三个难度层级：L0评估单条规则理解能力，L1评估规则间的依赖推理能力，L2评估上下文中定义的全新策略框架的适应能力。为确保基准质量，我们通过严格的多阶段流程构建并验证该测试集。基于SafePyramid对10个前沿大语言模型和5个策略可配置护栏的评估表明，上下文策略护栏仍面临严峻挑战：即使表现最佳的GPT-5.5，在L0、L1和L2层级上能准确识别所有违规规则的案例比例也分别仅为54.0%、35.3%和12.9%。这些结果揭示了当前护栏的局限性，亟需开发更强的上下文策略护栏，使其能够可靠执行策略、解决规则依赖关系并适应新型策略框架。

English

In real-world applications, guardrails are often expected to identify unsafe user-model interactions according to application-specific safety policies, rather than relying on predefined risk taxonomies. In this work, we study this setting under the paradigm of in-context policy guardrailing, where guardrails predict safety violations based on policy specifications provided in context. To systematically evaluate this capability, we introduce SafePyramid, a safety benchmark comprising 1,000 multi-turn conversations across 10 domains and 3,000 corresponding application-specific policies, which together contain 61,699 distinct natural-language rules. SafePyramid organizes the evaluation into three difficulty levels: L0 evaluates individual-rule understanding, L1 evaluates reasoning over rule dependencies, and L2 evaluates adaptation of full novel policy frameworks defined in context. To ensure benchmark quality, we employ a rigorous multi-stage pipeline to construct and validate the benchmark. Using SafePyramid, we evaluate 10 frontier LLMs and 5 policy-configurable guardrails and find that in-context policy guardrailing remains highly challenging: even the best-performing model, GPT-5.5, exactly identifies the full set of violated rules in only 54.0%, 35.3%, and 12.9% cases on L0, L1, and L2, respectively. These results highlight the limitations of current guardrails and call for stronger in-context policy guardrails that can reliably execute policies, resolve rule dependencies, and adapt to novel policy frameworks.