SafePyramid：一個用於上下文內政策防護的階層式基準

摘要

在實際應用中，防護機制（guardrails）往往需依據特定應用的安全政策（safety policies）來識別不安全的使用者-模型互動，而非依賴預先定義的風險分類法。本研究在「情境內政策防護」（in-context policy guardrailing）的範式下探討此設定，其中防護機制根據情境中提供的政策規範來預測安全違規行為。為系統性評估此能力，我們提出SafePyramid，這是一個涵蓋10個領域、1,000輪多輪對話及3,000條對應應用特定政策的安全基準測試，總計包含61,699條以自然語言撰寫的規則。SafePyramid將評估分為三個難度層級：L0評估單一規則的理解能力，L1評估對規則依賴關係的推理能力，L2評估對情境中定義的全新政策框架的適應能力。為確保基準品質，我們採用嚴謹的多階段流程來建構並驗證該基準。透過SafePyramid，我們評估了10個前沿大型語言模型（LLM）與5個可配置政策的防護機制，發現情境內政策防護仍極具挑戰性：即使表現最佳的模型GPT-5.5，在L0、L1與L2層級上，能完整識別所有違規規則的比例僅分別為54.0%、35.3%與12.9%。這些結果凸顯當前防護機制的局限性，並呼籲發展更強大的情境內政策防護機制，以可靠執行政策、解決規則依賴關係，並適應新穎的政策框架。

English

In real-world applications, guardrails are often expected to identify unsafe user-model interactions according to application-specific safety policies, rather than relying on predefined risk taxonomies. In this work, we study this setting under the paradigm of in-context policy guardrailing, where guardrails predict safety violations based on policy specifications provided in context. To systematically evaluate this capability, we introduce SafePyramid, a safety benchmark comprising 1,000 multi-turn conversations across 10 domains and 3,000 corresponding application-specific policies, which together contain 61,699 distinct natural-language rules. SafePyramid organizes the evaluation into three difficulty levels: L0 evaluates individual-rule understanding, L1 evaluates reasoning over rule dependencies, and L2 evaluates adaptation of full novel policy frameworks defined in context. To ensure benchmark quality, we employ a rigorous multi-stage pipeline to construct and validate the benchmark. Using SafePyramid, we evaluate 10 frontier LLMs and 5 policy-configurable guardrails and find that in-context policy guardrailing remains highly challenging: even the best-performing model, GPT-5.5, exactly identifies the full set of violated rules in only 54.0%, 35.3%, and 12.9% cases on L0, L1, and L2, respectively. These results highlight the limitations of current guardrails and call for stronger in-context policy guardrails that can reliably execute policies, resolve rule dependencies, and adapt to novel policy frameworks.