IHEval：評估語言模型在遵循指令層次結構上的表現

IHEval: Evaluating Language Models on Following the Instruction Hierarchy

February 12, 2025

作者: Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, Yichuan Li, Qingyu Yin, Bing Yin, Meng Jiang

cs.AI

摘要

指令層級結構，從系統訊息到用戶訊息、對話歷史及工具輸出，確立了優先順序，對於確保語言模型（LMs）行為的一致性和安全性至關重要。儘管其重要性不言而喻，這一主題卻鮮少受到關注，且缺乏全面評估模型遵循指令層級能力的基準測試。我們通過引入IHEval填補了這一空白，這是一個包含3,538個範例、涵蓋九項任務的新穎基準，這些任務涉及不同優先級指令間既協調又衝突的情況。對主流LMs的評估揭示出它們在識別指令優先級上的困境。所有被評估的模型在面對衝突指令時，相比於其原有的指令遵循表現，均出現了顯著的性能下降。此外，最具競爭力的開源模型在解決此類衝突時僅達到了48%的準確率。我們的結果凸顯了在未來LMs開發中進行針對性優化的必要性。

English

The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs). Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models' ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark comprising 3,538 examples across nine tasks, covering cases where instructions in different priorities either align or conflict. Our evaluation of popular LMs highlights their struggle to recognize instruction priorities. All evaluated models experience a sharp performance decline when facing conflicting instructions, compared to their original instruction-following performance. Moreover, the most competitive open-source model only achieves 48% accuracy in resolving such conflicts. Our results underscore the need for targeted optimization in the future development of LMs.