AISPA：以用户为中心的大语言模型应用系统提示词审计

摘要

系统提示词是由开发者配置的指令，用于规范AI应用中基础模型的行为。它们被广泛应用于商业AI产品，却很少向公众或监管机构披露，这导致AI系统的大规模部署存在严重的信任与问责缺口。本文提出了人工智能系统提示词保障（AISPA）——一个以用户为中心的框架，用于系统性地审计AI系统中的系统提示词。AISPA对系统提示词的具体组成部分进行审查，并沿八个对用户至关重要的维度对其进行评估。我们随后运用该框架对88个商业AI产品中系统提示词的3，249条指令进行了审查，将每条指令分类为保护性（保护用户）或问题性。我们的审计揭示了四项核心发现。第一，系统提示词的设计在不同产品和开发者之间存在显著差异，部分组织平均每个产品包含超过60条保护性指令，而其他组织平均不足5条。第二，保护性指令得到广泛采用但覆盖深度不足：98.9%的产品至少包含一条保护性指令，但仅有24%的产品涵盖AISPA分类体系的全部八个维度。第三，系统提示词的长度和对用户的保护性持续增长，表明用户保护正成为商业提示词设计中日益受关注的问题。第四，尽管取得了上述进展，问题性指令仍然普遍存在：约40%的产品至少包含一条违背用户利益的指令，且保护性与问题性指令常常共存于同一提示词中。我们的研究结果表明，商业AI产品中的系统提示词亟需更高的透明度、标准化水平以及独立的监督机制。

English

System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regulators, creating a serious trust and accountability gap in the wide deployment of AI systems. In this paper, we introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems. AISPA examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users. We then use this framework to review 3,249 instructions from system prompts in 88 commercial AI products, classifying each instruction as either protective (of users) or problematic. Our audit surfaces four core findings. First, system prompt design varies substantially across products and developers, with some organizations averaging over 60 protective instructions per product while others average fewer than 5. Second, protective instructions are widely adopted but shallow in scope: 98.9% of products contain at least one, yet only 24% cover all eight dimensions of the AISPA taxonomy. Third, system prompts have grown steadily longer and more protective of users, suggesting that user protection is becoming a more visible concern in commercial prompt design. Fourth, despite this progress, problematic instructions remain pervasive: roughly 40% of products contain at least one instruction that works against user interests, and protective and problematic instructions frequently coexist within the same prompt. Our findings highlight the need for greater transparency, standardization, and independent oversight for system prompts in commercial AI products.