Doc-PP：大型視覺語言模型文件政策遵循基準測試

摘要

大型視覺語言模型在實際文件問答應用中的部署，常受到動態用戶自訂策略的制約，這些策略會根據情境規範資訊揭露範圍。儘管確保遵守明確約束條件至關重要，現有安全研究主要聚焦於隱性社會規範或純文字情境，未能充分考量多模態文件的複雜性。本文提出Doc-PP（文件策略保全基準），這項創新基準建構自真實世界報告，需在嚴格保密政策下對異質視覺與文字元素進行跨模態推理。我們的評估揭示出系統性的「推理誘發安全漏洞」：當答案需透過複雜合成或跨模態聚合推斷時，模型頻繁洩露敏感資訊，實質上繞過現有安全防護機制。更進一步發現，提供提取文字雖能提升感知能力，卻意外助長資訊洩漏。為應對這些弱點，我們提出DVA（分解-驗證-聚合）結構化推理框架，將推理過程與策略驗證解耦。實驗結果顯示，DVA顯著優於標準提示防禦方案，為合規文件理解提供了強健的基準框架。

English

The deployment of Large Vision-Language Models (LVLMs) for real-world document question answering is often constrained by dynamic, user-defined policies that dictate information disclosure based on context. While ensuring adherence to these explicit constraints is critical, existing safety research primarily focuses on implicit social norms or text-only settings, overlooking the complexities of multimodal documents. In this paper, we introduce Doc-PP (Document Policy Preservation Benchmark), a novel benchmark constructed from real-world reports requiring reasoning across heterogeneous visual and textual elements under strict non-disclosure policies. Our evaluation highlights a systemic Reasoning-Induced Safety Gap: models frequently leak sensitive information when answers must be inferred through complex synthesis or aggregated across modalities, effectively circumventing existing safety constraints. Furthermore, we identify that providing extracted text improves perception but inadvertently facilitates leakage. To address these vulnerabilities, we propose DVA (Decompose-Verify-Aggregation), a structural inference framework that decouples reasoning from policy verification. Experimental results demonstrate that DVA significantly outperforms standard prompting defenses, offering a robust baseline for policy-compliant document understanding

Doc-PP：大型視覺語言模型文件政策遵循基準測試

Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models

摘要

Support