在大型語言和視覺-語言模型中學習適應性風險管理的共形棄權政策

Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models

February 8, 2025

作者: Sina Tayebati, Divake Kumar, Nastaran Darabi, Dinithi Jayasuriya, Ranganath Krishnan, Amit Ranjan Trivedi

cs.AI

摘要

大型語言和視覺-語言模型（LLMs/VLMs）越來越多地應用於安全關鍵應用中，然而它們不透明的決策過程使風險評估和可靠性變得複雜。不確定性量化（UQ）有助於評估預測的信心並在不確定性高時實現棄權。符合性預測（CP），作為一種主要的UQ方法，提供統計保證，但依賴靜態閾值，無法適應任務複雜性和不斷變化的數據分佈，導致準確性、覆蓋範圍和信息量之間的次優折衷。為了解決這個問題，我們提出了可學習的符合性棄權，將強化學習（RL）與CP相結合，以動態優化棄權閾值。通過將CP閾值視為適應性行動，我們的方法平衡多個目標，最小化預測集大小同時保持可靠的覆蓋範圍。在各種LLM/VLM基準測試中進行了廣泛評估，結果顯示我們的方法優於最不明確分類器（LAC）和自適應預測集（APS），將準確性提高了最多3.2％，將幻覺檢測的AUROC提高了22.19％，將基於不確定性的選擇性生成（AUARC）提高了21.17％，並將校準誤差降低了70％-85％。這些改進在多個模型和數據集中持續存在，同時始終滿足90％的覆蓋目標，確立了我們的方法作為在安全關鍵應用中進行可靠決策的更有效靈活的解決方案。代碼可在以下鏈接找到：{https://github.com/sinatayebati/vlm-uncertainty}。

English

Large Language and Vision-Language Models (LLMs/VLMs) are increasingly used in safety-critical applications, yet their opaque decision-making complicates risk assessment and reliability. Uncertainty quantification (UQ) helps assess prediction confidence and enables abstention when uncertainty is high. Conformal prediction (CP), a leading UQ method, provides statistical guarantees but relies on static thresholds, which fail to adapt to task complexity and evolving data distributions, leading to suboptimal trade-offs in accuracy, coverage, and informativeness. To address this, we propose learnable conformal abstention, integrating reinforcement learning (RL) with CP to optimize abstention thresholds dynamically. By treating CP thresholds as adaptive actions, our approach balances multiple objectives, minimizing prediction set size while maintaining reliable coverage. Extensive evaluations across diverse LLM/VLM benchmarks show our method outperforms Least Ambiguous Classifiers (LAC) and Adaptive Prediction Sets (APS), improving accuracy by up to 3.2%, boosting AUROC for hallucination detection by 22.19%, enhancing uncertainty-guided selective generation (AUARC) by 21.17%, and reducing calibration error by 70%-85%. These improvements hold across multiple models and datasets while consistently meeting the 90% coverage target, establishing our approach as a more effective and flexible solution for reliable decision-making in safety-critical applications. The code is available at: {https://github.com/sinatayebati/vlm-uncertainty}.