面向可靠大语言模型智能体的可约不确定性建模研究

摘要

大型语言模型（LLM）的不确定性量化（UQ）是日常LLM应用安全防护机制的关键组成部分。然而，尽管LLM智能体正被日益部署于高度复杂的任务中，现有UQ研究仍主要聚焦于单轮问答场景。我们认为UQ研究必须转向具有交互能力的智能体所面临的真实场景，并需要建立面向智能体UQ的新理论框架。本文首次提出涵盖多类现有UQ设置的智能体UQ通用表述，通过该框架揭示以往研究实质是将LLM的UQ视为不确定性累积过程——这一观点在开放世界的交互式智能体场景中已不再适用。与此相对，我们提出条件不确定性消减这一新视角，通过强调行动的"交互性"来显式建模智能体行动轨迹中的可消减不确定性。基于此视角，我们构建了概念框架，为LLM智能体场景中的UQ设计提供可操作的指导。最后，本文探讨了智能体UQ在前沿LLM开发和领域专用应用中的实践意义，并指出尚未解决的关键问题。

English

Uncertainty quantification (UQ) for large language models (LLMs) is a key building block for safety guardrails of daily LLM applications. Yet, even as LLM agents are increasingly deployed in highly complex tasks, most UQ research still centers on single-turn question-answering. We argue that UQ research must shift to realistic settings with interactive agents, and that a new principled framework for agent UQ is needed. This paper presents the first general formulation of agent UQ that subsumes broad classes of existing UQ setups. Under this formulation, we show that prior works implicitly treat LLM UQ as an uncertainty accumulation process, a viewpoint that breaks down for interactive agents in an open world. In contrast, we propose a novel perspective, a conditional uncertainty reduction process, that explicitly models reducible uncertainty over an agent's trajectory by highlighting "interactivity" of actions. From this perspective, we outline a conceptual framework to provide actionable guidance for designing UQ in LLM agent setups. Finally, we conclude with practical implications of the agent UQ in frontier LLM development and domain-specific applications, as well as open remaining problems.

面向可靠大语言模型智能体的可约不确定性建模研究

Towards Reducible Uncertainty Modeling for Reliable Large Language Model Agents

摘要

Support