面向可靠大语言模型智能体的可约不确定性建模研究

摘要

大语言模型(LLM)的不确定性量化(UQ)是日常LLM应用安全护栏的核心构件。然而，尽管LLM智能体日益应用于高度复杂的任务，当前大多数UQ研究仍聚焦于单轮问答场景。我们认为UQ研究必须转向具有交互能力的智能体现实场景，并需要建立全新的智能体UQ理论框架。本文首次提出涵盖现有多种UQ范式的通用智能体UQ表述，通过该表述揭示既往研究实质是将LLM的UQ视为不确定性累积过程——这种观点在开放世界的交互式智能体中存在根本缺陷。与此相反，我们提出条件性不确定性消减的新视角，通过强调行动的"交互性"显式建模智能体轨迹中的可消减不确定性。基于此视角，我们构建了概念框架，为LLM智能体场景的UQ设计提供可操作指导。最后，我们探讨了智能体UQ在尖端LLM开发和领域专用应用中的实践意义，并指出尚未解决的关键问题。

English

Uncertainty quantification (UQ) for large language models (LLMs) is a key building block for safety guardrails of daily LLM applications. Yet, even as LLM agents are increasingly deployed in highly complex tasks, most UQ research still centers on single-turn question-answering. We argue that UQ research must shift to realistic settings with interactive agents, and that a new principled framework for agent UQ is needed. This paper presents the first general formulation of agent UQ that subsumes broad classes of existing UQ setups. Under this formulation, we show that prior works implicitly treat LLM UQ as an uncertainty accumulation process, a viewpoint that breaks down for interactive agents in an open world. In contrast, we propose a novel perspective, a conditional uncertainty reduction process, that explicitly models reducible uncertainty over an agent's trajectory by highlighting "interactivity" of actions. From this perspective, we outline a conceptual framework to provide actionable guidance for designing UQ in LLM agent setups. Finally, we conclude with practical implications of the agent UQ in frontier LLM development and domain-specific applications, as well as open remaining problems.

面向可靠大语言模型智能体的可约不确定性建模研究

Towards Reducible Uncertainty Modeling for Reliable Large Language Model Agents

摘要

Support