GenCtrl——生成式模型的形式化可控性工具包

摘要

随着生成模型日益普及，对生成过程进行细粒度控制的需求日益迫切。然而尽管从提示工程到微调的各种受控生成方法层出不穷，一个根本性问题始终悬而未决：这些模型是否真的具备可控性？本研究提出理论框架以系统回答该问题。通过将人机交互建模为控制过程，我们创新性地提出一种估测对话场景中模型可控集的算法。值得注意的是，我们建立了关于样本复杂度函数估计误差的形式化保证：推导出具有概率近似正确性的可控集估计边界，该边界无需分布假设、仅要求输出有界性条件，且适用于任何黑盒非线性控制系统（即任意生成模型）。我们在对话流程控制的多个任务中，针对语言模型和文生图模型进行了理论框架的实证验证。研究结果表明，模型可控性出人意料地脆弱，且高度依赖实验设置。这凸显了进行严格可控性分析的必要性，应将研究重点从单纯尝试控制转向首先理解其根本局限。

English

As generative models become ubiquitous, there is a critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fine-tuning proliferate, a fundamental question remains unanswered: are these models truly controllable in the first place? In this work, we provide a theoretical framework to formally answer this question. Framing human-model interaction as a control process, we propose a novel algorithm to estimate the controllable sets of models in a dialogue setting. Notably, we provide formal guarantees on the estimation error as a function of sample complexity: we derive probably-approximately correct bounds for controllable set estimates that are distribution-free, employ no assumptions except for output boundedness, and work for any black-box nonlinear control system (i.e., any generative model). We empirically demonstrate the theoretical framework on different tasks in controlling dialogue processes, for both language models and text-to-image generation. Our results show that model controllability is surprisingly fragile and highly dependent on the experimental setting. This highlights the need for rigorous controllability analysis, shifting the focus from simply attempting control to first understanding its fundamental limits.

GenCtrl——生成式模型的形式化可控性工具包

GenCtrl -- A Formal Controllability Toolkit for Generative Models

摘要

Support