關於開放基金會模型對社會的影響

On the Societal Impact of Open Foundation Models

February 27, 2024

作者: Sayash Kapoor, Rishi Bommasani, Kevin Klyman, Shayne Longpre, Ashwin Ramaswami, Peter Cihon, Aspen Hopkins, Kevin Bankston, Stella Biderman, Miranda Bogen, Rumman Chowdhury, Alex Engler, Peter Henderson, Yacine Jernite, Seth Lazar, Stefano Maffulli, Alondra Nelson, Joelle Pineau, Aviya Skowron, Dawn Song, Victor Storchan, Daniel Zhang, Daniel E. Ho, Percy Liang, Arvind Narayanan

cs.AI

摘要

基礎模型是強大的技術：它們如何公開發布直接塑造了它們的社會影響。在這份立場文件中，我們專注於開放的基礎模型，這裡定義為具有廣泛可用的模型權重（例如Llama 2、Stable Diffusion XL）。我們確定了開放的基礎模型具有五個獨特特性（例如更大的可定製性、監控不足），這些特性既帶來了好處也帶來了風險。開放的基礎模型提供了顯著的好處，但也有一些注意事項，涵蓋了創新、競爭、決策權的分配以及透明度。為了了解它們被誤用的風險，我們設計了一個風險評估框架，用於分析它們的邊際風險。在多個誤用向量（例如網絡攻擊、生物武器）中，我們發現目前的研究不足以有效地表徵開放的基礎模型相對於現有技術的邊際風險。該框架有助於解釋為什麼在某些情況下邊際風險很低，通過揭示過去的工作集中在不同的框架子集上並具有不同的假設，澄清了有關誤用風險的爭議，並提出了更具建設性辯論的前進方向。總的來說，我們的工作有助於支持對開放基礎模型的社會影響進行更為扎實的評估，並概述了需要進行的研究以實證驗證它們的理論好處和風險。

English

Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.