开放性对人工超人类智能至关重要。

Open-Endedness is Essential for Artificial Superhuman Intelligence

June 6, 2024

作者: Edward Hughes, Michael Dennis, Jack Parker-Holder, Feryal Behbahani, Aditi Mavalankar, Yuge Shi, Tom Schaul, Tim Rocktaschel

cs.AI

摘要

近年来，人工智能系统的整体能力出现了巨大增长，主要是通过在互联网规模数据上训练基础模型实现的。然而，创造出开放式、不断自我改进的人工智能仍然是难以实现的。在这篇立场论文中，我们认为现在已经具备了实现人工智能系统相对于人类观察者具有开放性的要素。此外，我们主张这种开放性是任何人工超人类智能的基本属性。我们首先通过新颖性和可学习性的视角提供了开放性的具体形式定义。然后，我们阐明了通过建立在基础模型之上的开放式系统的路径，这些系统能够做出新颖的、与人类相关的发现，从而实现人工超人类智能。最后，我们考察了具有普遍能力的开放式人工智能的安全影响。我们预计，开放式基础模型将在不久的将来证明是一个越来越富有成效和安全关键的研究领域。

English

In recent years there has been a tremendous surge in the general capabilities of AI systems, mainly fuelled by training foundation models on internetscale data. Nevertheless, the creation of openended, ever self-improving AI remains elusive. In this position paper, we argue that the ingredients are now in place to achieve openendedness in AI systems with respect to a human observer. Furthermore, we claim that such open-endedness is an essential property of any artificial superhuman intelligence (ASI). We begin by providing a concrete formal definition of open-endedness through the lens of novelty and learnability. We then illustrate a path towards ASI via open-ended systems built on top of foundation models, capable of making novel, humanrelevant discoveries. We conclude by examining the safety implications of generally-capable openended AI. We expect that open-ended foundation models will prove to be an increasingly fertile and safety-critical area of research in the near future.