混沌边缘的智能

Intelligence at the Edge of Chaos

October 3, 2024

作者: Shiyang Zhang, Aakash Patel, Syed A Rizvi, Nianchen Liu, Sizhuang He, Amin Karbasi, Emanuele Zappala, David van Dijk

cs.AI

摘要

通过研究基于规则系统的复杂性如何影响训练用于预测这些规则的模型的能力，我们探讨了人工系统中智能行为的出现。我们的研究集中在基本元胞自动机（ECA）上，这是一种简单但功能强大的一维系统，能够产生从琐碎到高度复杂的行为。通过在不同的ECA上训练不同的大型语言模型（LLMs），我们评估了规则行为的复杂性与LLMs展现的智能之间的关系，这体现在它们在下游任务中的表现上。我们的研究发现，具有更高复杂性的规则会导致模型展现出更大的智能，这表现在它们在推理和国际象棋走法预测任务上的表现上。无论是均匀还是周期性系统，甚至是高度混沌的系统，都导致下游表现较差，突显了有利于智能的复杂性的平衡点。我们推测智能源于预测复杂性的能力，并且创造智能可能只需要接触复杂性。

English

We explore the emergence of intelligent behavior in artificial systems by investigating how the complexity of rule-based systems influences the capabilities of models trained to predict these rules. Our study focuses on elementary cellular automata (ECA), simple yet powerful one-dimensional systems that generate behaviors ranging from trivial to highly complex. By training distinct Large Language Models (LLMs) on different ECAs, we evaluated the relationship between the complexity of the rules' behavior and the intelligence exhibited by the LLMs, as reflected in their performance on downstream tasks. Our findings reveal that rules with higher complexity lead to models exhibiting greater intelligence, as demonstrated by their performance on reasoning and chess move prediction tasks. Both uniform and periodic systems, and often also highly chaotic systems, resulted in poorer downstream performance, highlighting a sweet spot of complexity conducive to intelligence. We conjecture that intelligence arises from the ability to predict complexity and that creating intelligence may require only exposure to complexity.