AI科学家：迈向完全自动化的开放式科学发现

The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery

August 12, 2024

作者: Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, David Ha

cs.AI

摘要

人工通用智能的一个重大挑战是开发能够进行科学研究和发现新知识的代理程序。尽管前沿模型已被用作辅助人类科学家，例如用于头脑风暴创意、编写代码或预测任务，但它们仍然只完成科学过程的一小部分。本文提出了第一个全自动科学发现的综合框架，使前沿大型语言模型能够独立进行研究并传达其发现。我们介绍了AI科学家，它能够生成新颖的研究思路，编写代码，执行实验，可视化结果，通过撰写完整的科学论文描述其发现，然后运行模拟评审过程进行评估。原则上，这个过程可以重复进行，以开放式方式迭代地发展思路，就像人类科学界一样。我们展示了其多功能性，将其应用于机器学习的三个不同子领域：扩散建模、基于Transformer的语言建模和学习动态。每个思路的实现和发展成完整论文的成本不到每篇15美元。为了评估生成的论文，我们设计并验证了一个自动评审者，我们展示其在评估论文分数方面达到了接近人类的表现。AI科学家可以生成超过自动评审者认可阈值的论文，这一方法标志着机器学习科学发现的新时代的开始：将AI代理程序的变革性益处带入AI研究过程的整个过程，并让我们更接近一个世界，在这个世界中，无尽的创造力和创新可以释放到世界上最具挑战性的问题上。我们的代码在https://github.com/SakanaAI/AI-Scientist 上开源。

English

One of the grand challenges of artificial general intelligence is developing agents capable of conducting scientific research and discovering new knowledge. While frontier models have already been used as aids to human scientists, e.g. for brainstorming ideas, writing code, or prediction tasks, they still conduct only a small part of the scientific process. This paper presents the first comprehensive framework for fully automatic scientific discovery, enabling frontier large language models to perform research independently and communicate their findings. We introduce The AI Scientist, which generates novel research ideas, writes code, executes experiments, visualizes results, describes its findings by writing a full scientific paper, and then runs a simulated review process for evaluation. In principle, this process can be repeated to iteratively develop ideas in an open-ended fashion, acting like the human scientific community. We demonstrate its versatility by applying it to three distinct subfields of machine learning: diffusion modeling, transformer-based language modeling, and learning dynamics. Each idea is implemented and developed into a full paper at a cost of less than $15 per paper. To evaluate the generated papers, we design and validate an automated reviewer, which we show achieves near-human performance in evaluating paper scores. The AI Scientist can produce papers that exceed the acceptance threshold at a top machine learning conference as judged by our automated reviewer. This approach signifies the beginning of a new era in scientific discovery in machine learning: bringing the transformative benefits of AI agents to the entire research process of AI itself, and taking us closer to a world where endless affordable creativity and innovation can be unleashed on the world's most challenging problems. Our code is open-sourced at https://github.com/SakanaAI/AI-Scientist