大型语言模型对科学发现的影响：使用 GPT-4 进行的初步研究

The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4

November 13, 2023

作者: Microsoft Research AI4Science, Microsoft Azure Quantum

cs.AI

摘要

近年来，自然语言处理方面的突破性进展导致了强大的大型语言模型（LLMs）的出现，这些模型在包括自然语言的理解、生成和翻译以及超越语言处理的任务在内的广泛领域展现出了显著的能力。在本报告中，我们深入探讨了LLMs在科学发现背景下的表现，重点关注了目前最先进的语言模型GPT-4。我们的调查涵盖了涵盖药物发现、生物学、计算化学（密度泛函理论（DFT）和分子动力学（MD））、材料设计以及偏微分方程（PDE）等多样科学领域。评估GPT-4在科学任务上的表现对于揭示其在各种研究领域的潜力、验证其特定领域的专业知识、加速科学进展、优化资源分配、指导未来模型发展以及促进跨学科研究至关重要。我们的探索方法主要包括基于专家案例评估，这些评估提供了关于模型对复杂科学概念和关系的理解的定性见解，以及偶尔进行的基准测试，从而定量评估模型解决明确定义的特定领域问题的能力。我们的初步探索表明，GPT-4在各种科学应用方面展现出了有希望的潜力，显示出其处理复杂问题解决和知识整合任务的能力。总体而言，我们评估了GPT-4的知识库、科学理解、科学数值计算能力以及各种科学预测能力。

English

In recent years, groundbreaking advancements in natural language processing have culminated in the emergence of powerful large language models (LLMs), which have showcased remarkable capabilities across a vast array of domains, including the understanding, generation, and translation of natural language, and even tasks that extend beyond language processing. In this report, we delve into the performance of LLMs within the context of scientific discovery, focusing on GPT-4, the state-of-the-art language model. Our investigation spans a diverse range of scientific areas encompassing drug discovery, biology, computational chemistry (density functional theory (DFT) and molecular dynamics (MD)), materials design, and partial differential equations (PDE). Evaluating GPT-4 on scientific tasks is crucial for uncovering its potential across various research domains, validating its domain-specific expertise, accelerating scientific progress, optimizing resource allocation, guiding future model development, and fostering interdisciplinary research. Our exploration methodology primarily consists of expert-driven case assessments, which offer qualitative insights into the model's comprehension of intricate scientific concepts and relationships, and occasionally benchmark testing, which quantitatively evaluates the model's capacity to solve well-defined domain-specific problems. Our preliminary exploration indicates that GPT-4 exhibits promising potential for a variety of scientific applications, demonstrating its aptitude for handling complex problem-solving and knowledge integration tasks. Broadly speaking, we evaluate GPT-4's knowledge base, scientific understanding, scientific numerical calculation abilities, and various scientific prediction capabilities.