SQL-PaLM：改进的大型语言模型适应文本到SQL

SQL-PaLM: Improved Large Language ModelAdaptation for Text-to-SQL

May 26, 2023

作者: Ruoxi Sun, Sercan O Arik, Hootan Nakhost, Hanjun Dai, Rajarishi Sinha, Pengcheng Yin, Tomas Pfister

cs.AI

摘要

大型语言模型（LLMs）令人印象深刻的新兴能力之一是生成代码，包括数据库的结构化查询语言（SQL）。对于将自然语言文本转换为SQL查询的任务，即文本到SQL，LLMs的调整在上下文学习和微调设置中至关重要，这取决于使用的调整数据量。在本文中，我们提出了一种基于LLMs的文本到SQL模型SQL-PaLM，利用了PaLM-2，推动了这两种设置的最新技术水平。Few-shot SQL-PaLM基于一种基于执行的自一致提示方法，专为文本到SQL设计，其在Spider测试套件上实现了77.3%的准确性，据我们所知，这是首次通过微调显著超越之前的最新技术水平，提高了4%。此外，我们证明了经过微调的SQL-PALM进一步提高了1%。为了将SQL-PaLM应用于现实场景，我们进一步评估了其在Spider的其他具有挑战性变体上的稳健性，并展示了SQL-PaLM卓越的泛化能力。此外，通过广泛的案例研究，我们展示了基于LLMs的文本到SQL的令人印象深刻的智能能力和各种成功因素。

English

One impressive emergent capability of large language models (LLMs) is generation of code, including Structured Query Language (SQL) for databases. For the task of converting natural language text to SQL queries, Text-to-SQL, adaptation of LLMs is of paramount importance, both in in-context learning and fine-tuning settings, depending on the amount of adaptation data used. In this paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is based on an execution-based self-consistency prompting approach designed for Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our best knowledge is the first to outperform previous state-of-the-art with fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying SQL-PaLM to real-world scenarios we further evaluate its robustness on other challenging variants of Spider and demonstrate the superior generalization capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate the impressive intelligent capabilities and various success enablers of LLM-based Text-to-SQL.