SQL-PaLM：針對文本到SQL的大型語言模型適應性改進

SQL-PaLM: Improved Large Language ModelAdaptation for Text-to-SQL

May 26, 2023

作者: Ruoxi Sun, Sercan O Arik, Hootan Nakhost, Hanjun Dai, Rajarishi Sinha, Pengcheng Yin, Tomas Pfister

cs.AI

摘要

大型語言模型（LLMs）的一項令人印象深刻的新興能力是生成代碼，包括用於數據庫的結構化查詢語言（SQL）。對於將自然語言文本轉換為SQL查詢的任務，即文本到SQL，LLMs的適應在上下文學習和微調設置中至關重要，具體取決於使用的適應數據量。在本文中，我們提出了一個基於LLMs的文本到SQL模型SQL-PaLM，利用PaLM-2，推動了兩種設置的最新技術。Few-shot SQL-PaLM基於一種基於執行的自一致提示方法，旨在用於文本到SQL，並在Spider上實現了77.3％的測試套件準確性，據我們所知，這是第一個通過微調明顯優於先前最先進技術的模型，提高了4％。此外，我們展示微調的SQL-PALM進一步提高了1％。為了將SQL-PaLM應用於現實情境，我們進一步評估了其在Spider的其他具有挑戰性變體上的穩健性，並展示了SQL-PaLM卓越的泛化能力。此外，通過廣泛的案例研究，我們展示了基於LLMs的文本到SQL的印象深刻的智能能力和各種成功因素。

English

One impressive emergent capability of large language models (LLMs) is generation of code, including Structured Query Language (SQL) for databases. For the task of converting natural language text to SQL queries, Text-to-SQL, adaptation of LLMs is of paramount importance, both in in-context learning and fine-tuning settings, depending on the amount of adaptation data used. In this paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is based on an execution-based self-consistency prompting approach designed for Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our best knowledge is the first to outperform previous state-of-the-art with fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying SQL-PaLM to real-world scenarios we further evaluate its robustness on other challenging variants of Spider and demonstrate the superior generalization capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate the impressive intelligent capabilities and various success enablers of LLM-based Text-to-SQL.