Table-GPT：針對多樣表格任務進行調整的 GPT

Table-GPT: Table-tuned GPT for Diverse Table Tasks

October 13, 2023

作者: Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, Surajit Chaudhuri

cs.AI

摘要

語言模型，如GPT-3.5和ChatGPT，展示了出色的能力，能夠遵循多樣的人類指示並執行各種任務。然而，通過使用一系列基本的表格理解任務來探測語言模型時，我們觀察到今天的語言模型在許多與表格相關的任務上仍然不夠優化，這可能是因為它們主要是在一維自然語言文本上預先訓練，而關係表格是二維對象。 在這項工作中，我們提出了一個新的「表格微調」範式，我們繼續訓練/微調像GPT-3.5和ChatGPT這樣的語言模型，使用從真實表格合成的多樣化表格任務作為訓練數據，旨在增強語言模型理解表格並執行表格任務的能力。我們展示了我們的結果Table-GPT模型展示了（1）更好的表格理解能力，通過在廣泛的表格任務中持續優於普通的GPT-3.5和ChatGPT，包括保留未見過的任務，以及（2）強大的泛化能力，它能夠回應多樣的人類指示來執行新的表格任務，方式類似於GPT-3.5和ChatGPT。

English

Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable abilities to follow diverse human instructions and perform a wide range of tasks. However, when probing language models using a range of basic table-understanding tasks, we observe that today's language models are still sub-optimal in many table-related tasks, likely because they are pre-trained predominantly on one-dimensional natural-language texts, whereas relational tables are two-dimensional objects. In this work, we propose a new "table-tuning" paradigm, where we continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using diverse table-tasks synthesized from real tables as training data, with the goal of enhancing language models' ability to understand tables and perform table tasks. We show that our resulting Table-GPT models demonstrate (1) better table-understanding capabilities, by consistently outperforming the vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout unseen tasks, and (2) strong generalizability, in its ability to respond to diverse human instructions to perform new table-tasks, in a manner similar to GPT-3.5 and ChatGPT.