GPT-4是一位優秀的資料分析師嗎？

摘要

隨著大型語言模型（LLMs）在許多領域和任務中展示了其強大的能力，包括上下文理解、程式碼生成、語言生成、數據敘事等，許多數據分析師可能會擔心他們的工作是否會被人工智慧取代。這個爭議性話題引起了公眾的廣泛關注。然而，我們仍然處於意見分歧的階段，沒有任何明確的結論。受此啟發，我們在本研究中提出了一個研究問題：“GPT-4是否是一位優秀的數據分析師？”並旨在通過進行一對一的比較研究來回答這個問題。具體而言，我們將GPT-4視為一名數據分析師，以執行來自各個領域的數據庫的端對端數據分析。我們提出了一個框架來解決這些問題，通過精心設計GPT-4的提示來進行實驗。我們還設計了幾個特定任務的評估指標，以系統地比較幾位專業的人類數據分析師和GPT-4之間的表現。實驗結果顯示，GPT-4能夠達到與人類可比擬的性能。我們還就我們的結果進行了深入討論，以為在我們得出GPT-4可以取代數據分析師的結論之前，提供進一步研究的啟示。

English

As large language models (LLMs) have demonstrated their powerful capabilities in plenty of domains and tasks, including context understanding, code generation, language generation, data storytelling, etc., many data analysts may raise concerns if their jobs will be replaced by AI. This controversial topic has drawn a lot of attention in public. However, we are still at a stage of divergent opinions without any definitive conclusion. Motivated by this, we raise the research question of "is GPT-4 a good data analyst?" in this work and aim to answer it by conducting head-to-head comparative studies. In detail, we regard GPT-4 as a data analyst to perform end-to-end data analysis with databases from a wide range of domains. We propose a framework to tackle the problems by carefully designing the prompts for GPT-4 to conduct experiments. We also design several task-specific evaluation metrics to systematically compare the performance between several professional human data analysts and GPT-4. Experimental results show that GPT-4 can achieve comparable performance to humans. We also provide in-depth discussions about our results to shed light on further studies before we reach the conclusion that GPT-4 can replace data analysts.

GPT-4是一位優秀的資料分析師嗎？

Is GPT-4 a Good Data Analyst?

摘要

Support