GPT 模型能否成為財務分析師？對 ChatGPT 和 GPT-4 在模擬 CFA 考試上的評估

Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams

October 12, 2023

作者: Ethan Callanan, Amarachi Mbakwe, Antony Papadimitriou, Yulong Pei, Mathieu Sibue, Xiaodan Zhu, Zhiqiang Ma, Xiaomo Liu, Sameena Shah

cs.AI

摘要

大型語言模型（LLMs）在各種自然語言處理（NLP）任務上展現出卓越的表現，通常能夠匹敵甚至超越最先進的任務特定模型。本研究旨在評估LLMs在財務推理能力方面的表現。我們利用特許金融分析師（CFA）課程的模擬考試題目，對ChatGPT和GPT-4在財務分析中進行全面評估，考慮零編碼（ZS）、思維鏈（CoT）和少編碼（FS）情境。我們對模型的表現和限制進行了深入分析，並評估它們通過CFA考試的可能性。最後，我們概述了潛在策略和改進措施，以增強LLMs在金融領域的應用性。從這個角度來看，我們希望這項工作為未來的研究開拓道路，持續通過嚴格評估來增強LLMs在財務推理方面的能力。

English

Large Language Models (LLMs) have demonstrated remarkable performance on a wide range of Natural Language Processing (NLP) tasks, often matching or even beating state-of-the-art task-specific models. This study aims at assessing the financial reasoning capabilities of LLMs. We leverage mock exam questions of the Chartered Financial Analyst (CFA) Program to conduct a comprehensive evaluation of ChatGPT and GPT-4 in financial analysis, considering Zero-Shot (ZS), Chain-of-Thought (CoT), and Few-Shot (FS) scenarios. We present an in-depth analysis of the models' performance and limitations, and estimate whether they would have a chance at passing the CFA exams. Finally, we outline insights into potential strategies and improvements to enhance the applicability of LLMs in finance. In this perspective, we hope this work paves the way for future studies to continue enhancing LLMs for financial reasoning through rigorous evaluation.