FinTral：一系列GPT-4级别的多模态金融大型语言模型

FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models

February 16, 2024

作者: Gagan Bhatia, El Moatez Billah Nagoudi, Hasan Cavusoglu, Muhammad Abdul-Mageed

cs.AI

摘要

我们介绍了FinTral，这是一套基于Mistral-7b模型构建的最先进的多模态大型语言模型（LLMs），专为金融分析定制。FinTral集成了文本、数值、表格和图像数据。我们通过利用为本研究策划的大量文本和视觉数据集，增强了FinTral的领域特定预训练、指导微调和RLAIF训练。我们还推出了一个包含九项任务和25个数据集用于评估的广泛基准测试，其中包括金融领域的幻觉。我们的FinTral模型通过采用先进的工具和检索方法进行直接偏好优化训练，命名为FinTral-DPO-T&R，展示了出色的零样本性能。在所有任务中，它都优于ChatGPT-3.5，并在九项任务中的五项中超越了GPT-4，标志着人工智能驱动的金融技术取得了重大进展。我们还证明了FinTral有潜力在不同金融环境中实现实时分析和决策。

English

We introduce FinTral, a suite of state-of-the-art multimodal large language models (LLMs) built upon the Mistral-7b model and tailored for financial analysis. FinTral integrates textual, numerical, tabular, and image data. We enhance FinTral with domain-specific pretraining, instruction fine-tuning, and RLAIF training by exploiting a large collection of textual and visual datasets we curate for this work. We also introduce an extensive benchmark featuring nine tasks and 25 datasets for evaluation, including hallucinations in the financial domain. Our FinTral model trained with direct preference optimization employing advanced Tools and Retrieval methods, dubbed FinTral-DPO-T&R, demonstrates an exceptional zero-shot performance. It outperforms ChatGPT-3.5 in all tasks and surpasses GPT-4 in five out of nine tasks, marking a significant advancement in AI-driven financial technology. We also demonstrate that FinTral has the potential to excel in real-time analysis and decision-making in diverse financial contexts.