数据中心化的金融大型语言模型

摘要

大型语言模型（LLMs）在自然语言任务方面表现出潜力，但在直接应用于金融等复杂领域时却遇到困难。LLMs在推理和整合所有相关信息方面存在困难。我们提出了一种以数据为中心的方法，以使LLMs更好地处理金融任务。我们的关键见解是，与其一次性向LLM超载所有内容，预处理和预理解数据更为有效。我们使用多任务提示驱动的微调来创建金融LLM（FLLM），以实现数据预处理和预理解。然而，每个任务的标记数据都很稀缺。为了克服手动注释成本，我们采用推断增强推理（AAR）来通过修改FLLM自身输出的伪标签自动生成训练数据。实验证明，我们基于数据的FLLM与AAR明显优于为原始文本设计的基准金融LLMs，在金融分析和解释任务上达到了最先进水平。我们还开源了一个新的金融分析和解释基准。我们的方法为释放LLMs在复杂现实世界领域潜力提供了一个有前途的途径。

English

Large language models (LLMs) show promise for natural language tasks but struggle when applied directly to complex domains like finance. LLMs have difficulty reasoning about and integrating all relevant information. We propose a data-centric approach to enable LLMs to better handle financial tasks. Our key insight is that rather than overloading the LLM with everything at once, it is more effective to preprocess and pre-understand the data. We create a financial LLM (FLLM) using multitask prompt-based finetuning to achieve data pre-processing and pre-understanding. However, labeled data is scarce for each task. To overcome manual annotation costs, we employ abductive augmentation reasoning (AAR) to automatically generate training data by modifying the pseudo labels from FLLM's own outputs. Experiments show our data-centric FLLM with AAR substantially outperforms baseline financial LLMs designed for raw text, achieving state-of-the-art on financial analysis and interpretation tasks. We also open source a new benchmark for financial analysis and interpretation. Our methodology provides a promising path to unlock LLMs' potential for complex real-world domains.

数据中心化的金融大型语言模型

Data-Centric Financial Large Language Models

摘要

Support