FinAudio：面向金融应用的音频大语言模型基准

FinAudio: A Benchmark for Audio Large Language Models in Financial Applications

March 26, 2025

作者: Yupeng Cao, Haohang Li, Yangyang Yu, Shashidhar Reddy Javaji, Yueru He, Jimin Huang, Zining Zhu, Qianqian Xie, Xiao-yang Liu, Koduvayur Subbalakshmi, Meikang Qiu, Sophia Ananiadou, Jian-Yun Nie

cs.AI

摘要

音频大语言模型（AudioLLMs）已获得广泛关注，并在对话、音频理解及自动语音识别（ASR）等音频任务上显著提升了性能。尽管取得了这些进展，但在金融场景中评估AudioLLMs的基准仍属空白，而诸如财报电话会议和CEO演讲等音频数据，是金融分析与投资决策的关键资源。本文中，我们推出了FinAudio，这是首个旨在评估AudioLLMs在金融领域能力的基准。我们首先根据金融领域的独特性质定义了三大任务：1）短金融音频的ASR，2）长金融音频的ASR，以及3）长金融音频的摘要生成。随后，我们分别精选了两个短音频和两个长音频数据集，并开发了一个全新的金融音频摘要数据集，共同构成了FinAudio基准。接着，我们在FinAudio上评估了七种主流AudioLLMs。评估结果揭示了现有AudioLLMs在金融领域的局限性，并为改进AudioLLMs提供了洞见。所有数据集与代码将予以公开。

English

Audio Large Language Models (AudioLLMs) have received widespread attention and have significantly improved performance on audio tasks such as conversation, audio understanding, and automatic speech recognition (ASR). Despite these advancements, there is an absence of a benchmark for assessing AudioLLMs in financial scenarios, where audio data, such as earnings conference calls and CEO speeches, are crucial resources for financial analysis and investment decisions. In this paper, we introduce FinAudio, the first benchmark designed to evaluate the capacity of AudioLLMs in the financial domain. We first define three tasks based on the unique characteristics of the financial domain: 1) ASR for short financial audio, 2) ASR for long financial audio, and 3) summarization of long financial audio. Then, we curate two short and two long audio datasets, respectively, and develop a novel dataset for financial audio summarization, comprising the FinAudio benchmark. Then, we evaluate seven prevalent AudioLLMs on FinAudio. Our evaluation reveals the limitations of existing AudioLLMs in the financial domain and offers insights for improving AudioLLMs. All datasets and codes will be released.