FinTagging：一個適用於大型語言模型的財務資訊提取與結構化基準

FinTagging: An LLM-ready Benchmark for Extracting and Structuring Financial Information

May 27, 2025

作者: Yan Wang, Yang Ren, Lingfei Qian, Xueqing Peng, Keyi Wang, Yi Han, Dongji Feng, Xiao-Yang Liu, Jimin Huang, Qianqian Xie

cs.AI

摘要

我们推出了FinTagging，这是首个全面、表格感知的XBRL基准测试，旨在评估大型语言模型（LLMs）在基于XBRL的财务报告中的结构化信息提取与语义对齐能力。与以往将XBRL标记简化为扁平多类分类并仅关注叙述性文本的基准不同，FinTagging将XBRL标记问题分解为两个子任务：FinNI用于财务实体提取，FinCL用于分类驱动的概念对齐。它要求模型在非结构化文本和结构化表格中联合提取事实，并将其与完整的10,000+美国通用会计准则（US-GAAP）分类体系对齐，从而实现真实且细粒度的评估。我们在零样本设置下评估了多种LLMs，系统分析了它们在两个子任务及整体标记准确率上的表现。我们的结果表明，尽管LLMs在信息提取方面展现出强大的泛化能力，但在细粒度概念对齐上存在困难，尤其是在区分紧密相关的分类条目时。这些发现凸显了现有LLMs在完全自动化XBRL标记方面的局限性，并强调了改进语义推理和模式感知建模以满足准确财务披露需求的必要性。代码可在我们的GitHub仓库获取，数据则存放于Hugging Face仓库。

English

We introduce FinTagging, the first full-scope, table-aware XBRL benchmark designed to evaluate the structured information extraction and semantic alignment capabilities of large language models (LLMs) in the context of XBRL-based financial reporting. Unlike prior benchmarks that oversimplify XBRL tagging as flat multi-class classification and focus solely on narrative text, FinTagging decomposes the XBRL tagging problem into two subtasks: FinNI for financial entity extraction and FinCL for taxonomy-driven concept alignment. It requires models to jointly extract facts and align them with the full 10k+ US-GAAP taxonomy across both unstructured text and structured tables, enabling realistic, fine-grained evaluation. We assess a diverse set of LLMs under zero-shot settings, systematically analyzing their performance on both subtasks and overall tagging accuracy. Our results reveal that, while LLMs demonstrate strong generalization in information extraction, they struggle with fine-grained concept alignment, particularly in disambiguating closely related taxonomy entries. These findings highlight the limitations of existing LLMs in fully automating XBRL tagging and underscore the need for improved semantic reasoning and schema-aware modeling to meet the demands of accurate financial disclosure. Code is available at our GitHub repository and data is at our Hugging Face repository.