大语言模型的金融推理是否可信？基于长期陈述的现实世界测试

摘要

大型语言模型（LLMs）是否具备真正的结构性推理，抑或只是依赖表层模式匹配？金融领域对数值精度与长上下文上的多步逻辑要求严苛，是理想的试验场。然而，现有基准主要依赖基于截断表格的选择题或单跳问答，忽视了复杂的跨报表动态关系与跨期去累计，难以反映真实工业场景的复杂性。为弥补这一空白，我们提出 FinIndices——一个在未截断的财务报表（最长32K token）上评估数据处理保真度的大规模基准。借助带有对抗性陷阱的自动化合成流水线，FinIndices 涵盖单一指标计算与表格式指标编制两类任务，用以测试复杂的领域、时间与口径推理。我们的评估揭示了 LLM 的两大严重缺陷。其一，“知识瓶颈”：尽管模型在预训练阶段记忆了公式，其模式匹配依然脆弱。移除显式公式提示后，性能急剧下降（例如，Gemini-3.1-Pro 在表格任务上从 70.70% 跌至 38.22%），暴露出跨期去累计与存量-流量口径不匹配方面的致命问题。其二，“结构瓶颈”：生成多指标、多周期表格所带来的高强度认知负荷会持续消耗推理能力。在结构压力下，原本能够完美执行孤立推导的 LLM 会退化为浅层启发式，例如取用错误的相邻列，或用偷懒的字面算术替代深度会计调整。最后，监督微调（SFT）在零提示条件下带来了显著增益（单一指标 +8.54%，表格 +3.82%），验证了结构化逻辑可以通过以数据为中心的对齐得到部分恢复。

English

Do Large Language Models (LLMs) possess genuine structural reasoning, or merely rely on surface-level pattern matching? The financial domain, demanding numerical precision and multi-step logic over long contexts, is an ideal testbed. Existing benchmarks fail to capture real-world industrial complexity, predominantly relying on multiple-choice questions or single-hop QA over cropped tables while ignoring intricate cross-statement dynamics and temporal de-cumulation. To bridge this gap, we introduce FinIndices, a large-scale benchmark evaluating data-processing fidelity over uncropped financial statements (up to 32K tokens). Utilizing an automated synthesis pipeline with adversarial traps, FinIndices encompasses Single-Index computation and Table-Index tabulation to test complex domain, temporal, and caliber reasoning. Our evaluation reveals two severe LLM vulnerabilities. First, a "Knowledge Bottleneck": despite memorizing formulas during pre-training, models demonstrate fragile pattern matching. Removing explicit formula hints causes performance to collapse (e.g., Gemini-3.1-Pro drops from 70.70% to 38.22% on table tasks), exposing fatal flaws in temporal de-cumulation and stock-flow caliber mismatch. Second, a "Structural Bottleneck": the intense cognitive load of generating multi-metric, multi-period tables actively drains reasoning capacity. Under structural pressure, LLMs that flawlessly execute isolated derivations regress to shallow heuristics, such as fetching incorrect adjacent columns or substituting deep accounting adjustments with lazy literal arithmetic. Finally, Supervised Fine-Tuning (SFT) yields substantial zero-hint gains (+8.54% Single, +3.82% Table), validating that structured logic can be partially restored via data-centric alignment.