當大型語言模型粗心閱讀表格時：衡量與減少資料引用錯誤

摘要

儘管大型語言模型（LLMs）在表格任務上表現良好，但它們仍會出現資料引用錯誤（DREs），即在理解表格結構的前提下，不正確地引用或遺漏表格中的數值。除了最終答案的準確性外，DREs 直接影響中間推理步驟的正確性與可靠性。然而，先前的研究僅提供了有限且小規模的分析。在本研究中，我們首次針對不同模型與任務，對表格資料引用錯誤進行系統性評估。結果顯示，所有受測模型（參數量為 1.7B 至 20B）均會出現 DREs。此外，我們證明了將資料引用作為評判模型（critic）能顯著提升答案準確率（最高達 12.0%），透過基於評判的過濾與拒絕取樣實現。最後，我們訓練了一個輕量級的 4B 參數評判模型，在檢測分佈內與分佈外 DREs 時，平均 F1 分數達到 78.2%，並能有效協助較大模型的推理。

English

While large language models (LLMs) perform well on table tasks, they still make data referencing errors (DREs), i.e., incorrectly citing or omitting table values, despite understanding the table structure. Beyond final-answer accuracy, DREs directly compromise the correctness and reliability of intermediate reasoning steps. Yet prior studies have only offered limited, small-scale analyses. In this work, we present the first systematic evaluation of tabular data referencing errors across different models and tasks. Our results show that DREs occur across all tested models (1.7B to 20B parameters). Furthermore, we demonstrate that incorporating data referencing as a critic significantly improves answer accuracy up to 12.0%, through critic-based filtering and rejection sampling. Finally, we trained a lightweight 4B-parameter critic model that achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-distribution DREs, and effectively assists inference for larger models.