視覺文件理解中無需座標或區域標籤的證據歸因

摘要

可靠的視覺文件理解要求模型將每個答案歸因於支持該答案的證據區域。近期的基準測試與系統透過座標介面來實現此步驟：模型輸出標記文件中證據區域的邊界框座標。在此介面下，視覺語言模型即使回答正確，也時常無法識別正確區域，此情況稱為「歸因幻覺」。我們提出一項研究，探討此失敗是否部分受限於模型透過座標所能表達的範圍。在一個經驗證的雙語 CiteVQA 子集上，我們比較座標介面與語言介面：後者中模型僅輸出文字，逐字引用其證據，並由多模態檢索器返回每個引用的位置，作為版面解析器所提出的頁面區域（表格與圖形則透過其標題或註釋引用）；此比較在六個開放式視覺語言模型上重複進行。與座標介面相比，證據召回率從最高 8 個百分點提升至 26 到 47 之間，且幻覺率約減半，而答案品質幾乎不變。基於此比較，我們將相同的引用與檢索流程用作訓練支架：由於針對長文件收集區域級證據標籤成本高昂，我們引入一種 GRPO 方法，其獎勵為評判者對黃金答案及檢索區域裁切內容的解讀，訓練模型在無任何區域標籤的情況下引用更好的證據，並將 8B 基礎模型嚴格的歸因準確率從 22.4 提升至 33.8。這些發現指出一條實際路徑，可在無需座標介面且無需昂貴的區域級監督下改善歸因表現。

English

Reliable visual document understanding requires a model to attribute each answer to the evidence regions that support it. Recent benchmarks and systems express this step through a coordinate interface: the model outputs the coordinates of bounding boxes that mark the evidence regions in the document. Under this interface, vision-language models often fail to identify the right regions even when the answer is correct, a failure known as Attribution Hallucination. We present a study that investigates whether this failure is partially limited by what the model can express through coordinates. On a verified bilingual CiteVQA subset, we compare the coordinate interface with a language interface in which the model outputs only text, quoting its evidence verbatim, and a multimodal retriever returns the location of each quote as a page region proposed by a layout parser (tables and figures are quoted through their captions or notes); the comparison is repeated over six open vision-language models. Compared with the coordinate interface, evidence recall rises from at most 8 points to between 26 and 47 and the hallucination rate roughly halves, with little change in answer quality. Building on this comparison, we use the same quote-and-retrieve pipeline as a training scaffold: because region-level evidence labels are expensive to collect for long documents, we introduce a GRPO recipe whose reward is a judge's reading of the gold answer and crops of the retrieved regions, training the model to quote better evidence without any region labels and raising an 8B backbone's strict attributed accuracy from 22.4 to 33.8. These findings indicate a practical path to improve attribution"without a coordinate interface and without costly region-level supervision.