解碼閱讀中眼動所揭示的開放式資訊尋求目標

Decoding Open-Ended Information Seeking Goals from Eye Movements in Reading

May 4, 2025

作者: Cfir Avraham Hadar, Omer Shubi, Yoav Meiri, Yevgeni Berzak

cs.AI

摘要

在閱讀時，我們往往對文本中的特定信息感興趣。例如，您可能正在閱讀這篇論文，因為您對大型語言模型（LLMs）在閱讀眼動中的應用、實驗設計感到好奇，或者您只關心一個問題：「但它真的有效嗎？」更廣泛地說，在日常生活中，人們會根據各種文本特定的目標來引導其閱讀行為。在本研究中，我們首次探討了是否能夠從閱讀眼動中自動解碼出開放式的閱讀目標。為解答這一問題，我們引入了目標分類與目標重建任務及其評估框架，並利用大規模的英語閱讀眼動數據，涵蓋了數百種文本特定的信息搜尋任務。我們開發並比較了多種結合眼動與文本的判別式與生成式多模態LLMs，用於目標分類與目標重建。實驗結果顯示，這兩項任務均取得了顯著成功，表明LLMs能夠從眼動中提取出有關讀者文本特定目標的寶貴信息。

English

When reading, we often have specific information that interests us in a text. For example, you might be reading this paper because you are curious about LLMs for eye movements in reading, the experimental design, or perhaps you only care about the question ``but does it work?''. More broadly, in daily life, people approach texts with any number of text-specific goals that guide their reading behavior. In this work, we ask, for the first time, whether open-ended reading goals can be automatically decoded from eye movements in reading. To address this question, we introduce goal classification and goal reconstruction tasks and evaluation frameworks, and use large-scale eye tracking for reading data in English with hundreds of text-specific information seeking tasks. We develop and compare several discriminative and generative multimodal LLMs that combine eye movements and text for goal classification and goal reconstruction. Our experiments show considerable success on both tasks, suggesting that LLMs can extract valuable information about the readers' text-specific goals from eye movements.