通过阅读中的眼动解码开放式信息获取目标

Decoding Open-Ended Information Seeking Goals from Eye Movements in Reading

May 4, 2025

作者: Cfir Avraham Hadar, Omer Shubi, Yoav Meiri, Yevgeni Berzak

cs.AI

摘要

在阅读过程中，我们往往对文本中的特定信息抱有浓厚兴趣。例如，您阅读本文，可能是出于对大型语言模型（LLMs）在阅读眼动研究中的应用、实验设计的好奇，或者仅仅是想知道“它是否真的有效？”更广泛地说，在日常生活中，人们带着各式各样的文本相关目标去接触文本，这些目标引导着他们的阅读行为。本研究首次探讨了是否能够通过阅读时的眼动数据自动解码出这些开放式的阅读目标。为此，我们引入了目标分类与目标重建任务及其评估框架，并利用大规模的英语阅读眼动追踪数据，涵盖了数百项针对特定文本信息寻求的任务。我们开发并比较了多种结合眼动与文本信息的判别式与生成式多模态LLMs，用于目标分类与重建。实验结果表明，在这两项任务上均取得了显著成功，这暗示了LLMs能够从眼动中提取出关于读者文本特定目标的有价值信息。

English

When reading, we often have specific information that interests us in a text. For example, you might be reading this paper because you are curious about LLMs for eye movements in reading, the experimental design, or perhaps you only care about the question ``but does it work?''. More broadly, in daily life, people approach texts with any number of text-specific goals that guide their reading behavior. In this work, we ask, for the first time, whether open-ended reading goals can be automatically decoded from eye movements in reading. To address this question, we introduce goal classification and goal reconstruction tasks and evaluation frameworks, and use large-scale eye tracking for reading data in English with hundreds of text-specific information seeking tasks. We develop and compare several discriminative and generative multimodal LLMs that combine eye movements and text for goal classification and goal reconstruction. Our experiments show considerable success on both tasks, suggesting that LLMs can extract valuable information about the readers' text-specific goals from eye movements.