MultiVENT 2.0：一个用于事件中心视频检索的大规模多语言基准测试集。

MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval

October 15, 2024

作者: Reno Kriz, Kate Sanders, David Etter, Kenton Murray, Cameron Carpenter, Kelly Van Ochten, Hannah Recknor, Jimena Guallar-Blasco, Alexander Martin, Ronald Colaianni, Nolan King, Eugene Yang, Benjamin Van Durme

cs.AI

摘要

高效地从大规模多模态集合中检索和合成信息已经成为一个关键挑战。然而，现有的视频检索数据集存在范围限制，主要集中在将描述性但模糊的查询与小规模、专业编辑的、以英语为中心的视频集进行匹配。为了填补这一空白，我们介绍了MultiVENT 2.0，一个大规模、多语言事件中心视频检索基准，包含超过218,000个新闻视频和3,906个针对特定世界事件的查询。这些查询专门针对视频的视觉内容、音频、嵌入式文本和文本元数据中的信息，要求系统利用所有这些来源才能成功完成任务。初步结果显示，最先进的视觉语言模型在这项任务上遇到了重大困难，而替代方法虽然显示出一定的潜力，但仍然不足以充分解决这个问题。这些发现强调了更强大的多模态检索系统的必要性，因为有效的视频检索是通往多模态内容理解和生成任务的关键一步。

English

Efficiently retrieving and synthesizing information from large-scale multimodal collections has become a critical challenge. However, existing video retrieval datasets suffer from scope limitations, primarily focusing on matching descriptive but vague queries with small collections of professionally edited, English-centric videos. To address this gap, we introduce MultiVENT 2.0, a large-scale, multilingual event-centric video retrieval benchmark featuring a collection of more than 218,000 news videos and 3,906 queries targeting specific world events. These queries specifically target information found in the visual content, audio, embedded text, and text metadata of the videos, requiring systems leverage all these sources to succeed at the task. Preliminary results show that state-of-the-art vision-language models struggle significantly with this task, and while alternative approaches show promise, they are still insufficient to adequately address this problem. These findings underscore the need for more robust multimodal retrieval systems, as effective video retrieval is a crucial step towards multimodal content understanding and generation tasks.