CLBench-V：从基础锚定到知识获取的多模态上下文学习评估

摘要

现实任务常要求模型从任务特定情境中学习，而非仅依赖预训练知识。尽管近期研究将这一能力称为情境学习，但现有评估主要聚焦于文本情境。然而在实际场景中，待学习的情境往往是多模态的：科学结论通过图表呈现，金融指标分散于转换后的报告中，空间决策依赖于地图、场景或网页。我们提出CLBench-V——一个面向多模态情境学习的基准测试，通过围绕三个维度组织任务来解决情境使用失效点的定位难题：情境基础、新信息应用与新知识学习。CLBench-V融合了经过转换的公开基准数据集与全新构建的数据集，涵盖科学、金融、长文档理解、空间推理及基于网页的视觉问答等领域。为降低构建领域特定情境学习任务的成本，我们进一步采用自动化构建与过滤流程处理新增数据集。在3443个实例和六种近期多模态模型的测试中，最佳整体得分仅达0.2847，表明多模态情境学习仍远未饱和。此外，InternVL3.5-30B-A3B在情境基础与新知识学习维度表现最优，而Qwen3.5-Plus在新信息应用维度领先。我们进一步分析了裁判可靠性、上下文长度、图像数量及代表性失败案例。代码已开源在 https://github.com/IamLihua/CLBench-V。

English

Real-world tasks often require models to learn from task-specific context rather than relying only on pre-trained knowledge. While recent work has highlighted this capability as context learning, existing evaluations mainly focus on textual contexts. In many practical settings, however, the context to be learned from is multimodal: scientific findings are conveyed through figures and tables, financial indicators are scattered across converted reports, and spatial decisions depend on maps, scenes, or web pages. We introduce CLBench-V, a benchmark for multimodal context learning that addresses the difficulty of localizing where context use breaks down by organizing tasks around three dimensions: context grounding, new information application, and new knowledge learning. CLBench-V combines converted public benchmarks with newly constructed datasets spanning domains such as science, finance, long-document understanding, spatial reasoning, and web-based visual question answering. To reduce the cost of constructing domain-specific context-learning tasks, we further use automated construction and filtering procedures for our newly built datasets. Across 3,443 instances and six recent multimodal models, the best overall score is only 0.2847, indicating that multimodal context learning remains far from saturated. Moreover, InternVL3.5-30B-A3B performs best on context grounding and new knowledge learning, while Qwen3.5-Plus performs best on new information application. We further analyze judge reliability, context length, image count, and representative failure cases. Code is available at https://github.com/IamLihua/CLBench-V.