VideoJev：面向视频理解与交互的系统1模型

VideoJev: System One Model for Video Understanding and Interaction

October 13, 2026

作者: Yuhan Zhu, Xiangyu Zeng, Qingyi Si, Chenxu Yang, Yifei Li, Luozheng Qin, Junhao Zhou, Dingyu Yao, Jun Zhang, Chuanyu Qin, Zhiqiu Zhang, Yuandong Yang, Yuqi Xu, Tianyu Zhang, Hongcheng Guo, Jiaqi Wang, Nan Duan, Limin Wang

cs.AI

摘要

视觉智能体必须将视频观测转化为及时决策，以跟踪进展并选择下一步行动。基于新兴的 System One 范式，我们提出 VideoJev，一系列将视频理解与交互建模为结构化决策任务的模型。我们通过用一个共享预测头替代自回归答案解码来适配预训练多模态大语言模型（MLLMs），该预测头根据视频证据对时间区间和文本选项进行评分。这些分数产生事件时间跨度或答案选项，从而统一时间定位与多项选择视频问答。这将 MLLMs 的语义灵活性与智能体可直接使用的高效决策结合起来。随着交互展开，新的目标和中间结果会促使对同一视觉历史提出进一步查询。因此，VideoJev 通过缓存推理（Cached inference）在查询到达时复用视觉计算，并通过联合推理（Joint inference）在单次前向传播中处理多个查询。在三个时间定位基准上，VideoJev-4B 和 VideoJev-35B-A3B 相比各自主干模型分别将平均 mIoU 提升 6.7 和 10.8 个百分点，实测推理加速为 2.8–3.1 倍。在四个视频问答基准上，二者在 Thinking 模式下均达到与其主干模型相当的性能，实测推理加速超过 100 倍。这些结果提供了一个统一、可复用的决策接口，用于将结构化视觉判断组合成复杂、开放式的工作流。

English

Visual agents must turn video observations into timely decisions to track progress and choose what to do next. Building on the emerging System One paradigm, we introduce VideoJev, a family of models that cast video understanding and interaction as structured decision tasks. We adapt pretrained multimodal large language models (MLLMs) by replacing autoregressive answer decoding with a shared head that scores temporal intervals and textual options against video evidence. These scores yield event spans or answer choices, unifying temporal grounding and multiple-choice video question answering. This combines the semantic flexibility of MLLMs with efficient decisions that agents can use directly. As an interaction unfolds, new goals and intermediate results prompt further queries about the same visual history. VideoJev therefore reuses visual computation through Cached inference for queries as they arrive and Joint inference for multiple queries in a single forward pass. Across three temporal grounding benchmarks, VideoJev-4B and VideoJev-35B-A3B improve mean mIoU over their backbones by 6.7 and 10.8 points, respectively, with measured inference speedups of 2.8-3.1times. On four video QA benchmarks, both achieve performance comparable to their backbones in Thinking mode, with measured inference speedups exceeding 100times. These results offer a unified, reusable decision interface for composing structured visual judgments into complex, open-ended workflows.