VideoJev：影片理解與互動的系統一模型

VideoJev: System One Model for Video Understanding and Interaction

October 13, 2026

作者: Yuhan Zhu, Xiangyu Zeng, Qingyi Si, Chenxu Yang, Yifei Li, Luozheng Qin, Junhao Zhou, Dingyu Yao, Jun Zhang, Chuanyu Qin, Zhiqiu Zhang, Yuandong Yang, Yuqi Xu, Tianyu Zhang, Hongcheng Guo, Jiaqi Wang, Nan Duan, Limin Wang

cs.AI

摘要

視覺智能體必須將影片觀察轉化為即時決策，以追蹤進展並選擇下一步行動。立基於新興的 System One 典範，我們提出 VideoJev，一個將影片理解與互動視為結構化決策任務的模型系列。我們調整預訓練多模態大型語言模型（MLLMs），以共享頭取代自迴歸答案解碼，該共享頭根據影片證據對時間區間與文字選項進行評分。這些分數產生事件區間或答案選項，統一了時序定位與多選題影片問答。這結合了 MLLMs 的語意靈活性與智能體可直接使用的高效決策。隨著互動展開，新的目標與中間結果會促成對同一視覺歷史的進一步查詢。因此，VideoJev 透過在查詢到達時進行快取推論，以及在單次前向傳遞中對多個查詢進行聯合推論，來重用視覺運算。在三個時序定位基準上，VideoJev-4B 與 VideoJev-35B-A3B 相較於其主幹模型，平均 mIoU 分別提升 6.7 與 10.8 點，實測推論加速為 2.8 至 3.1 倍。在四個影片問答基準上，兩者在 Thinking 模式下皆達到與其主幹模型相當的效能，實測推論加速超過 100 倍。這些結果提供了一個統一且可重用的決策介面，用於將結構化視覺判斷組成複雜且開放式的工作流程。

English

Visual agents must turn video observations into timely decisions to track progress and choose what to do next. Building on the emerging System One paradigm, we introduce VideoJev, a family of models that cast video understanding and interaction as structured decision tasks. We adapt pretrained multimodal large language models (MLLMs) by replacing autoregressive answer decoding with a shared head that scores temporal intervals and textual options against video evidence. These scores yield event spans or answer choices, unifying temporal grounding and multiple-choice video question answering. This combines the semantic flexibility of MLLMs with efficient decisions that agents can use directly. As an interaction unfolds, new goals and intermediate results prompt further queries about the same visual history. VideoJev therefore reuses visual computation through Cached inference for queries as they arrive and Joint inference for multiple queries in a single forward pass. Across three temporal grounding benchmarks, VideoJev-4B and VideoJev-35B-A3B improve mean mIoU over their backbones by 6.7 and 10.8 points, respectively, with measured inference speedups of 2.8-3.1times. On four video QA benchmarks, both achieve performance comparable to their backbones in Thinking mode, with measured inference speedups exceeding 100times. These results offer a unified, reusable decision interface for composing structured visual judgments into complex, open-ended workflows.