在视频中定位一切：重新思考高效的生成式时空视频定位

摘要

时空视频定位（STVG）要求模型识别所指事件发生的时间，并在该时段内定位目标实体。现有的多模态大语言模型通常以自回归方式串行生成密集定位轨迹，导致解码延迟随轨迹管长度增长，且定位误差会随时间传播。我们提出并行管解码（PTD），这是一种生成式建模方法，将定位分解为时序块及随后的时间条件空间块，并同步解码。这消除了标记级与轨迹级依赖，将串行解码深度降为固定的1+1轮，与轨迹管长度无关。为实现并行空间生成，我们引入解耦块注意力，在保留共享视频-查询上下文的同时消除跨框依赖，并配合面向时序边界与空间几何的定位感知策略优化。在VidSTG上，与标准自回归解码相比，PTD将轨迹管完成延迟降低79倍，将空间解码吞吐量提升92倍，同时提高了定位准确度。凭借紧凑的4B骨干网络，我们的模型在VidSTG和HC-STVG上表现优异，并可零样本泛化至时序定位、定位视频问答和指代视频目标跟踪任务。我们的结果表明，并行轨迹管生成是视频中自回归定位的一种高效且有效的替代方案。

English

Spatio-temporal video grounding (STVG) requires models to identify when a referred event occurs and localize the target entity throughout that interval. Existing multimodal large language models typically serialize dense localization trajectories autoregressively, causing decoding latency to grow with tube length and allowing localization errors to propagate across time. We introduce Parallel Tube Decoding (PTD), a generative formulation that decomposes grounding into a temporal block followed by time-conditioned spatial blocks decoded simultaneously. This removes both token-level and trajectory-level dependencies, reducing the sequential decoding depth to a fixed 1 + 1 rounds, independent of tube length. To enable parallel spatial generation, we introduce Decoupled Block Attention, which preserves access to shared video-query context while eliminating cross-box dependencies, together with localization-aware policy optimization for temporal boundaries and spatial geometry. On VidSTG, PTD reduces Tube Completion Latency by 79x and increases spatial decoding throughput by 92x over standard autoregressive decoding, while also improving grounding accuracy. With a compact 4B backbone, our model performs favorably well on VidSTG and HC-STVG, and generalizes zero-shot to temporal grounding, grounded VideoQA, and referring video object tracking. Our results show parallel tube generation is an efficient and effective alternative to autoregressive localization in videos.