視頻中定位一切：重新思考高效生成式時空視頻定位

摘要

時空影片定位（STVG）要求模型識別所指事件發生的時間，並在該時段內定位目標實體。現有的多模態大型語言模型通常以自迴歸方式序列化稠密定位軌跡，導致解碼延遲隨時空管長度增加，並使定位誤差隨時間傳播。我們提出並行時空管解碼（Parallel Tube Decoding, PTD），這是一種生成式框架，將定位分解為一個時間區塊，後接多個時間條件化的空間區塊並同時解碼。這同時消除了標記層級與軌跡層級的依賴，將順序解碼深度降至固定的 1+1 輪，且與時空管長度無關。為實現並行空間生成，我們引入解耦區塊注意力（Decoupled Block Attention），在保留共享影片-查詢上下文存取的同時消除跨框依賴，並搭配針對時間邊界與空間幾何的定位感知策略最佳化。在 VidSTG 上，與標準自迴歸解碼相比，PTD 將時空管完成延遲降低 79 倍，空間解碼吞吐量提升 92 倍，同時也提升定位準確度。在緊湊型 4B 骨幹下，我們的模型在 VidSTG 與 HC-STVG 上表現良好，並能零樣本泛化至時間定位、定位式影片問答及指稱性影片物件追蹤。我們的結果顯示，並行時空管生成是影片中自迴歸定位的一種高效且有效的替代方案。

English

Spatio-temporal video grounding (STVG) requires models to identify when a referred event occurs and localize the target entity throughout that interval. Existing multimodal large language models typically serialize dense localization trajectories autoregressively, causing decoding latency to grow with tube length and allowing localization errors to propagate across time. We introduce Parallel Tube Decoding (PTD), a generative formulation that decomposes grounding into a temporal block followed by time-conditioned spatial blocks decoded simultaneously. This removes both token-level and trajectory-level dependencies, reducing the sequential decoding depth to a fixed 1 + 1 rounds, independent of tube length. To enable parallel spatial generation, we introduce Decoupled Block Attention, which preserves access to shared video-query context while eliminating cross-box dependencies, together with localization-aware policy optimization for temporal boundaries and spatial geometry. On VidSTG, PTD reduces Tube Completion Latency by 79x and increases spatial decoding throughput by 92x over standard autoregressive decoding, while also improving grounding accuracy. With a compact 4B backbone, our model performs favorably well on VidSTG and HC-STVG, and generalizes zero-shot to temporal grounding, grounded VideoQA, and referring video object tracking. Our results show parallel tube generation is an efficient and effective alternative to autoregressive localization in videos.