AnyGroundBench：視覺語言模型中影片定位的專業領域基準

摘要

視覺-語言模型（VLM）在時空影片定位（STVG）任務中展現出極大潛力。然而，目前的評估標準主要侷限於一般日常生活基準上的零樣本測試。這與專業領域中的實際應用產生了嚴重脫節——在此類領域中，模型無可避免地會遭遇罕見的視覺概念與複雜的時空動態。由於對無限數據分佈進行全面預訓練不可行，因此模型適應新領域的能力至關重要。為填補此缺口，我們提出 AnyGroundBench——一個專為領域適應設計的基準，旨在將 STVG 評估典範從靜態的零樣本測試轉向嚴謹的領域適應。AnyGroundBench 針對五個專業領域（動物、工業、運動、手術與公共安全），將新拍攝的影片（例如專家標註的小鼠行為）與既有數據集配對，並透過密集、高保真的時空標註予以統一。關鍵在於，該基準提供了專門的訓練子集，以系統性地衡量領域適應能力。我們廣泛評估了 15 個最先進的 VLM，在實際的計算限制下檢視其零樣本泛化能力與情境學習（ICL）能力。最終，我們的研究結果顯示，當面對專業領域時，現有模型無論在零樣本還是在基於 ICL 的適應中皆宣告失敗，暴露了時空推理上的關鍵缺陷——這些缺陷正是未來研究必須解決的課題。

English

Vision-Language Models (VLMs) have demonstrated immense promise in Spatio-Temporal Video Grounding (STVG). However, current evaluation protocols are largely confined to zero-shot assessments on general, daily-life benchmarks. This creates a critical disconnect from real-world applications in specialized fields, where models inevitably encounter rare visual concepts and complex spatio-temporal dynamics. Since exhaustive pre-training across infinite data distributions is infeasible, the ability to adapt to novel domains is essential. To bridge this gap, we introduce AnyGroundBench, a domain-adaptation benchmark designed to shift the STVG evaluation paradigm from static zero-shot testing to rigorous domain adaptation. Targeting five specialized domains (animal, industry, sports, surgery, and public security), AnyGroundBench pairs newly captured videos such as expert-annotated mouse behaviors with established datasets, unifying them through dense, high-fidelity spatio-temporal annotations. Crucially, the benchmark provides dedicated training subsets to systematically measure domain adaptability. We extensively evaluate 15 state-of-the-art VLMs, assessing their zero-shot generalization and In-Context Learning (ICL) capabilities under practical computational constraints. Ultimately, our findings reveal that current models fail in both zero-shot and ICL-based adaptation when confronted with specialized domains, exposing critical flaws in spatio-temporal reasoning that future research must address.