為什麼我打不開抽屜？在零樣本組合動作辨識中緩解物件驅動的捷徑

摘要

零樣本組合動作識別（ZS-CAR）要求辨識由先前觀察到的基本元素組成的新型動詞-物件組合。在本研究中，我們針對一個關鍵的失敗模式進行處理：模型透過物件驅動的捷徑（即依賴標註的物件類別）而非時間證據來預測動詞。我們認為稀疏的組合監督與動詞-物件學習不對稱性可能促進物件驅動的捷徑學習。透過提出的診斷指標分析顯示，現有方法過度擬合訓練共現模式，且未充分利用時間性動詞線索，導致對未見組合的泛化能力薄弱。為解決物件驅動捷徑，我們提出強健組合表徵（RCORE），包含兩個組成部分。共現先驗正則化（CPR）為未見組合添加明確監督，並透過將其視為困難負樣本，對模型進行正則化以對抗頻繁共現先驗。組合時序順序正則化（TORC）則強制模型具備時間順序敏感性，以學習基於時間的動詞表徵。在Sth-com與EK100-com數據集上，RCORE降低了捷徑診斷指標，進而改善了組合泛化能力。

English

Zero-Shot Compositional Action Recognition (ZS-CAR) requires recognizing novel verb-object combinations composed of previously observed primitives. In this work, we tackle a key failure mode: models predict verbs via object-driven shortcuts (i.e., relying on the labeled object class) rather than temporal evidence. We argue that sparse compositional supervision and verb-object learning asymmetry can promote object-driven shortcut learning. Our analysis with proposed diagnostic metrics shows that existing methods overfit to training co-occurrence patterns and underuse temporal verb cues, resulting in weak generalization to unseen compositions. To address object-driven shortcuts, we propose Robust COmpositional REpresentations (RCORE) with two components. Co-occurrence Prior Regularization (CPR) adds explicit supervision for unseen compositions and regularizes the model against frequent co-occurrence priors by treating them as hard negatives. Temporal Order Regularization for Composition (TORC) enforces temporal-order sensitivity to learn temporally grounded verb representations. Across Sth-com and EK100-com, RCORE reduces shortcut diagnostics and consequently improves compositional generalization.