CoVeR：基於覆蓋率的 Token 剪枝，用於 VLM 中的多視角 3D 推理

摘要

將三維場景表示為多視角影像，使 2D VLM 能藉由重用預訓練所得的先驗知識進行三維推理，從而繞過標註三維資料稀缺的問題。然而，這會產生數千個冗餘視覺符元，其成本隨每個視角增加而攀升。現有的視覺符元剪枝器可分為兩類，但在三維多視角設定下各有其限制。學習式重要性方法根據注意力或編碼器特徵對符元進行排序；由於此處的冗餘本質上屬於空間性，這些方法會保留來自少數顯著區域的近重複符元，而使場景的大部分區域未被表示。體素化方法改善了空間覆蓋率，但無法強制確切的符元預算，且當多視角觀測在三維空間中重疊時會趨於飽和，使保留率遠低於目標。我們展示空間覆蓋率與三維推理效能相關，並提出 CoVeR：一種確定性、免訓練的選擇器，僅使用符元座標，而不依賴任何學習訊號。CoVeR 所選出的符元能共同覆蓋場景中的每個區域，並解決這兩類方法的限制：它強制確切的每場景預算、突破體素化飽和平台期，並避免基於學習的重要性方法所產生的近重複選擇。大量實驗顯示，CoVeR 在所有三個三維推理基準上均優於先前的 SOTA，並作為即插即用模組在四種 VLM 上測試時展現泛化能力。值得注意的是，僅使用約 8% 的視覺符元時，它仍保留完整符元效能的 93.5%，在跨基準平均上超越 SOTA 3.9 個百分點。

English

Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotated 3D data. However, it produces thousands of redundant visual tokens whose cost grows with every view. Existing visual token pruners fall into two families, each limited in the 3D multi-view setting. Learned importance methods rank tokens by attention or encoder features; because redundancy here is fundamentally spatial, they keep near-duplicate tokens from a few prominent regions and leave most of the scene unrepresented. Voxelization methods improve spatial coverage but cannot enforce an exact token budget and saturate as multi-view observations overlap in 3D, capping retention well below the target. We show that spatial coverage is associated with 3D reasoning performance and introduce CoVeR, a deterministic, training-free selector that uses only token coordinates, with no learned signals. CoVeR selects tokens that collectively cover every region of the scene, and solves the limitations of both families: it enforces an exact per-scene budget, breaks the voxelization saturation plateau, and avoids the near-duplicate selections of learned importance. Extensive experiments show CoVeR outperforms prior SOTAs on all three 3D reasoning benchmarks and generalizes as a plug-and-play module tested across four VLMs. Notably, with only approx8% of visual tokens, it preserves 93.5% of full-token performance, surpassing SOTA by 3.9 percentage points on average across benchmarks.