OmniScope：面向全模態大型語言模型的模態解耦令牌壓縮

摘要

現有的全模態大型語言模型詞元壓縮方法通常依賴單一模態來決定另一模態中應保留的內容。我們證明此假設經常失效：對於同一查詢，音訊與視訊的相關性往往在不同時刻達到峰值。這種跨模態顯著性錯配使得單向引導在激進壓縮下容易丟棄答案關鍵線索。我們提出 OmniScope，一個免訓練的詞元壓縮框架，以查詢作為共享語義錨點，同時分別估計音訊與視訊的相關性。OmniScope 分配模態專屬詞元預算，使用錨點-差值策略剪枝視覺詞元，同時保留全域上下文與時序變化，並以每秒為單位合併音訊詞元，以減少冗餘同時維持時序連續性。在四個音訊-視訊基準測試與兩個 Qwen2.5-Omni 模型規模上，OmniScope 在所有壓縮設定下均達到最佳平均準確率。在25%整體詞元保留率下，它實現高達3.53倍的預填充加速與超過15%的GPU記憶體減少，平均準確率僅下降0.35個百分點。這些結果為 OmniLLM 推論提供了一個簡單的設計原則：跨模態共享查詢，但不共享顯著性估計。程式碼可在 https://github.com/MAC-AutoML/OmniScope 取得。

English

Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different moments. This cross-modal salience mismatch makes unidirectional guidance prone to discarding answer-critical cues under aggressive compression. We propose OmniScope, a training-free token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and video. OmniScope allocates modality-specific token budgets, prunes visual tokens with an anchor-delta strategy that preserves both global context and temporal changes, and merges audio tokens within each second to reduce redundancy while maintaining temporal continuity. Across four audio-video benchmarks and two Qwen2.5-Omni model scales, OmniScope achieves the best average accuracy across all compression settings. At 25% overall token retention, it delivers up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35-point drop in average accuracy. These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available at https://github.com/MAC-AutoML/OmniScope.