OmniScope：面向全模态大语言模型的模态解耦令牌压缩

摘要

现有的面向全模态大语言模型的token压缩方法通常依赖单一模态来决定另一模态中应保留的内容。我们证明这一假设经常失效：对于同一查询，音频和视频的相关性往往在不同时刻达到峰值。这种跨模态显著性不匹配使得单向引导在激进压缩下容易丢弃回答关键线索。我们提出OmniScope，一种免训练的token压缩框架，以查询作为共享语义锚点，同时分别评估音频和视频的相关性。OmniScope分配模态特定的token预算，采用锚点-增量策略修剪视觉token，既保留全局上下文又保留时间变化，并在每秒内合并音频token以减少冗余同时维持时间连续性。在四个音视频基准测试和两种Qwen2.5-Omni模型规模下，OmniScope在所有压缩设置中均取得最佳平均准确率。在整体token保留率为25%时，它实现了高达3.53倍的预填充加速和超过15%的GPU显存减少，平均准确率仅下降0.35个百分点。这些结果表明了一个简单的OmniLLM推理设计原则：跨模态共享查询，但不共享显著性估计。代码可在https://github.com/MAC-AutoML/OmniScope获取。

English

Existing token compression methods for omnimodal large language models typically rely on one modality to determine what to retain in the other. We show that this assumption often breaks down: for the same query, audio and video relevance often peaks at different moments. This cross-modal salience mismatch makes unidirectional guidance prone to discarding answer-critical cues under aggressive compression. We propose OmniScope, a training-free token compression framework that uses the query as a shared semantic anchor while estimating relevance separately for audio and video. OmniScope allocates modality-specific token budgets, prunes visual tokens with an anchor-delta strategy that preserves both global context and temporal changes, and merges audio tokens within each second to reduce redundancy while maintaining temporal continuity. Across four audio-video benchmarks and two Qwen2.5-Omni model scales, OmniScope achieves the best average accuracy across all compression settings. At 25% overall token retention, it delivers up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35-point drop in average accuracy. These results suggest a simple design principle for OmniLLM inference: share the query across modalities, but not the salience estimates. The code is available at https://github.com/MAC-AutoML/OmniScope.