為何影片依然如此昂貴？影片與視聽大型語言模型中推論效率機制之綜述

摘要

影片理解已迅速演進為影片大型語言模型（VideoLLMs）：這些系統將影片表徵與預訓練大型語言模型結合，並以文字提示作為生成條件。它們在字幕生成、問答、檢索與時序定位上表現優異，但其運算與記憶體成本會隨影格數與上下文長度增加，限制了在即時、行動與資源受限環境中的部署。本綜述涵蓋視覺與視聽 VideoLLMs 的推論效率機制，這些機制能具體降低參數量、每次輸入的 FLOPs、延遲、記憶體，或視覺與音訊符元數量。我們分析跨影格取樣、模態編碼、連接器層級的符元縮減，以及 LLM 預填充與解碼的瓶頸。我們依方法作用的管線階段來組織方法，涵蓋自 2022 年底以來開發的 VideoLLMs，以及仍為當前管線組成部分的早期影格取樣與視覺編碼器機制。我們彙整在可取得的情況下，共享宿主模型與輸入協定下文獻報告的準確度–成本比較，將其與異質的跨論文證據區分，並指出視聽效率與標準化評估的缺口。我們在 https://github.com/momentslab/awesome-efficient-videollm 維護一個儲存庫。

English

Video understanding has rapidly evolved toward video large language models (VideoLLMs): systems that couple video representations with pretrained large language models and condition generation on a textual prompt. Their strong performance on captioning, question answering, retrieval and temporal grounding comes at a computation and memory cost that grows with frame count and context length, limiting deployment in real-time, mobile and resource-constrained settings. This survey covers inference-efficiency mechanisms for visual and audiovisual VideoLLMs that report concrete reductions in parameter count, FLOPs per input, latency, memory, or visual and audio token count. We analyze bottlenecks across frame sampling, modality encoding, connector-level token reduction, and LLM prefilling and decoding. We organize methods by the pipeline stage at which they act, covering VideoLLMs developed since late 2022 together with earlier frame-sampling and vision-encoder mechanisms that remain components of current pipelines. We assemble literature-reported accuracy--cost comparisons under shared host models and input protocols wherever available, distinguish them from heterogeneous cross-paper evidence, and identify gaps in audiovisual efficiency and standardized evaluation. We maintain a repository at https://github.com/momentslab/awesome-efficient-videollm.