Sol-Attn：透過即時注意力稀疏化加速影片生成推論

摘要

擴散變換器對高保真度影片生成至關重要，但長序列的標記使得注意力機制成為推論的主要瓶頸。無訓練的動態稀疏注意力通過僅計算選定的鍵值區塊來緩解此瓶頸，然而現有方法在有效且準確地稀疏化注意力方面存在兩個難點：（1）剛性、不可預測且成本高昂的路由選擇：根據代理分數選擇固定比例的最高排名區塊會帶來固定預算，而保留區塊以達到目標累積代理機率質量則會產生動態但可能不平衡的預算；這兩種方式都因計算和實例化代理分數而產生不可忽略的開銷。（2）有損的保留或捨棄稀疏化：未被選取的區塊被完全丟棄，在高稀疏度下會降低準確度。這些限制促使我們尋求更廉價的動態預算路由，同時限制精度下降。本文提出無訓練的Sol-Attn（在線稀疏注意力），該方法在單一在線softmax過程中統一了動態路由、稀疏計算與近似校正，實現了稀疏注意力中更好的精度與效率權衡。Sol-Attn的核心是即時區塊門檻值設定與代理分數重用，通過在在線softmax過程中將區塊代理分數與門檻值進行比較來選取關鍵區塊。此設計在無需實例化代理映射圖的情況下達成動態且可控的區塊預算，同時直接重用未選取區塊的代理分數來近似其貢獻。在圖像與影片生成任務上的實驗顯示，Sol-Attn推進了無訓練稀疏注意力的品質與效率邊界，分別為影片生成與編輯帶來了2.1倍與2.3倍的端到端加速，同時保持視覺品質。

English

Diffusion transformers are essential for high-fidelity video generation, but long token sequences make attention a dominant inference bottleneck. Training-free dynamic sparse attention alleviates this bottleneck by computing only selected key-value blocks, yet existing methods struggle to sparsify attention both efficiently and accurately for two reasons: (1) Rigid, unpredictable, and costly routing: selecting a fixed fraction of top-ranked blocks by proxy score imposes fixed budgets, whereas retaining blocks to reach a target cumulative proxy probability mass yields dynamic but potentially imbalanced budgets; both incur non-negligible overhead from computing and materializing proxy scores. (2) Lossy keep-or-drop sparsification: unselected blocks are discarded entirely, degrading accuracy under aggressive sparsity. These limitations motivate cheaper dynamic-budget routing while limiting accuracy degradation. In this paper, we introduce training-free Sol-Attn (Sparsifying online attention), which unifies dynamic routing, sparse computation, and approximation correction in a single online-softmax pass, achieving a better accuracy-efficiency trade-off in sparse attention. The core of Sol-Attn is on-the-fly block thresholding with proxy-score reuse, which selects critical blocks by comparing block proxy scores against a threshold during online softmax. This design enables dynamic yet controllable block budgets without materializing the proxy map, while directly reusing the proxy scores of unselected blocks to approximate their contribution. Experiments across image and video generation tasks show that Sol-Attn advances the quality-efficiency frontier of training-free sparse attention, delivering 2.1 times and 2.3 times end-to-end speedups for video generation and editing, respectively, while preserving visual quality.