Sol-Attn：通过即时注意力稀疏化加速视频生成推理

摘要

扩散变换器对于高保真视频生成至关重要，但长令牌序列使得注意力机制成为主要的推理瓶颈。免训练的动态稀疏注意力通过仅计算选定的键值块来缓解这一瓶颈，然而现有方法在高效且精确地稀疏化注意力方面存在两个难点：（1）刚性、不可预测且高代价的路由：通过代理分数选取固定比例的前置块会施加固定预算，而保留块以达到目标累积代理概率质量虽能实现动态预算，却可能带来预算失衡；这两种方法均需因计算和物化代理分数而产生不可忽视的开销。（2）有损的保留或丢弃稀疏化：未选中的块被完全丢弃，在激进稀疏化条件下会降低精度。这些局限性促使我们探索更低成本的动态预算路由，同时限制精度损失。本文提出免训练的Sol-Attn（在线注意力稀疏化），该方法在单个在线softmax过程中统一了动态路由、稀疏计算和近似修正，在稀疏注意力中实现了更优的精度-效率权衡。Sol-Attn的核心是通过代理分数复用实现即时块阈值判断——在在线softmax过程中，通过将块代理分数与阈值进行比较来选择关键块。该设计无需物化代理地图即可实现动态可控的块预算，同时直接复用未选中块的代理分数来近似其贡献。在图像和视频生成任务上的实验表明，Sol-Attn推动了免训练稀疏注意力的质量-效率前沿，在保持视觉质量的同时，为视频生成和编辑分别实现了2.1倍和2.3倍的端到端加速。

English

Diffusion transformers are essential for high-fidelity video generation, but long token sequences make attention a dominant inference bottleneck. Training-free dynamic sparse attention alleviates this bottleneck by computing only selected key-value blocks, yet existing methods struggle to sparsify attention both efficiently and accurately for two reasons: (1) Rigid, unpredictable, and costly routing: selecting a fixed fraction of top-ranked blocks by proxy score imposes fixed budgets, whereas retaining blocks to reach a target cumulative proxy probability mass yields dynamic but potentially imbalanced budgets; both incur non-negligible overhead from computing and materializing proxy scores. (2) Lossy keep-or-drop sparsification: unselected blocks are discarded entirely, degrading accuracy under aggressive sparsity. These limitations motivate cheaper dynamic-budget routing while limiting accuracy degradation. In this paper, we introduce training-free Sol-Attn (Sparsifying online attention), which unifies dynamic routing, sparse computation, and approximation correction in a single online-softmax pass, achieving a better accuracy-efficiency trade-off in sparse attention. The core of Sol-Attn is on-the-fly block thresholding with proxy-score reuse, which selects critical blocks by comparing block proxy scores against a threshold during online softmax. This design enables dynamic yet controllable block budgets without materializing the proxy map, while directly reusing the proxy scores of unselected blocks to approximate their contribution. Experiments across image and video generation tasks show that Sol-Attn advances the quality-efficiency frontier of training-free sparse attention, delivering 2.1 times and 2.3 times end-to-end speedups for video generation and editing, respectively, while preserving visual quality.