分割支持，重建殘差：用於影片生成與世界模型的免訓練稀疏注意力

摘要

免訓練的區塊稀疏注意力能加速影片Transformer，但逐行的注意力集中本身並不足以定義一個可執行的稀疏運算子。共享同一區塊路由的查詢，其支撐集可能重疊度很低；然而僅靠保留的注意力質量，無法決定因跳過交互所產生的softmax後誤差。我們證明，分割幾何同時影響匯總的支撐集，以及從稀疏輸出預測剩餘殘差的可預測性。我們提出SparsePR，它將響應耦合分割（Response-Coupled Partitioning）與探針擬合殘差重建（Probe-Fitted Residual Reconstruction）結合起來。取樣查詢的鍵響應形成配對的K/V群組，這些群組的質心產生用於共享路由的查詢-響應座標。接著，少量精確的查詢行在探針殘差所觀察到的輸出子空間內，對稀疏輸出校準一個呼叫特定的仿射校正。在四個異質的影片生成與世界模型中，SparsePR一致地降低了注意力重建誤差。消融實驗顯示，探針擬合貢獻了此降幅的大部分，而響應耦合分割在有限探針預算下降低了硬丟棄誤差並改善了重建。SparsePR在22.0%–26.0%的實際執行配對密度下保持生成品質，同時實現1.48倍至2.61倍的端到端加速。專案頁面：https://pardistaghavi.github.io/SparsePR-website/

English

Training-free block-sparse attention can accelerate video transformers, but row-wise attention concentration does not by itself specify an executable sparse operator. Queries sharing a block route may have poorly overlapping supports, while retained attention mass alone does not determine the post-softmax error from skipped interactions. We show that partition geometry affects both pooled support and the predictability of the remaining residual from the sparse output. We introduce SparsePR, which combines Response-Coupled Partitioning with Probe-Fitted Residual Reconstruction. Sampled-query key responses form paired K/V groups, whose centroids induce query-response coordinates for shared routing. A small set of exact query rows then calibrates a call-specific affine correction from the sparse output within the output subspace observed in the probe residuals. Across four heterogeneous video generation and world models, SparsePR consistently reduces attention-reconstruction error. Ablations show that probe fitting accounts for most of this reduction, while response-coupled partitioning lowers hard-drop error and improves reconstruction under a finite probe budget. SparsePR preserves generation quality at 22.0-26.0% realized executed-pair density while achieving 1.48x-2.61x end-to-end speedups. Project page: https://pardistaghavi.github.io/SparsePR-website/