SeaCache：面向扩散模型加速的光谱演化感知缓存系统

摘要

扩散模型作为视觉生成任务的强大基础架构，其固有的序列化去噪过程导致推理速度缓慢。现有加速方法通过缓存并基于相邻时间步的特征距离复用中间输出，但这类缓存策略通常依赖原始特征差异，未能解耦内容与噪声。这种设计忽视了频谱演化规律——低频结构早期形成而高频细节后期细化。我们提出频谱演化感知缓存（SeaCache），这是一种无需重新训练的缓存调度方案，其复用决策基于频谱对齐的表征。通过理论与实证分析，我们推导出频谱演化感知（SEA）滤波器，能在抑制噪声的同时保留内容相关成分。采用经SEA滤波的输入特征估计冗余度，可生成动态调度策略，既能适应内容特性又遵循扩散模型的频谱先验。在多样化视觉生成模型及基线方法上的大量实验表明，SeaCache实现了最优的延迟-质量权衡。

English

Diffusion models are a strong backbone for visual generation, but their inherently sequential denoising process leads to slow inference. Previous methods accelerate sampling by caching and reusing intermediate outputs based on feature distances between adjacent timesteps. However, existing caching strategies typically rely on raw feature differences that entangle content and noise. This design overlooks spectral evolution, where low-frequency structure appears early and high-frequency detail is refined later. We introduce Spectral-Evolution-Aware Cache (SeaCache), a training-free cache schedule that bases reuse decisions on a spectrally aligned representation. Through theoretical and empirical analysis, we derive a Spectral-Evolution-Aware (SEA) filter that preserves content-relevant components while suppressing noise. Employing SEA-filtered input features to estimate redundancy leads to dynamic schedules that adapt to content while respecting the spectral priors underlying the diffusion model. Extensive experiments on diverse visual generative models and the baselines show that SeaCache achieves state-of-the-art latency-quality trade-offs.

SeaCache：面向扩散模型加速的光谱演化感知缓存系统

SeaCache: Spectral-Evolution-Aware Cache for Accelerating Diffusion Models

摘要

Support