從 SRA 到 Self-Flow：資料增強抑或自我監督？

摘要

表示对齐已成为加速扩散变压器训练和提升生成质量的有效途径。近年来，如SRA和Self-Flow等自对齐方法通过在扩散模型内部构建对齐，进一步消除对外部预训练编码器的依赖。然而，从SRA到Self-Flow的改进机制——双时间调度——尚未得到充分研究：Self-Flow将其增益归因于不同噪声水平下token之间的交互，其中较干净的token帮助推断较嘈杂的token。在本工作中，我们重新审视这一解释，并探究该增益是否源于沿噪声维度的数据增强。为厘清这些因素，我们引入注意力分离（Attention Separation）方法，该方法在保留与Self-Flow相同的双时间步输入的同时，阻断不同噪声水平下token之间的注意力交互。令人惊讶的是，移除这种交互并不会降低性能，甚至可能提升性能，这表明从SRA到Self-Flow的改进主要来自数据增强。此外，我们证明注意力分离本身通过将单张图像分割成多个有效训练部分来扩展训练数据，从而提供增强效果。基于这些发现，我们将自表示对齐与双时间步和注意力分离增强相结合，并在ImageNet上验证了该设计的有效性。

English

Representation alignment has become an effective way to accelerate diffusion transformer training and improve generation quality. Recent self-alignment methods, such as SRA and Self-Flow, further remove the dependency on external pretrained encoders by constructing alignment within the diffusion model itself. However, the mechanism behind the improvement from SRA to Self-Flow, dual-time scheduling, remains under-examined: Self-Flow attributes its gain to interactions between tokens at different noise levels, where cleaner tokens help infer noisier ones. In this work, we revisit this explanation and ask whether the gain instead comes from data augmentation along the noise dimension. To disentangle these factors, we introduce Attention Separation, which preserves the same dual-timestep input as Self-Flow while blocking attention between tokens assigned to different noise levels. Surprisingly, removing such interaction does not degrade performance and can even improve it, suggesting that the improvement from SRA to Self-Flow mainly comes from data augmentation. Furthermore,We show that Attention Separation itself provides an augmentation effect by splitting a single image into multiple effective training parts to expand the training data. Based on these observations, we combine self-representation alignment with dual-timestep and attention-separation augmentation, and demonstrate the effectiveness of this design on ImageNet.