FlowMo：基於方差的流動引導技術，用於視頻生成中的連貫運動

摘要

文本到視頻擴散模型在建模時間相關方面，如運動、物理和動態交互，存在顯著的局限性。現有方法通過重新訓練模型或引入外部條件信號來強制時間一致性，以應對這一限制。在本研究中，我們探討是否能夠直接從預訓練模型的預測中提取有意義的時間表示，而無需任何額外的訓練或輔助輸入。我們提出了FlowMo，這是一種新穎的無需訓練的引導方法，它僅利用模型在每個擴散步驟中的自身預測來增強運動連貫性。FlowMo首先通過測量對應於連續幀的潛在變量之間的距離，推導出外觀去偏的時間表示，這突出了模型預測的隱含時間結構。隨後，它通過測量時間維度上的逐塊方差來估計運動連貫性，並在採樣過程中動態引導模型減少這一方差。跨多個文本到視頻模型的廣泛實驗表明，FlowMo在不犧牲視覺質量或提示對齊的情況下，顯著提升了運動連貫性，為增強預訓練視頻擴散模型的時間保真度提供了一種有效的即插即用解決方案。

English

Text-to-video diffusion models are notoriously limited in their ability to model temporal aspects such as motion, physics, and dynamic interactions. Existing approaches address this limitation by retraining the model or introducing external conditioning signals to enforce temporal consistency. In this work, we explore whether a meaningful temporal representation can be extracted directly from the predictions of a pre-trained model without any additional training or auxiliary inputs. We introduce FlowMo, a novel training-free guidance method that enhances motion coherence using only the model's own predictions in each diffusion step. FlowMo first derives an appearance-debiased temporal representation by measuring the distance between latents corresponding to consecutive frames. This highlights the implicit temporal structure predicted by the model. It then estimates motion coherence by measuring the patch-wise variance across the temporal dimension and guides the model to reduce this variance dynamically during sampling. Extensive experiments across multiple text-to-video models demonstrate that FlowMo significantly improves motion coherence without sacrificing visual quality or prompt alignment, offering an effective plug-and-play solution for enhancing the temporal fidelity of pre-trained video diffusion models.

FlowMo：基於方差的流動引導技術，用於視頻生成中的連貫運動

FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation

摘要

Support