Light-A-Video：透過漸進式光融合實現無需訓練的視頻燈光調整

Light-A-Video: Training-free Video Relighting via Progressive Light Fusion

February 12, 2025

作者: Yujie Zhou, Jiazi Bu, Pengyang Ling, Pan Zhang, Tong Wu, Qidong Huang, Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Anyi Rao, Jiaqi Wang, Li Niu

cs.AI

摘要

最近在影像燈光調整模型方面的進展，受到大規模數據集和預訓練擴散模型的推動，已經實現了一致的照明。然而，視頻燈光調整仍然滯後，主要是由於訓練成本過高以及多樣性和高質量視頻燈光調整數據集的稀缺。將影像燈光調整模型在逐幀應用會導致幾個問題：照明來源不一致和燈光調整外觀不一致，導致生成的視頻中出現閃爍。在這項工作中，我們提出了Light-A-Video，這是一種無需訓練的方法，用於實現時間上平滑的視頻燈光調整。Light-A-Video從影像燈光調整模型中借鑒，引入了兩個關鍵技術來增強照明一致性。首先，我們設計了一個一致燈光關注（CLA）模塊，通過增強自注意力層內的跨幀交互作用，以穩定生成背景照明來源。其次，利用光傳輸獨立性的物理原則，我們在源視頻外觀和燈光調整外觀之間應用線性混合，採用漸進式光融合（PLF）策略，以確保照明中的平滑時間過渡。實驗表明，Light-A-Video改善了燈光調整視頻的時間一致性，同時保持了圖像質量，確保了幀間一致的照明過渡。項目頁面：https://bujiazi.github.io/light-a-video.github.io/。

English

Recent advancements in image relighting models, driven by large-scale datasets and pre-trained diffusion models, have enabled the imposition of consistent lighting. However, video relighting still lags, primarily due to the excessive training costs and the scarcity of diverse, high-quality video relighting datasets. A simple application of image relighting models on a frame-by-frame basis leads to several issues: lighting source inconsistency and relighted appearance inconsistency, resulting in flickers in the generated videos. In this work, we propose Light-A-Video, a training-free approach to achieve temporally smooth video relighting. Adapted from image relighting models, Light-A-Video introduces two key techniques to enhance lighting consistency. First, we design a Consistent Light Attention (CLA) module, which enhances cross-frame interactions within the self-attention layers to stabilize the generation of the background lighting source. Second, leveraging the physical principle of light transport independence, we apply linear blending between the source video's appearance and the relighted appearance, using a Progressive Light Fusion (PLF) strategy to ensure smooth temporal transitions in illumination. Experiments show that Light-A-Video improves the temporal consistency of relighted video while maintaining the image quality, ensuring coherent lighting transitions across frames. Project page: https://bujiazi.github.io/light-a-video.github.io/.