LightLab：利用擴散模型控制圖像中的光源

LightLab: Controlling Light Sources in Images with Diffusion Models

May 14, 2025

作者: Nadav Magar, Amir Hertz, Eric Tabellion, Yael Pritch, Alex Rav-Acha, Ariel Shamir, Yedid Hoshen

cs.AI

摘要

我們提出了一種基於擴散模型的簡單而有效的方法，用於對圖像中的光源進行細粒度、參數化的控制。現有的重光照方法要么依賴於多個輸入視圖在推理時進行逆向渲染，要么無法提供對光照變化的顯式控制。我們的方法在少量真實原始照片對上微調擴散模型，並輔以大規模的合成渲染圖像，以激發其用於重光照的逼真先驗。我們利用光的線性特性來合成描繪目標光源或環境光照受控變化的圖像對。使用這些數據和適當的微調方案，我們訓練了一個模型，能夠實現精確的光照變化，並對光強度和顏色進行顯式控制。最後，我們展示了我們的方法如何實現引人注目的光照編輯效果，並在用戶偏好方面優於現有方法。

English

We present a simple, yet effective diffusion-based method for fine-grained, parametric control over light sources in an image. Existing relighting methods either rely on multiple input views to perform inverse rendering at inference time, or fail to provide explicit control over light changes. Our method fine-tunes a diffusion model on a small set of real raw photograph pairs, supplemented by synthetically rendered images at scale, to elicit its photorealistic prior for relighting. We leverage the linearity of light to synthesize image pairs depicting controlled light changes of either a target light source or ambient illumination. Using this data and an appropriate fine-tuning scheme, we train a model for precise illumination changes with explicit control over light intensity and color. Lastly, we show how our method can achieve compelling light editing results, and outperforms existing methods based on user preference.