LightLab：基于扩散模型的图像光源控制系统

摘要

我们提出了一种简单而有效的基于扩散模型的方法，用于对图像中的光源进行细粒度、参数化的控制。现有的重光照方法要么依赖多视角输入在推理时执行逆向渲染，要么无法提供对光照变化的显式控制。我们的方法通过在少量真实原始照片对上进行微调，并辅以大规模合成的渲染图像，来激发扩散模型在重光照任务中的照片级真实感先验。我们利用光的线性特性，合成了描绘目标光源或环境光照受控变化的图像对。借助这些数据及适当的微调策略，我们训练了一个能够精确调整光照的模型，实现对光照强度和颜色的显式控制。最后，我们展示了该方法如何实现引人注目的光照编辑效果，并在用户偏好方面超越了现有方法。

English

We present a simple, yet effective diffusion-based method for fine-grained, parametric control over light sources in an image. Existing relighting methods either rely on multiple input views to perform inverse rendering at inference time, or fail to provide explicit control over light changes. Our method fine-tunes a diffusion model on a small set of real raw photograph pairs, supplemented by synthetically rendered images at scale, to elicit its photorealistic prior for relighting. We leverage the linearity of light to synthesize image pairs depicting controlled light changes of either a target light source or ambient illumination. Using this data and an appropriate fine-tuning scheme, we train a model for precise illumination changes with explicit control over light intensity and color. Lastly, we show how our method can achieve compelling light editing results, and outperforms existing methods based on user preference.

LightLab：基于扩散模型的图像光源控制系统

LightLab: Controlling Light Sources in Images with Diffusion Models

摘要

Support