BRDFusion：物理与生成结合的城市场景逆渲染

摘要

从捕获视频中逆渲染城市场景可实现众多应用，包括内容生成和自动驾驶模拟。基于物理的渲染方法遵循并控制光照物理规律，但存在重建与渲染伪影问题。生成模型虽能产生逼真视频，但一致性和可控性有限。我们提出BRDFusion——一个结合两种互补模型的统一框架，用于逆渲染与正向渲染。具体而言，BRDFusion通过物理建模恢复显式、一致的场景属性，并利用生成先验缓解优化歧义。在正向渲染阶段，物理模型基于场景配置提供可控渲染，生成模型则负责降噪与伪影修复。因此，本方法能在实现精准控制的同时生成高质量视频，在真实与合成场景中均优于基线方法。此外，BRDFusion支持新视角重打光、夜间模拟以及动态物体插入/编辑。项目主页：https://shigon255.github.io/brdfusion-page/

English

Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation. Physically-based rendering methods follow and control lighting physics, but suffer from reconstruction and rendering artifacts. While generative models produce realistic videos, they offer limited consistency and controllability. We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering. Specifically, BRDFusion recovers explicit, consistent scene properties with physical modeling and alleviates optimization ambiguity with generative priors. During forward rendering, the physical model provides controllable rendering from the scene configuration, and the generative model denoises and fixes artifacts. Therefore, our method produces high-quality videos while allowing precise control, outperforming baselines in real and synthetic scenes. Moreover, BRDFusion supports novel-view relighting, night simulation, and dynamic object insertion/editing. Project page: https://shigon255.github.io/brdfusion-page/