从合成到去除：物理驱动的反射模拟与基于扩散模型的视频去反射

摘要

通过玻璃拍摄的视频常常包含反射，这些反射会降低视觉质量并干扰下游视觉任务。尽管单幅图像反射去除已被广泛研究，但视频反射去除仍很大程度上未被充分探索，原因在于缺乏成对视频数据、时序连贯的去除模型以及专门的评估基准。我们提出了一个闭环框架，统一了基于物理的反射模拟、基于扩散的视频去反射以及基准评估。我们的 S2R-Synthesis 管线通过在结构空间中执行基于物理的增强，并使用训练好的视频扩散渲染器渲染逼真的反射视频，来生成成对的含反射与无反射视频；该增强模拟了关键的玻璃相关效应，包括粗糙度引起的模糊、厚度引起的重影以及反射率变化。基于合成数据，我们提出了 S2R-Removal，这是首个基于扩散的视频反射去除模型，它通过反射感知的潜在适配和单步像素-几何细化来适配预训练的视频扩散先验，在单次去噪步骤中恢复干净的透射。我们进一步构建了 S2R-Bench，这是首个用于视频反射去除的基准，支持全参考评估和真实世界的人类感知评估。在 S2R-Bench 和多个公开图像基准上的实验表明，该方法取得了最先进的性能，且推理速度甚至快于非扩散基线，同时验证了 S2R-Synthesis 的有效性。项目页面：https://codingwzp.github.io/VideoDereflection_S2R。

English

Videos captured through glass often contain reflections that degrade visual quality and interfere with downstream vision tasks. Although single-image reflection removal has been extensively studied, video reflection removal remains largely underexplored due to the lack of paired video data, temporally coherent removal models, and dedicated evaluation benchmarks. We present a closed-loop framework that unifies physics-grounded reflection simulation, diffusion-based video dereflection, and benchmark evaluation. Our S2R-Synthesis pipeline generates paired reflected and reflection-free videos by performing physics-grounded augmentation in the structure space and rendering realistic reflected videos with a trained video diffusion renderer; the augmentation models key glass-related effects including roughness-induced blur, thickness-induced ghosting, and reflectance variation. Based on the synthesized data, we introduce S2R-Removal, the first diffusion-based video reflection removal model, which adapts a pretrained video diffusion prior through reflection-aware latent adaptation and one-step pixel-geometric refinement, recovering the clean transmission in a single denoising step. We further build S2R-Bench, the first benchmark for video reflection removal, supporting both full-reference evaluation and real-world human perceptual assessment. Experiments on S2R-Bench and multiple public image benchmarks demonstrate state-of-the-art performance and faster inference than even non-diffusion baselines, and validate the effectiveness of S2R-Synthesis. Project page: https://codingwzp.github.io/VideoDereflection_S2R.