DyBluRF：用于模糊单目视频的动态去模糊神经辐射场

DyBluRF: Dynamic Deblurring Neural Radiance Fields for Blurry Monocular Video

December 21, 2023

作者: Minh-Quan Viet Bui, Jongmin Park, Jihyong Oh, Munchurl Kim

cs.AI

摘要

视频视图合成允许从任意视角和时间点创建视觉上吸引人的帧，提供沉浸式的观看体验。神经辐射场，特别是最初为静态场景开发的NeRF，已经推动了各种视频视图合成方法的产生。然而，视频视图合成的挑战在于运动模糊，这是由于曝光期间物体或摄像机移动而导致的，这会妨碍对清晰时空视图的精确合成。为此，我们提出了一种新颖的用于模糊单目视频的动态去模糊NeRF框架，称为DyBluRF，包括交错射线细化（IRR）阶段和基于运动分解的去模糊（MDD）阶段。我们的DyBluRF是首个针对模糊单目视频进行新颖视图合成的方法。IRR阶段联合重建动态3D场景，并改进不准确的摄像机姿势信息，以对抗从给定模糊帧中提取的不精确姿势信息。MDD阶段是一种新颖的逐步潜在锐射预测（ILSP）方法，通过将潜在锐射分解为全局摄像机运动和局部物体运动分量，用于模糊单目视频帧。大量实验结果表明，我们的DyBluRF在质量和数量上优于最近的最先进方法。我们的项目页面包括源代码和预训练模型，可在https://kaist-viclab.github.io/dyblurf-site/公开获取。

English

Video view synthesis, allowing for the creation of visually appealing frames from arbitrary viewpoints and times, offers immersive viewing experiences. Neural radiance fields, particularly NeRF, initially developed for static scenes, have spurred the creation of various methods for video view synthesis. However, the challenge for video view synthesis arises from motion blur, a consequence of object or camera movement during exposure, which hinders the precise synthesis of sharp spatio-temporal views. In response, we propose a novel dynamic deblurring NeRF framework for blurry monocular video, called DyBluRF, consisting of an Interleave Ray Refinement (IRR) stage and a Motion Decomposition-based Deblurring (MDD) stage. Our DyBluRF is the first that addresses and handles the novel view synthesis for blurry monocular video. The IRR stage jointly reconstructs dynamic 3D scenes and refines the inaccurate camera pose information to combat imprecise pose information extracted from the given blurry frames. The MDD stage is a novel incremental latent sharp-rays prediction (ILSP) approach for the blurry monocular video frames by decomposing the latent sharp rays into global camera motion and local object motion components. Extensive experimental results demonstrate that our DyBluRF outperforms qualitatively and quantitatively the very recent state-of-the-art methods. Our project page including source codes and pretrained model are publicly available at https://kaist-viclab.github.io/dyblurf-site/.