SIGNeRF：场景集成生成神经辐射场

摘要

最近图像扩散模型的进展显著改善了高质量图像的生成。结合神经辐射场（NeRFs），它们为3D生成带来了新机遇。然而，大多数生成式3D方法以物体为中心，并将它们应用于编辑现有逼真场景并不是一件简单的事。我们提出了SIGNeRF，这是一种新颖的快速可控的NeRF场景编辑和场景整合物体生成方法。一种新的生成式更新策略确保了编辑后图像的3D一致性，而无需迭代优化。我们发现，深度调节扩散模型固有地具有通过请求图像网格而不是单个视图生成3D一致视图的能力。基于这些见解，我们引入了一组修改图像的多视图参考表。我们的方法根据参考表一致地更新图像集合，并一次性用新生成的图像集对原始NeRF进行精炼。通过利用图像扩散模型的深度调节机制，我们可以对编辑的空间位置进行精细控制，并通过选定区域或外部网格强制形状引导。

English

Advances in image diffusion models have recently led to notable improvements in the generation of high-quality images. In combination with Neural Radiance Fields (NeRFs), they enabled new opportunities in 3D generation. However, most generative 3D approaches are object-centric and applying them to editing existing photorealistic scenes is not trivial. We propose SIGNeRF, a novel approach for fast and controllable NeRF scene editing and scene-integrated object generation. A new generative update strategy ensures 3D consistency across the edited images, without requiring iterative optimization. We find that depth-conditioned diffusion models inherently possess the capability to generate 3D consistent views by requesting a grid of images instead of single views. Based on these insights, we introduce a multi-view reference sheet of modified images. Our method updates an image collection consistently based on the reference sheet and refines the original NeRF with the newly generated image set in one go. By exploiting the depth conditioning mechanism of the image diffusion model, we gain fine control over the spatial location of the edit and enforce shape guidance by a selected region or an external mesh.

SIGNeRF：场景集成生成神经辐射场

SIGNeRF: Scene Integrated Generation for Neural Radiance Fields

摘要

Support