Seal-3D：用于神经辐射场的交互式像素级编辑

摘要

随着隐式神经表示或神经辐射场（NeRF）的流行，迫切需要编辑方法与隐式3D模型进行交互，用于后期处理重建场景和3D内容创作等任务。尽管先前的研究从不同角度探讨了NeRF编辑，但在编辑灵活性、质量和速度方面存在限制，无法提供直接的编辑响应和即时预览。关键挑战在于构想一种可在本地进行编辑的神经表示，能够直接反映编辑指令并实时更新。为了弥合这一差距，我们提出了一种新的交互式编辑方法和系统，名为Seal-3D，允许用户以像素级和自由方式编辑NeRF模型，采用广泛的类似NeRF的骨干，并即时预览编辑效果。为实现这些效果，我们提出了代理函数，将编辑指令映射到NeRF模型的原始空间，并采用师生训练策略，进行本地预训练和全局微调。构建了一个NeRF编辑系统，展示了各种编辑类型。我们的系统可以以约1秒的交互速度实现引人注目的编辑效果。

English

With the popularity of implicit neural representations, or neural radiance fields (NeRF), there is a pressing need for editing methods to interact with the implicit 3D models for tasks like post-processing reconstructed scenes and 3D content creation. While previous works have explored NeRF editing from various perspectives, they are restricted in editing flexibility, quality, and speed, failing to offer direct editing response and instant preview. The key challenge is to conceive a locally editable neural representation that can directly reflect the editing instructions and update instantly. To bridge the gap, we propose a new interactive editing method and system for implicit representations, called Seal-3D, which allows users to edit NeRF models in a pixel-level and free manner with a wide range of NeRF-like backbone and preview the editing effects instantly. To achieve the effects, the challenges are addressed by our proposed proxy function mapping the editing instructions to the original space of NeRF models and a teacher-student training strategy with local pretraining and global finetuning. A NeRF editing system is built to showcase various editing types. Our system can achieve compelling editing effects with an interactive speed of about 1 second.

Seal-3D：用于神经辐射场的交互式像素级编辑

Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields

摘要

Support