SANeRF-HQ：高质量NeRF的任意分割

摘要

最近，片段任意模型（Segment Anything Model，SAM）展示了零-shot 分割的显著能力，而神经辐射场（Neural Radiance Fields，NeRF）作为一种方法在新视角合成之外也在各种 3D 问题中变得流行。尽管存在将这两种方法纳入 3D 分割的初步尝试，但它们面临着在复杂场景中准确且一致地分割对象的挑战。在本文中，我们介绍了用于在给定场景中实现任何对象高质量 3D 分割的 SANeRF-HQ（Segment Anything for NeRF in High Quality）。SANeRF-HQ 利用 SAM 进行由用户提供提示进行开放世界对象分割，同时利用 NeRF 从不同视角聚合信息。为了克服上述挑战，我们采用密度场和 RGB 相似性来增强聚合过程中分割边界的准确性。强调分割准确性，我们在多个 NeRF 数据集上定量评估了我们的方法，其中提供了高质量的地面真实数据或手动注释。SANeRF-HQ 在 NeRF 对象分割方面显示出明显的质量改进，为对象定位提供了更高的灵活性，并在多个视角下实现了更一致的对象分割。更多信息请访问 https://lyclyc52.github.io/SANeRF-HQ/。

English

Recently, the Segment Anything Model (SAM) has showcased remarkable capabilities of zero-shot segmentation, while NeRF (Neural Radiance Fields) has gained popularity as a method for various 3D problems beyond novel view synthesis. Though there exist initial attempts to incorporate these two methods into 3D segmentation, they face the challenge of accurately and consistently segmenting objects in complex scenarios. In this paper, we introduce the Segment Anything for NeRF in High Quality (SANeRF-HQ) to achieve high quality 3D segmentation of any object in a given scene. SANeRF-HQ utilizes SAM for open-world object segmentation guided by user-supplied prompts, while leveraging NeRF to aggregate information from different viewpoints. To overcome the aforementioned challenges, we employ density field and RGB similarity to enhance the accuracy of segmentation boundary during the aggregation. Emphasizing on segmentation accuracy, we evaluate our method quantitatively on multiple NeRF datasets where high-quality ground-truths are available or manually annotated. SANeRF-HQ shows a significant quality improvement over previous state-of-the-art methods in NeRF object segmentation, provides higher flexibility for object localization, and enables more consistent object segmentation across multiple views. Additional information can be found at https://lyclyc52.github.io/SANeRF-HQ/.

SANeRF-HQ：高质量NeRF的任意分割

SANeRF-HQ: Segment Anything for NeRF in High Quality

摘要

Support