SlotLifter：用于学习以物体为中心的辐射场的槽引导特征提取

SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fields

August 13, 2024

作者: Yu Liu, Baoxiong Jia, Yixin Chen, Siyuan Huang

cs.AI

摘要

从复杂的视觉场景中提炼出以物体为中心的抽象能力是实现人类级泛化的基础。尽管在以物体为中心的学习方法方面取得了显著进展，但在3D物理世界中学习以物体为中心的表示仍然是一个关键挑战。在这项工作中，我们提出了SlotLifter，一种新颖的以物体为中心的辐射模型，通过基于槽引导的特征提升来共同解决场景重建和分解问题。这种设计将以物体为中心的学习表示和基于图像的渲染方法结合起来，在四个具有挑战性的合成数据集和四个复杂的真实世界数据集上，提供了在场景分解和新视角合成方面的最先进性能，远远超过现有的3D以物体为中心的学习方法。通过大量的消融研究，我们展示了SlotLifter设计的有效性，揭示了潜在未来方向的关键见解。

English

The ability to distill object-centric abstractions from intricate visual scenes underpins human-level generalization. Despite the significant progress in object-centric learning methods, learning object-centric representations in the 3D physical world remains a crucial challenge. In this work, we propose SlotLifter, a novel object-centric radiance model addressing scene reconstruction and decomposition jointly via slot-guided feature lifting. Such a design unites object-centric learning representations and image-based rendering methods, offering state-of-the-art performance in scene decomposition and novel-view synthesis on four challenging synthetic and four complex real-world datasets, outperforming existing 3D object-centric learning methods by a large margin. Through extensive ablative studies, we showcase the efficacy of designs in SlotLifter, revealing key insights for potential future directions.