SlotLifter：用於學習以物體為中心的輝度場的槽引導特徵提取

SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fields

August 13, 2024

作者: Yu Liu, Baoxiong Jia, Yixin Chen, Siyuan Huang

cs.AI

摘要

從複雜的視覺場景中提煉以物件為中心的抽象能力，是支撐人類級泛化的基礎。儘管在以物件為中心的學習方法方面取得了顯著進展，但在3D物理世界中學習以物件為中心的表示仍然是一個重要挑戰。在這項工作中，我們提出了SlotLifter，一種新穎的以物件為中心的輝度模型，通過插槽引導的特徵提升，共同解決場景重建和分解。這種設計結合了以物件為中心的學習表示和基於圖像的渲染方法，在四個具有挑戰性的合成數據集和四個複雜的真實世界數據集上，提供了最先進的場景分解和新視角合成性能，遠遠優於現有的3D以物件為中心的學習方法。通過大量的消融研究，我們展示了SlotLifter設計的有效性，揭示了潛在未來方向的關鍵見解。

English

The ability to distill object-centric abstractions from intricate visual scenes underpins human-level generalization. Despite the significant progress in object-centric learning methods, learning object-centric representations in the 3D physical world remains a crucial challenge. In this work, we propose SlotLifter, a novel object-centric radiance model addressing scene reconstruction and decomposition jointly via slot-guided feature lifting. Such a design unites object-centric learning representations and image-based rendering methods, offering state-of-the-art performance in scene decomposition and novel-view synthesis on four challenging synthetic and four complex real-world datasets, outperforming existing 3D object-centric learning methods by a large margin. Through extensive ablative studies, we showcase the efficacy of designs in SlotLifter, revealing key insights for potential future directions.