LEIA：隱式3D關節的潛在視角不變嵌入

摘要

神經輝度場（Neural Radiance Fields，NeRFs）已經在重建靜態場景和3D物體方面引起了革命性變化，提供了前所未有的品質。然而，將NeRFs擴展到建模動態物體或物體關節仍然是一個具有挑戰性的問題。先前的研究通過專注於物體的部分級重建和運動估計來應對這個問題，但它們常常依賴於有關移動部件或物體類別數量的經驗法則，這可能限制了它們的實際應用。在這項研究中，我們介紹了LEIA，一種用於表示動態3D物體的新方法。我們的方法涉及在不同時間步驟或“狀態”下觀察物體，並在當前狀態上條件一個超網絡，用此來對我們的NeRF進行參數化。這種方法使我們能夠為每個狀態學習一個與視角無關的潛在表示。我們進一步展示，通過在這些狀態之間進行插值，我們可以生成在3D空間中以前從未見過的新的關節配置。我們的實驗結果突出了我們的方法在以一種與觀看角度和關節配置無關的方式關節化物體方面的有效性。值得注意的是，我們的方法優於依賴運動信息進行關節註冊的先前方法。

English

Neural Radiance Fields (NeRFs) have revolutionized the reconstruction of static scenes and objects in 3D, offering unprecedented quality. However, extending NeRFs to model dynamic objects or object articulations remains a challenging problem. Previous works have tackled this issue by focusing on part-level reconstruction and motion estimation for objects, but they often rely on heuristics regarding the number of moving parts or object categories, which can limit their practical use. In this work, we introduce LEIA, a novel approach for representing dynamic 3D objects. Our method involves observing the object at distinct time steps or "states" and conditioning a hypernetwork on the current state, using this to parameterize our NeRF. This approach allows us to learn a view-invariant latent representation for each state. We further demonstrate that by interpolating between these states, we can generate novel articulation configurations in 3D space that were previously unseen. Our experimental results highlight the effectiveness of our method in articulating objects in a manner that is independent of the viewing angle and joint configuration. Notably, our approach outperforms previous methods that rely on motion information for articulation registration.

LEIA：隱式3D關節的潛在視角不變嵌入

LEIA: Latent View-invariant Embeddings for Implicit 3D Articulation

摘要

Support