LEIA：隐式3D关节的潜在视图不变嵌入

LEIA: Latent View-invariant Embeddings for Implicit 3D Articulation

September 10, 2024

作者: Archana Swaminathan, Anubhav Gupta, Kamal Gupta, Shishira R. Maiya, Vatsal Agarwal, Abhinav Shrivastava

cs.AI

摘要

神经辐射场（Neural Radiance Fields，NeRFs）已经彻底改变了在3D中重建静态场景和物体的方法，提供了前所未有的质量。然而，将NeRFs扩展到建模动态物体或物体关节仍然是一个具有挑战性的问题。先前的研究通过专注于部分级别的重建和物体的运动估计来解决这个问题，但它们通常依赖于关于移动部件或物体类别数量的启发式方法，这可能限制了它们的实际应用。在这项工作中，我们介绍了LEIA，一种用于表示动态3D物体的新方法。我们的方法涉及在不同的时间步长或“状态”下观察物体，并在当前状态上使用超网络来对我们的NeRF进行参数化。这种方法使我们能够为每个状态学习一个视角不变的潜在表示。我们进一步展示，通过在这些状态之间进行插值，我们可以生成以前未曾见过的3D空间中的新颖关节配置。我们的实验结果突显了我们的方法在关节化物体方面的有效性，这种方法与观察角度和关节配置无关。值得注意的是，我们的方法胜过依赖于运动信息进行关节注册的先前方法。

English

Neural Radiance Fields (NeRFs) have revolutionized the reconstruction of static scenes and objects in 3D, offering unprecedented quality. However, extending NeRFs to model dynamic objects or object articulations remains a challenging problem. Previous works have tackled this issue by focusing on part-level reconstruction and motion estimation for objects, but they often rely on heuristics regarding the number of moving parts or object categories, which can limit their practical use. In this work, we introduce LEIA, a novel approach for representing dynamic 3D objects. Our method involves observing the object at distinct time steps or "states" and conditioning a hypernetwork on the current state, using this to parameterize our NeRF. This approach allows us to learn a view-invariant latent representation for each state. We further demonstrate that by interpolating between these states, we can generate novel articulation configurations in 3D space that were previously unseen. Our experimental results highlight the effectiveness of our method in articulating objects in a manner that is independent of the viewing angle and joint configuration. Notably, our approach outperforms previous methods that rely on motion information for articulation registration.