ResFields：用于时空信号的残差神经场

摘要

神经场是一类经过训练以表示高频信号的神经网络，近年来因其在建模复杂的3D数据方面表现出色而备受关注，特别是通过单个多层感知器（MLP）模拟大型神经符号距离（SDFs）或辐射场（NeRFs）。然而，尽管使用MLP表示信号具有强大和简单的特性，但由于MLP的容量有限，这些方法在建模大型和复杂的时间信号时仍然面临挑战。本文提出了一种有效的方法来解决这一局限性，即将时间残差层纳入神经场中，命名为ResFields，这是一种专门设计用于有效表示复杂时间信号的新型网络类别。我们对ResFields的特性进行了全面分析，并提出了一种矩阵分解技术，以减少可训练参数的数量并增强泛化能力。重要的是，我们的公式与现有技术无缝集成，并在各种具有挑战性的任务中始终改善结果：2D视频逼近、通过时间SDFs进行动态形状建模以及动态NeRF重建。最后，我们通过展示ResFields在从轻量级捕获系统的稀疏感知输入中捕获动态3D场景方面的有效性，展示了ResFields的实际效用。

English

Neural fields, a category of neural networks trained to represent high-frequency signals, have gained significant attention in recent years due to their impressive performance in modeling complex 3D data, especially large neural signed distance (SDFs) or radiance fields (NeRFs) via a single multi-layer perceptron (MLP). However, despite the power and simplicity of representing signals with an MLP, these methods still face challenges when modeling large and complex temporal signals due to the limited capacity of MLPs. In this paper, we propose an effective approach to address this limitation by incorporating temporal residual layers into neural fields, dubbed ResFields, a novel class of networks specifically designed to effectively represent complex temporal signals. We conduct a comprehensive analysis of the properties of ResFields and propose a matrix factorization technique to reduce the number of trainable parameters and enhance generalization capabilities. Importantly, our formulation seamlessly integrates with existing techniques and consistently improves results across various challenging tasks: 2D video approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF reconstruction. Lastly, we demonstrate the practical utility of ResFields by showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory inputs of a lightweight capture system.

ResFields：用于时空信号的残差神经场

ResFields: Residual Neural Fields for Spatiotemporal Signals

摘要

Support