ResFields：用於時空信號的殘差神經場。

摘要

神經場是一類訓練用於表示高頻信號的神經網絡，近年來因其在建模複雜的3D數據，特別是大型神經符號距離（SDFs）或輻射場（NeRFs）方面表現出色而受到重視，透過單個多層感知器（MLP）。然而，儘管使用MLP表示信號的能力和簡單性，這些方法在建模大型和複雜的時間信號時仍然面臨挑戰，這是由於MLP的容量有限。在本文中，我們提出了一種有效的方法來解決這一限制，即將時間殘差層納入神經場中，被稱為ResFields，這是一類新型網絡，專門設計用於有效表示複雜的時間信號。我們對ResFields的特性進行了全面分析，並提出了一種矩陣分解技術，以減少可訓練參數的數量並增強泛化能力。重要的是，我們的公式與現有技術無縫集成，並在各種具有挑戰性的任務中持續改進結果：2D視頻近似、通過時間SDFs進行動態形狀建模以及動態NeRF重建。最後，我們通過展示ResFields在從輕量級捕捉系統的稀疏感測輸入中捕捉動態3D場景的有效性，展示了ResFields的實際應用價值。

English

Neural fields, a category of neural networks trained to represent high-frequency signals, have gained significant attention in recent years due to their impressive performance in modeling complex 3D data, especially large neural signed distance (SDFs) or radiance fields (NeRFs) via a single multi-layer perceptron (MLP). However, despite the power and simplicity of representing signals with an MLP, these methods still face challenges when modeling large and complex temporal signals due to the limited capacity of MLPs. In this paper, we propose an effective approach to address this limitation by incorporating temporal residual layers into neural fields, dubbed ResFields, a novel class of networks specifically designed to effectively represent complex temporal signals. We conduct a comprehensive analysis of the properties of ResFields and propose a matrix factorization technique to reduce the number of trainable parameters and enhance generalization capabilities. Importantly, our formulation seamlessly integrates with existing techniques and consistently improves results across various challenging tasks: 2D video approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF reconstruction. Lastly, we demonstrate the practical utility of ResFields by showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory inputs of a lightweight capture system.

ResFields：用於時空信號的殘差神經場。

ResFields: Residual Neural Fields for Spatiotemporal Signals

摘要

Support