学习异构场景专家混合模型以实现大规模神经辐射场

Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural Radiance Fields

May 4, 2025

作者: Zhenxing Mi, Ping Yin, Xue Xiao, Dan Xu

cs.AI

摘要

近期在大规模场景下的NeRF方法强调了场景分解对于可扩展NeRF的重要性。尽管已实现合理的可扩展性，但仍存在几个关键问题尚未探索，即可学习的分解、场景异质性建模以及建模效率。本文中，我们提出了Switch-NeRF++，一种异构哈希专家混合（HMoHE）网络，该网络在一个统一框架内解决了这些挑战。它是一个高度可扩展的NeRF，能够以端到端的方式高效学习大规模场景的异构分解和异构NeRF。在我们的框架中，一个门控网络学习分解场景并将3D点分配给专门的NeRF专家。通过我们提出的稀疏门控专家混合（MoE）NeRF框架，该门控网络与专家共同优化。我们引入了一个基于哈希的门控网络和不同的异构哈希专家。基于哈希的门控高效学习大规模场景的分解。不同的异构哈希专家由不同分辨率范围的哈希网格组成，能够有效学习不同场景部分的异构表示。这些设计选择使我们的框架成为面向现实世界大规模场景建模的端到端且高度可扩展的NeRF解决方案，实现了质量与效率的双重提升。我们在现有的大规模NeRF数据集和来自UrbanBIS的超大规模场景（>6.5平方公里）新数据集上评估了我们的准确性和可扩展性。大量实验表明，我们的方法能够轻松扩展到各种大规模场景，并达到最先进的场景渲染精度。此外，与Switch-NeRF相比，我们的方法在训练速度上提升了8倍，渲染速度提升了16倍，显著提高了效率。代码将在https://github.com/MiZhenxing/Switch-NeRF 发布。

English

Recent NeRF methods on large-scale scenes have underlined the importance of scene decomposition for scalable NeRFs. Although achieving reasonable scalability, there are several critical problems remaining unexplored, i.e., learnable decomposition, modeling scene heterogeneity, and modeling efficiency. In this paper, we introduce Switch-NeRF++, a Heterogeneous Mixture of Hash Experts (HMoHE) network that addresses these challenges within a unified framework. It is a highly scalable NeRF that learns heterogeneous decomposition and heterogeneous NeRFs efficiently for large-scale scenes in an end-to-end manner. In our framework, a gating network learns to decomposes scenes and allocates 3D points to specialized NeRF experts. This gating network is co-optimized with the experts, by our proposed Sparsely Gated Mixture of Experts (MoE) NeRF framework. We incorporate a hash-based gating network and distinct heterogeneous hash experts. The hash-based gating efficiently learns the decomposition of the large-scale scene. The distinct heterogeneous hash experts consist of hash grids of different resolution ranges, enabling effective learning of the heterogeneous representation of different scene parts. These design choices make our framework an end-to-end and highly scalable NeRF solution for real-world large-scale scene modeling to achieve both quality and efficiency. We evaluate our accuracy and scalability on existing large-scale NeRF datasets and a new dataset with very large-scale scenes (>6.5km^2) from UrbanBIS. Extensive experiments demonstrate that our approach can be easily scaled to various large-scale scenes and achieve state-of-the-art scene rendering accuracy. Furthermore, our method exhibits significant efficiency, with an 8x acceleration in training and a 16x acceleration in rendering compared to Switch-NeRF. Codes will be released in https://github.com/MiZhenxing/Switch-NeRF.