超场：从文本实现零-shot 生成 NeRFs

摘要

我们介绍了HyperFields，这是一种通过单次前向传递（可选进行一些微调）生成文本条件的神经辐射场（NeRFs）的方法。我们方法的关键在于：（i）动态超网络，学习从文本标记嵌入到NeRFs空间的平滑映射；（ii）NeRF蒸馏训练，将编码在各个NeRFs中的场景蒸馏成一个动态超网络。这些技术使得单个网络能够适应超过一百个独特场景。我们进一步证明了HyperFields学习了更通用的文本与NeRFs之间的映射，因此能够预测新颖的分布内和分布外场景，包括零样本或经过少量微调步骤。通过学习的通用映射，对HyperFields进行微调可以加速收敛，并且能够比现有的基于神经优化的方法更快地合成新颖场景，速度提高了5到10倍。我们的消融实验表明，动态架构和NeRF蒸馏对于HyperFields的表达能力至关重要。

English

We introduce HyperFields, a method for generating text-conditioned Neural Radiance Fields (NeRFs) with a single forward pass and (optionally) some fine-tuning. Key to our approach are: (i) a dynamic hypernetwork, which learns a smooth mapping from text token embeddings to the space of NeRFs; (ii) NeRF distillation training, which distills scenes encoded in individual NeRFs into one dynamic hypernetwork. These techniques enable a single network to fit over a hundred unique scenes. We further demonstrate that HyperFields learns a more general map between text and NeRFs, and consequently is capable of predicting novel in-distribution and out-of-distribution scenes -- either zero-shot or with a few finetuning steps. Finetuning HyperFields benefits from accelerated convergence thanks to the learned general map, and is capable of synthesizing novel scenes 5 to 10 times faster than existing neural optimization-based methods. Our ablation experiments show that both the dynamic architecture and NeRF distillation are critical to the expressivity of HyperFields.

超场：从文本实现零-shot 生成 NeRFs

HyperFields: Towards Zero-Shot Generation of NeRFs from Text

摘要

Support