行走于隐式表示：基于神经场景表示的交互式世界探索

摘要

面向相机控制的世界探索的交互式视频生成系统，会逐步生成长序列的潜在视频帧，将状态转换与高频观测合成纠缠在一起。我们提出"隐式行走"（Walking in the Implicit），这是一种以场景为中心的范式，将滚动变量从帧潜在表示更改为固定长度、可渲染的隐式状态，称为神经隐式场景（NIS）。该范式将交互式生成分解为紧凑场景状态的随机转换，以及给定采样状态下基于姿态的条件确定性渲染。我们将这一范式实例化为NeuWorld：一个变换器VAE从稀疏的带姿态帧中学习局部锚定的NIS，而一个扩散变换器则根据未来相机轨迹和基于几何感知检索的历史信息来演化NIS。通过复用VAE编码器作为统一条件器，NeuWorld将相机、参考图像和历史线索映射到同一NIS模态中，避免了外部异构编码器。NeuWorld在公开的带姿态视角数据上从头训练，无需预训练的视频骨干网络或辅助的3D重建器，在实现强长期一致性的同时，保持了良好的推理效率。

English

Interactive video generation systems for camera-controlled world exploration roll out growing sequences of latent video frames, entangling state transition with high-frequency observation synthesis. We propose Walking in the Implicit, a scene-centric paradigm that changes the rollout variable from frame latents to a fixed-length, renderable implicit state, termed Neural Implicit Scene (NIS). This factorizes interactive generation into stochastic transition of a compact scene state and deterministic pose-conditioned rendering given the sampled state. We instantiate this paradigm as NeuWorld: a transformer VAE learns locally anchored NIS from sparse posed frames, and a diffusion transformer evolves NIS conditioned on future camera trajectories and geometry-aware retrieved history. By reusing the VAE encoder as a unified conditioner, NeuWorld maps camera, reference-image, and history cues into the same NIS modality, avoiding external heterogeneous encoders. Trained from scratch on public posed-view data without pretrained video backbones or auxiliary 3D reconstructors, NeuWorld achieves strong long-horizon consistency with favorable inference efficiency.