漫步隱式：基於神經場景表示的互動式世界探索

摘要

針對攝影機控制的世界探索，互動式影片生成系統會滾動生成越來越長的潛在影片幀序列，從而將狀態轉換與高頻觀測合成糾纏在一起。我們提出「漫步於隱含世界」（Walking in the Implicit），這是一種以場景為核心的範式，將生成變量從幀潛在變量改為固定長度、可渲染的隱含狀態，稱為神經隱含場景（Neural Implicit Scene, NIS）。此範式將互動式生成分解為：一個緊湊場景狀態的隨機轉換，以及針對取樣狀態進行的確定性姿態條件渲染。我們將此範式實例化為 NeuWorld：一個變換器變分自編碼器（transformer VAE）從稀疏的有位姿幀中學習局部錨定的 NIS，而一個擴散變換器（diffusion transformer）則根據未來攝影機軌跡與具幾何感知的歷史檢索來演化 NIS。通過重用 VAE 編碼器作為統一條件器，NeuWorld 將攝影機、參考圖像和歷史線索映射到相同的 NIS 模態中，從而避免使用外部異質編碼器。NeuWorld 從頭開始在公開的位姿視圖數據上訓練，無需預訓練的影片主幹網路或輔助的三維重建器，即可實現強大的長期一致性與良好的推理效率。

English

Interactive video generation systems for camera-controlled world exploration roll out growing sequences of latent video frames, entangling state transition with high-frequency observation synthesis. We propose Walking in the Implicit, a scene-centric paradigm that changes the rollout variable from frame latents to a fixed-length, renderable implicit state, termed Neural Implicit Scene (NIS). This factorizes interactive generation into stochastic transition of a compact scene state and deterministic pose-conditioned rendering given the sampled state. We instantiate this paradigm as NeuWorld: a transformer VAE learns locally anchored NIS from sparse posed frames, and a diffusion transformer evolves NIS conditioned on future camera trajectories and geometry-aware retrieved history. By reusing the VAE encoder as a unified conditioner, NeuWorld maps camera, reference-image, and history cues into the same NIS modality, avoiding external heterogeneous encoders. Trained from scratch on public posed-view data without pretrained video backbones or auxiliary 3D reconstructors, NeuWorld achieves strong long-horizon consistency with favorable inference efficiency.