MoVerse：基于全景高斯支架的实时视频世界建模

摘要

我们提出了MoVerse，一个实时视频世界模型，能够从单张窄视场图像创建可交互导航的场景。这一设定极具挑战性，因为输入仅观察到环境的极小部分，而交互式漫游需要完整的周围世界、持久几何、可控相机运动以及时间连贯的高保真观测。MoVerse将世界构建与观测渲染分离来解决此问题。它首先通过拓扑感知扩散将输入扩展为重力对齐的360°全景图，在三维推理前补全缺失的视场。接着，利用全景几何感知残差预测将全景图提升为持久的三维高斯支架，生成密集且可直接渲染的空间记忆。最后，一个高斯条件视频渲染器将沿用户指定相机轨迹的支架渲染结果转换为逼真的视频。为了使该渲染器适用于交互场景，我们训练了一个双向扩散教师模型用于高质量条件渲染，并将其蒸馏为因果自回归学生模型以实现有界延迟流式输出。这种设计结合了显式三维表示的可控性与长程一致性，以及生成式视频模型的感知质量。MoVerse在单个NVIDIA RTX 4090 GPU上支持8 FPS的实时场景漫游，展示了从单张图像创建交互式视频输出的实用路径。

English

We present MoVerse, a real-time video world model that creates an interactively navigable scene from a single narrow-field-of-view image. This setting is challenging because the input observes only a small fraction of the environment, while interactive roaming requires a complete surrounding world, persistent geometry, controllable camera motion, and temporally coherent high-fidelity observations. MoVerse addresses this problem by separating world construction from observation rendering. It first expands the input into a gravity-aligned 360^circ panorama with topology-aware diffusion, closing the missing field of view before 3D reasoning. It then lifts the panorama into a persistent 3D Gaussian scaffold using panoramic geometry-aware residual prediction, yielding a dense and directly renderable spatial memory. Finally, a Gaussian-conditioned video renderer translates scaffold renderings along user-specified camera trajectories into photorealistic video. To make this renderer practical for interaction, we train a bidirectional diffusion teacher for high-quality conditional rendering and distill it into a causal autoregressive student for bounded-latency streaming. This design combines the controllability and long-range consistency of explicit 3D representations with the perceptual quality of generative video models. MoVerse supports real-time scene roaming at 8~FPS on a single NVIDIA RTX~4090 GPU, demonstrating a practical path toward single-image world creation with interactive video output.