PanoWorld：真实世界全景生成

摘要

在本工作中，我们旨在通过利用全景表示的旋转等变性（旋转可视为一种隐式几何变换）来解决全景世界模型中的长程记忆挑战。基于这一洞见，我们提出了PanoWorld，它通过固定航向将相机轨迹简化为平移，并利用密集全景射线条件（DPRC）和几何感知记忆增强（GMA）分别实现当前动作建模与长程记忆。随后，我们引入了一个三阶段训练流程，逐步优化每个组件。为了更好地评估在现有数据集相对稳定的大尺度空间变化和多样化光照条件下的物理一致性，我们构建了World360，这是一个包含通过全景无人机采集的真实世界视频片段以及通过AirSim360生成的高质量模拟片段的大规模数据集。在World360上的大量实验证明了PanoWorld的有效性，其性能远超其他方法。我们的模型、训练代码和数据集将公开发布。更多信息可访问我们的项目页面：https://lihaoy-ux.github.io/panoworld-page/。

English

In this work, we aim to address the challenge of long-range memory in panoramic world models by exploiting the rotation-equivariant property of omnidirectional representations, where rotation can be treated as an implicit geometric transformation.Building on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current-action modeling and long-range memory through Dense Panoramic Ray-Conditioning (DPRC) and Geometry-aware Memory Augmentation (GMA).Then, a three-stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large-scale spatial variations and diverse illumination conditions, where existing datasets are relatively stable, we construct World360, a large-scale dataset consisting of both real-world video clips collected via panoramic unmanned aerial vehicles and high-quality simulated clips generated by AirSim360.Extensive experiments on World360 demonstrate the effectiveness of PanoWorld, outperforming alternative methods by a large margin.Our models, training code, and dataset will be publicly available. More information can be found on our project page: https://lihaoy-ux.github.io/panoworld-page/.