SpheRoPE: 基于球面RoPE的零样本、无需优化的360度全景生成

摘要

我们提出了一种零样本、无需训练且无需优化的框架，通过将球面先验直接注入预训练的扩散变换器，用于生成360度全景图像和视频。现有方法要么依赖对稀缺全景数据进行昂贵的微调，从而限制泛化能力，要么采用多步优化导致推理延迟过高。我们观察到，当代生成模型通过大规模训练已天然具备一定的全景先验能力。然而，这些涌现的能力仍不充分，因为模型从根本上无法满足等距柱状投影（ERP）所施加的严格拓扑约束。我们提出了一种零样本且无需优化的方法，在推理时解决这些约束。球面旋转位置编码（Spherical RoPE）替代了标准旋转位置嵌入：低频通道被重新参数化为3D笛卡尔坐标以天然编码球面流形，而高频通道则通过谐波量化强制实现精确周期性。结合显式引导几何的互补语义畸变无分类器引导（CFG），我们避免了重新训练，并继承了最先进模型的全部创作广度。该方法可泛化到多种骨干网络及360度生成模态。我们通过使用Flux.1、Flux.2和LTX-Video骨干网络在文本到全景图任务中进行了验证，在无需训练的前提下实现了与基线方法竞争的性能。项目页面：https://orhir.github.io/SpheRoPE

English

We present a zero-shot, training-free and optimization-free framework for generating 360 panoramic images and videos by directly injecting spherical priors into pre-trained diffusion transformers. Existing methods either rely on costly fine-tuning on scarce panoramic data that limits generalization, or leverage multi-step optimization that incurs prohibitive inference latency. We observe that contemporary generative models natively exhibit some panoramic priors from large-scale training. However, these emergent capabilities are insufficient, as the models fundamentally fail to satisfy the rigorous topological constraints imposed by equirectangular projection (ERP). We introduce a zero-shot and optimization-free approach that resolves these constraints at inference time. Spherical RoPE replaces standard rotary position embeddings: low-frequency channels are re-parameterized as 3D Cartesian coordinates to natively encode the spherical manifold, while high-frequency channels are harmonically quantized to enforce exact periodicity. Coupled with complementary Semantic Distortion classifier-free guidance (CFG) that explicitly steers geometry, we avoid retraining and inherit the full creative breadth of state-of-the-art models. Our approach generalizes across diverse backbones and 360 generation modalities. We demonstrate this across text-to-panorama using Flux.1, Flux.2, and LTX-Video backbones, achieving competitive performance against baselines, all while remaining training-free. Project page: https://orhir.github.io/SpheRoPE