透過幾何感知預訓練增強上下文全景生成

摘要

在本研究中，我們提出 Canvas360——一個兩階段的情境全景生成框架，結合幾何感知預訓練與下游任務特化微調。為解決缺乏針對情境全景任務的大規模高品質訓練資料之難題，我們建構了 Canvas360Dataset，包含100萬組高品質配對全景樣本，涵蓋風格轉換、修補、擴展與編輯等任務，能在多樣的情境生成場景中提供有效監督。在建模方面，Canvas360 透過並行深度生成、速度循環填充與相似性損失正則化強化文字至全景生成能力，使模型學習幾何感知表徵、捕捉物體變形細節，並增進幾何一致性與全局連貫性。此外，憑藉強大的全景先驗知識，Canvas360 實現統一的情境全景生成框架，透過詞元層級串接支援多樣下游任務，在任務覆蓋率與建模靈活性上均超越先前方法。大量實驗顯示，Canvas360 能提升全景影像真實度，特別在全景專屬 FAED 指標上表現優異，並在報告的各項量化評估中取得具有競爭力或領先的成果。更多資訊請參閱我們的專案頁面：https://zry000.github.io/Canvas360/

English

In this work, we present Canvas360, a two-stage framework for in-context panoramic generation that combines geometry-aware pretraining with downstream task-specific fine-tuning. To address the lack of large-scale, high-quality training data tailored to in-context panoramic tasks, we propose Canvas360Dataset, a collection of 1M high-quality paired panoramic samples for style transfer, inpainting, outpainting, and editing, enabling effective supervision across diverse in-context generation scenarios. On the modeling side, Canvas360 enhances text-to-panorama generation through parallel depth generation, velocity circular padding, and similarity loss regularization, enabling the model to learn geometry-aware representations, capture object distortion details, and improve geometric consistency and global coherence. Furthermore, empowered by strong panoramic priors, Canvas360 enables a unified in-context panoramic generation framework that supports diverse downstream tasks via token-level concatenation, surpassing prior methods in both task coverage and modeling flexibility. Extensive experiments show that Canvas360 improves panoramic image fidelity, achieving particularly strong performance on the panorama-specific FAED metric and competitive or leading results across the reported quantitative evaluations. More information can be found on our project page: https://zry000.github.io/Canvas360/