天際隕落-GS：基於衛星影像的沉浸式三維城市場景合成

摘要

合成大規模、可探索且幾何精確的三維城市場景，對於提供沉浸式與具身應用而言，是一項既具挑戰性又極具價值的任務。此挑戰主要源於缺乏大規模且高質量的真實世界三維掃描數據，以訓練具有泛化能力的生成模型。本文中，我們採取了一條替代路徑來創建大規模三維場景，即融合易於獲取的衛星影像——其提供了真實的粗略幾何信息——與開放域擴散模型——用於生成高質量的近景外觀。我們提出了Skyfall-GS，這是首個無需昂貴三維註釋即可實現城市街區尺度三維場景創建的框架，同時具備實時、沉浸式的三維探索功能。我們量身定制了一種課程驅動的迭代優化策略，逐步提升幾何完整度與照片級真實感的紋理。大量實驗表明，與現有最先進的方法相比，Skyfall-GS在跨視角一致的幾何結構與更為逼真的紋理表現上均有顯著提升。項目頁面：https://skyfall-gs.jayinnn.dev/

English

Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task in providing immersive and embodied applications. The challenges lie in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by synergizing the readily available satellite imagery that supplies realistic coarse geometry and the open-domain diffusion model for creating high-quality close-up appearances. We propose Skyfall-GS, the first city-block scale 3D scene creation framework without costly 3D annotations, also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic textures. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/

天際隕落-GS：基於衛星影像的沉浸式三維城市場景合成

Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery

摘要

Support