Skyfall-GS：从卫星图像合成沉浸式3D城市场景

摘要

合成大规模、可探索且几何精确的3D城市场景，对于提供沉浸式与具身化应用而言，是一项既具挑战性又极具价值的任务。其难点在于缺乏用于训练通用生成模型的大规模高质量真实世界3D扫描数据。本文中，我们另辟蹊径，通过整合易于获取的卫星影像（提供真实的粗略几何信息）与开放域扩散模型（用于生成高质量近景外观），来创建大规模3D场景。我们提出了Skyfall-GS，这是首个无需昂贵3D标注即可创建城市街区尺度3D场景的框架，同时支持实时沉浸式3D探索。我们定制了一套课程驱动的迭代优化策略，逐步提升几何完整性与照片级真实感纹理。大量实验表明，相较于现有最先进方法，Skyfall-GS在跨视角一致的几何结构与更逼真的纹理表现上均有显著提升。项目页面：https://skyfall-gs.jayinnn.dev/

English

Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task in providing immersive and embodied applications. The challenges lie in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by synergizing the readily available satellite imagery that supplies realistic coarse geometry and the open-domain diffusion model for creating high-quality close-up appearances. We propose Skyfall-GS, the first city-block scale 3D scene creation framework without costly 3D annotations, also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic textures. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/

Skyfall-GS：从卫星图像合成沉浸式3D城市场景

Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery

摘要

Support