NuiScene：探索無界戶外場景的高效生成

NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenes

March 20, 2025

作者: Han-Hung Lee, Qinghong Han, Angel X. Chang

cs.AI

摘要

本文探討了生成廣闊戶外場景的任務，範圍從城堡到高樓大廈。與以往主要關注的室內場景生成不同，戶外場景生成面臨獨特的挑戰，包括場景高度的巨大變化以及需要一種能夠快速生成大型景觀的方法。為此，我們提出了一種高效的方法，將場景塊編碼為均勻的向量集，相比於先前方法中使用的空間結構潛在變量，提供了更好的壓縮和性能。此外，我們訓練了一個顯式的外繪模型，用於無限制生成，這相比於基於重採樣的修補方案提高了連貫性，同時通過消除額外的擴散步驟加快了生成速度。為了促進這項任務，我們策劃了NuiScene43，這是一個小而高質量的場景集，經過預處理以進行聯合訓練。值得注意的是，當在不同風格的場景上進行訓練時，我們的模型能夠在同一場景中融合不同的環境，例如鄉村房屋和城市摩天大樓，這凸顯了我們策劃過程在利用異質場景進行聯合訓練方面的潛力。

English

In this paper, we explore the task of generating expansive outdoor scenes, ranging from castles to high-rises. Unlike indoor scene generation, which has been a primary focus of prior work, outdoor scene generation presents unique challenges, including wide variations in scene heights and the need for a method capable of rapidly producing large landscapes. To address this, we propose an efficient approach that encodes scene chunks as uniform vector sets, offering better compression and performance than the spatially structured latents used in prior methods. Furthermore, we train an explicit outpainting model for unbounded generation, which improves coherence compared to prior resampling-based inpainting schemes while also speeding up generation by eliminating extra diffusion steps. To facilitate this task, we curate NuiScene43, a small but high-quality set of scenes, preprocessed for joint training. Notably, when trained on scenes of varying styles, our model can blend different environments, such as rural houses and city skyscrapers, within the same scene, highlighting the potential of our curation process to leverage heterogeneous scenes for joint training.