CityDreamer：無限3D城市的組合生成模型

CityDreamer: Compositional Generative Model of Unbounded 3D Cities

September 1, 2023

作者: Haozhe Xie, Zhaoxi Chen, Fangzhou Hong, Ziwei Liu

cs.AI

摘要

近年來，廣泛的研究集中在3D自然場景生成上，但3D城市生成領域卻沒有受到同樣多的探索。這是因為3D城市生成帶來更大的挑戰，主要是因為人們對城市環境中結構變形更為敏感。此外，生成3D城市比生成3D自然場景更為複雜，因為作為同一類別的物體，建築物展現出比自然場景中樹木等相對一致外觀更廣泛的外觀範圍。為了應對這些挑戰，我們提出了CityDreamer，一種專門設計用於無邊界3D城市的組合生成模型，將建築實例的生成與其他背景物體（如道路、綠地和水域）的生成區分為不同模塊。此外，我們構建了兩個數據集，OSM和GoogleEarth，其中包含大量真實世界城市圖像，以增強生成的3D城市在佈局和外觀上的逼真度。通過廣泛的實驗，CityDreamer已經證明其在生成各種逼真3D城市方面優於最先進的方法。

English

In recent years, extensive research has focused on 3D natural scene generation, but the domain of 3D city generation has not received as much exploration. This is due to the greater challenges posed by 3D city generation, mainly because humans are more sensitive to structural distortions in urban environments. Additionally, generating 3D cities is more complex than 3D natural scenes since buildings, as objects of the same class, exhibit a wider range of appearances compared to the relatively consistent appearance of objects like trees in natural scenes. To address these challenges, we propose CityDreamer, a compositional generative model designed specifically for unbounded 3D cities, which separates the generation of building instances from other background objects, such as roads, green lands, and water areas, into distinct modules. Furthermore, we construct two datasets, OSM and GoogleEarth, containing a vast amount of real-world city imagery to enhance the realism of the generated 3D cities both in their layouts and appearances. Through extensive experiments, CityDreamer has proven its superiority over state-of-the-art methods in generating a wide range of lifelike 3D cities.