CityDreamer：无限3D城市的组合生成模型

CityDreamer: Compositional Generative Model of Unbounded 3D Cities

September 1, 2023

作者: Haozhe Xie, Zhaoxi Chen, Fangzhou Hong, Ziwei Liu

cs.AI

摘要

近年来，广泛的研究集中在3D自然场景生成上，但3D城市生成领域却没有得到同样多的探索。这是因为3D城市生成面临更大的挑战，主要是因为人类对城市环境中的结构失真更为敏感。此外，生成3D城市比生成3D自然场景更复杂，因为作为同一类别的对象，建筑物的外观范围比自然场景中树木等相对一致的外观要广泛。为了解决这些挑战，我们提出了CityDreamer，这是一种专为无边界3D城市设计的组合生成模型，将建筑物实例的生成与道路、绿地和水域等其他背景对象的生成分开为不同的模块。此外，我们构建了两个数据集，OSM和GoogleEarth，包含大量真实世界城市图像，以增强生成的3D城市在布局和外观上的逼真度。通过大量实验，CityDreamer已经证明在生成各种逼真的3D城市方面优于最先进的方法。

English

In recent years, extensive research has focused on 3D natural scene generation, but the domain of 3D city generation has not received as much exploration. This is due to the greater challenges posed by 3D city generation, mainly because humans are more sensitive to structural distortions in urban environments. Additionally, generating 3D cities is more complex than 3D natural scenes since buildings, as objects of the same class, exhibit a wider range of appearances compared to the relatively consistent appearance of objects like trees in natural scenes. To address these challenges, we propose CityDreamer, a compositional generative model designed specifically for unbounded 3D cities, which separates the generation of building instances from other background objects, such as roads, green lands, and water areas, into distinct modules. Furthermore, we construct two datasets, OSM and GoogleEarth, containing a vast amount of real-world city imagery to enhance the realism of the generated 3D cities both in their layouts and appearances. Through extensive experiments, CityDreamer has proven its superiority over state-of-the-art methods in generating a wide range of lifelike 3D cities.