SynCity：无需训练的3D世界生成

SynCity: Training-Free Generation of 3D Worlds

March 20, 2025

作者: Paul Engstler, Aleksandar Shtedritski, Iro Laina, Christian Rupprecht, Andrea Vedaldi

cs.AI

摘要

我们致力于解决从文本描述生成三维世界的挑战。我们提出了SynCity，一种无需训练和优化的方法，它结合了预训练三维生成模型的几何精度与二维图像生成器的艺术多样性，以创建大规模、高质量的三维空间。尽管大多数三维生成模型以物体为中心，无法生成大规模世界，但我们展示了如何将三维与二维生成器结合，以生成不断扩展的场景。通过基于区块的方法，我们实现了对场景布局和外观的精细控制。世界按区块逐一生成，每个新区块在其世界上下文中生成后与场景融合。SynCity生成引人入胜且细节丰富、多样化的沉浸式场景。

English

We address the challenge of generating 3D worlds from textual descriptions. We propose SynCity, a training- and optimization-free approach, which leverages the geometric precision of pre-trained 3D generative models and the artistic versatility of 2D image generators to create large, high-quality 3D spaces. While most 3D generative models are object-centric and cannot generate large-scale worlds, we show how 3D and 2D generators can be combined to generate ever-expanding scenes. Through a tile-based approach, we allow fine-grained control over the layout and the appearance of scenes. The world is generated tile-by-tile, and each new tile is generated within its world-context and then fused with the scene. SynCity generates compelling and immersive scenes that are rich in detail and diversity.