SynCity 3000: 自举式场景尺度3D扩散

摘要

我们提出SynCity 3000框架，用于生成全局连贯且支持细粒度布局控制的3D场景。基于当前图像到3D生成器从单张图像生成复杂3D资产的能力，我们通过将生成器改造为可卷积算子，将此能力扩展至整个场景的规模。为此，我们在场景类数据上对模型进行微调——这类数据由我们提出的新型合成数据引擎生成，以解决训练中3D场景数据稀缺的问题。随后，将卷积生成器应用于用户提示生成的全局等轴测图像，即可生成任意尺寸与复杂度的3D场景。在多样化的提示与布局下，SynCity 3000能够生成大型、连贯且细节丰富的场景，弥补了先前3D场景生成方法的不足。

English

We present SynCity 3000, a framework for generating 3D scenes that are globally coherent while enabling fine-grained layout control. Building on the ability of current image-to-3D generators to produce complex 3D assets from a single image, we extend this capability to the scale of entire scenes by adapting the generator to be applicable as a convolutional operator. We achieve this by fine-tuning the model on scene-like data generated by a new synthetic data engine, which we propose to address the scarcity of 3D scene data for training. The convolutional generator is then applied to a dimetric image of the entire scene, generated from the user prompt, resulting in 3D scenes of arbitrary size and complexity. Across diverse prompts and layouts, SynCity 3000 produces large, coherent, and detailed scenes, addressing the shortcomings of prior approaches to 3D scene generation.