从单张图像构建三维城镇

Constructing a 3D Town from a Single Image

May 21, 2025

作者: Kaizhi Zheng, Ruijian Zhang, Jing Gu, Jie Yang, Xin Eric Wang

cs.AI

摘要

获取精细的三维场景通常需要昂贵的设备、多视角数据或费时费力的建模过程。因此，一种轻量级的替代方案——从单一俯视图像生成复杂的三维场景，在实际应用中扮演着至关重要的角色。尽管近期的三维生成模型在物体级别上取得了显著成果，但将其扩展至全场景生成时，常出现几何不一致、布局幻觉及低质量网格等问题。本研究中，我们提出了3DTown，一个无需训练即可从单一俯视图像合成逼真且连贯三维场景的框架。我们的方法基于两大原则：基于区域的生成以提升图像到三维的对齐与分辨率，以及空间感知的三维修复以确保全局场景的一致性和高质量几何生成。具体而言，我们将输入图像分解为重叠区域，利用预训练的三维物体生成器分别生成各区域，随后通过掩码修正流修复过程填补缺失几何，同时保持结构连续性。这种模块化设计使我们能够克服分辨率瓶颈，保留空间结构，而无需三维监督或微调。跨多种场景的广泛实验表明，3DTown在几何质量、空间连贯性和纹理保真度方面均优于包括Trellis、Hunyuan3D-2和TripoSG在内的最先进基线方法。我们的成果证明，采用一种有原则、无需训练的方法，从单一图像实现高质量三维城镇生成是可行的。

English

Acquiring detailed 3D scenes typically demands costly equipment, multi-view data, or labor-intensive modeling. Therefore, a lightweight alternative, generating complex 3D scenes from a single top-down image, plays an essential role in real-world applications. While recent 3D generative models have achieved remarkable results at the object level, their extension to full-scene generation often leads to inconsistent geometry, layout hallucinations, and low-quality meshes. In this work, we introduce 3DTown, a training-free framework designed to synthesize realistic and coherent 3D scenes from a single top-down view. Our method is grounded in two principles: region-based generation to improve image-to-3D alignment and resolution, and spatial-aware 3D inpainting to ensure global scene coherence and high-quality geometry generation. Specifically, we decompose the input image into overlapping regions and generate each using a pretrained 3D object generator, followed by a masked rectified flow inpainting process that fills in missing geometry while maintaining structural continuity. This modular design allows us to overcome resolution bottlenecks and preserve spatial structure without requiring 3D supervision or fine-tuning. Extensive experiments across diverse scenes show that 3DTown outperforms state-of-the-art baselines, including Trellis, Hunyuan3D-2, and TripoSG, in terms of geometry quality, spatial coherence, and texture fidelity. Our results demonstrate that high-quality 3D town generation is achievable from a single image using a principled, training-free approach.