三維場景生成：綜述

3D Scene Generation: A Survey

May 8, 2025

作者: Beichen Wen, Haozhe Xie, Zhaoxi Chen, Fangzhou Hong, Ziwei Liu

cs.AI

摘要

三維場景生成旨在為沉浸式媒體、機器人技術、自動駕駛以及具身智能等應用合成具有空間結構、語義意義且逼真的環境。早期基於程序規則的方法雖具備可擴展性，但多樣性有限。近年來，深度生成模型（如GANs、擴散模型）與三維表示（如NeRF、3D高斯）的進展，使得學習真實世界場景分佈成為可能，從而提升了保真度、多樣性及視角一致性。擴散模型等最新技術通過將生成問題重新定義為圖像或視頻合成，架起了三維場景合成與逼真度之間的橋樑。本綜述系統性地概述了當前最先進的方法，將其歸納為四大範式：程序生成、基於神經網絡的三維生成、基於圖像的生成以及基於視頻的生成。我們分析了它們的技術基礎、權衡取捨及代表性成果，並回顧了常用的數據集、評估協議及下游應用。最後，我們探討了生成能力、三維表示、數據與註釋以及評估等方面的關鍵挑戰，並展望了包括更高保真度、物理感知與交互生成以及統一的感知-生成模型在內的潛在方向。本綜述梳理了三維場景生成的最新進展，並強調了生成式人工智能、三維視覺與具身智能交叉領域的潛在方向。為追蹤持續發展，我們維護了一個實時更新的項目頁面：https://github.com/hzxie/Awesome-3D-Scene-Generation。

English

3D scene generation seeks to synthesize spatially structured, semantically meaningful, and photorealistic environments for applications such as immersive media, robotics, autonomous driving, and embodied AI. Early methods based on procedural rules offered scalability but limited diversity. Recent advances in deep generative models (e.g., GANs, diffusion models) and 3D representations (e.g., NeRF, 3D Gaussians) have enabled the learning of real-world scene distributions, improving fidelity, diversity, and view consistency. Recent advances like diffusion models bridge 3D scene synthesis and photorealism by reframing generation as image or video synthesis problems. This survey provides a systematic overview of state-of-the-art approaches, organizing them into four paradigms: procedural generation, neural 3D-based generation, image-based generation, and video-based generation. We analyze their technical foundations, trade-offs, and representative results, and review commonly used datasets, evaluation protocols, and downstream applications. We conclude by discussing key challenges in generation capacity, 3D representation, data and annotations, and evaluation, and outline promising directions including higher fidelity, physics-aware and interactive generation, and unified perception-generation models. This review organizes recent advances in 3D scene generation and highlights promising directions at the intersection of generative AI, 3D vision, and embodied intelligence. To track ongoing developments, we maintain an up-to-date project page: https://github.com/hzxie/Awesome-3D-Scene-Generation.