3D-RE-GEN：基于生成式框架的室内场景三维重建
3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework
December 19, 2025
作者: Tobias Sautter, Jan-Niklas Dihlmann, Hendrik P. A. Lensch
cs.AI
摘要
尽管三维场景生成技术近期取得了视觉上令人瞩目的成果，但现有表征方式难以满足视觉特效和游戏开发领域艺术家对可编辑纹理化三维网格场景的工作流需求。当前纹理网格场景重建方法虽进步显著，却因存在物体分解错误、空间关系失准及背景缺失等问题，尚无法达到艺术创作实用标准。我们提出3D-RE-GEN组合框架，能够从单张图像重建包含纹理化三维物体与背景的场景。研究表明，通过整合特定领域的前沿模型，可实现当前最优的场景重建效果，满足艺术创作需求。
我们的重建管线集成了资源检测、重建与布局模型，推动部分模型突破其原有应用领域。通过将遮挡物体获取视为图像编辑任务，利用生成模型在统一光照和几何条件下进行场景级推理重建。与现有方法不同，3D-RE-GEN能生成完整背景，在优化过程中对物体进行空间约束，并为视觉特效和游戏中的真实光照与模拟任务提供基础。为实现物理合理的布局，我们采用新颖的四自由度可微分优化方法，使重建物体与估计地平面精确对齐。
3D-RE-GEN在单图像三维场景重建中达到业界领先水平，通过精确相机标定与空间优化指导的组合生成，创造出具有一致性且可编辑的场景。
English
Recent advances in 3D scene generation produce visually appealing output, but current representations hinder artists' workflows that require modifiable 3D textured mesh scenes for visual effects and game development. Despite significant advances, current textured mesh scene reconstruction methods are far from artist ready, suffering from incorrect object decomposition, inaccurate spatial relationships, and missing backgrounds. We present 3D-RE-GEN, a compositional framework that reconstructs a single image into textured 3D objects and a background. We show that combining state of the art models from specific domains achieves state of the art scene reconstruction performance, addressing artists' requirements.
Our reconstruction pipeline integrates models for asset detection, reconstruction, and placement, pushing certain models beyond their originally intended domains. Obtaining occluded objects is treated as an image editing task with generative models to infer and reconstruct with scene level reasoning under consistent lighting and geometry. Unlike current methods, 3D-RE-GEN generates a comprehensive background that spatially constrains objects during optimization and provides a foundation for realistic lighting and simulation tasks in visual effects and games. To obtain physically realistic layouts, we employ a novel 4-DoF differentiable optimization that aligns reconstructed objects with the estimated ground plane. 3D-RE-GEN~achieves state of the art performance in single image 3D scene reconstruction, producing coherent, modifiable scenes through compositional generation guided by precise camera recovery and spatial optimization.