重建四維空間智能：一項綜述

摘要

从视觉观察中重建四维空间智能，长期以来一直是计算机视觉领域中的核心且具挑战性的任务，其应用范围广泛，涵盖了从娱乐产业如电影制作——其中重点常在于重建基础视觉元素，到具身人工智能——强调交互建模与物理真实感。得益于三维表示与深度学习架构的飞速发展，该领域进展迅速，超越了以往综述的涵盖范围。此外，现有综述鲜少对四维场景重建的层次结构进行全面分析。为填补这一空白，我们提出了一种新视角，将现有方法组织为五个递进层次的四维空间智能：(1) 第一层次——低层次三维属性重建（如深度、姿态及点云图）；(2) 第二层次——三维场景组件重建（如物体、人物、结构）；(3) 第三层次——四维动态场景重建；(4) 第四层次——场景组件间交互建模；(5) 第五层次——物理定律与约束的融入。在综述的结尾，我们讨论了每一层次的关键挑战，并指出了向更丰富四维空间智能层次推进的潜在方向。为追踪最新进展，我们维护了一个实时更新的项目页面：https://github.com/yukangcao/Awesome-4D-Spatial-Intelligence。

English

Reconstructing 4D spatial intelligence from visual observations has long been a central yet challenging task in computer vision, with broad real-world applications. These range from entertainment domains like movies, where the focus is often on reconstructing fundamental visual elements, to embodied AI, which emphasizes interaction modeling and physical realism. Fueled by rapid advances in 3D representations and deep learning architectures, the field has evolved quickly, outpacing the scope of previous surveys. Additionally, existing surveys rarely offer a comprehensive analysis of the hierarchical structure of 4D scene reconstruction. To address this gap, we present a new perspective that organizes existing methods into five progressive levels of 4D spatial intelligence: (1) Level 1 -- reconstruction of low-level 3D attributes (e.g., depth, pose, and point maps); (2) Level 2 -- reconstruction of 3D scene components (e.g., objects, humans, structures); (3) Level 3 -- reconstruction of 4D dynamic scenes; (4) Level 4 -- modeling of interactions among scene components; and (5) Level 5 -- incorporation of physical laws and constraints. We conclude the survey by discussing the key challenges at each level and highlighting promising directions for advancing toward even richer levels of 4D spatial intelligence. To track ongoing developments, we maintain an up-to-date project page: https://github.com/yukangcao/Awesome-4D-Spatial-Intelligence.

重建四維空間智能：一項綜述

Reconstructing 4D Spatial Intelligence: A Survey

摘要

Support