重构四维空间智能：综述研究

摘要

从视觉观察中重建四维空间智能一直是计算机视觉领域核心且富有挑战性的任务，其应用范围广泛。这些应用从娱乐领域如电影制作——其中重点常在于重建基础视觉元素，到具身人工智能——强调交互建模与物理真实感。得益于三维表示与深度学习架构的快速发展，该领域迅速演进，已超越以往综述的涵盖范围。此外，现有综述鲜少对四维场景重建的层次结构进行全面分析。为填补这一空白，我们提出了一种新视角，将现有方法组织为五个渐进层次的四维空间智能：(1) 第一层次——重建低层次三维属性（如深度、姿态及点云图）；(2) 第二层次——重建三维场景组件（如物体、人物、结构）；(3) 第三层次——重建四维动态场景；(4) 第四层次——建模场景组件间的交互；(5) 第五层次——融入物理定律与约束。综述最后，我们讨论了每一层次的关键挑战，并指出了向更丰富四维空间智能迈进的有前景方向。为追踪最新进展，我们维护了一个实时更新的项目页面：https://github.com/yukangcao/Awesome-4D-Spatial-Intelligence。

English

Reconstructing 4D spatial intelligence from visual observations has long been a central yet challenging task in computer vision, with broad real-world applications. These range from entertainment domains like movies, where the focus is often on reconstructing fundamental visual elements, to embodied AI, which emphasizes interaction modeling and physical realism. Fueled by rapid advances in 3D representations and deep learning architectures, the field has evolved quickly, outpacing the scope of previous surveys. Additionally, existing surveys rarely offer a comprehensive analysis of the hierarchical structure of 4D scene reconstruction. To address this gap, we present a new perspective that organizes existing methods into five progressive levels of 4D spatial intelligence: (1) Level 1 -- reconstruction of low-level 3D attributes (e.g., depth, pose, and point maps); (2) Level 2 -- reconstruction of 3D scene components (e.g., objects, humans, structures); (3) Level 3 -- reconstruction of 4D dynamic scenes; (4) Level 4 -- modeling of interactions among scene components; and (5) Level 5 -- incorporation of physical laws and constraints. We conclude the survey by discussing the key challenges at each level and highlighting promising directions for advancing toward even richer levels of 4D spatial intelligence. To track ongoing developments, we maintain an up-to-date project page: https://github.com/yukangcao/Awesome-4D-Spatial-Intelligence.

重构四维空间智能：综述研究

Reconstructing 4D Spatial Intelligence: A Survey

摘要

Support