NoPA: 非参数在线三维场景图生成

摘要

经典的3D场景图生成方法由于环境映射的高计算成本以及需要生成中间点云表示，难以实现实时运行。为解决这一问题，近期研究摒弃了点云表示，转而采用轻量化的高斯分布来表征每个物体。这种近似方法大幅提升了推理速度，使实时3D场景图生成成为可能。然而，该表示存在两个关键缺陷：1）每个物体仅由单个3D高斯分布近似，导致三维几何细节严重丢失；2）这种近似与真实物体几何之间的差异，加剧了在线推理过程中物体候选的错误合并。针对上述问题，我们提出NoPA方法，将每个物体表示为独立的非参数分布。这种表述方式既能保留三维几何信息，又能维持参数化高斯公式的实时推理特性。基于这一新型物体表征，我们进一步设计了定制化合并策略以重建连续物体实例。具体而言，我们利用核密度估计的最大均值差异，在最小化计算复杂度的前提下，实现在线探索过程中物体候选的鲁棒合并。核心在于为每个物体维护固定的粒子集。此外，为修正由错误分类引起的关联损失，NoPA会传播高亲和性物体之间的关系。实验表明，NoPA在不牺牲实时推理速度的前提下，性能显著优于现有方法。

English

Classic 3D scene graph generation approaches fail to work in real-time due to the heavy computational cost of environment mapping and the need to generate intermediate point-cloud representations. To alleviate this issue, a recent work eschews point clouds in favor of a lightweight Gaussian distribution for each object. This approximation drastically speeds up inference and enables real-time 3D scene graph generation. However, the representation has two key weaknesses. 1) Each object is approximated by a single 3D Gaussian, which causes a severe loss of 3D geometric detail. 2) The discrepancy between this approximation and the true object geometry exacerbates the inaccurate merging of object candidates during online inference. To address these issues, we propose NoPA, which represents each object as a separate non-parametric distribution. This formulation retains 3D geometric information while preserving real-time inference of the parametric Gaussian formulation. To build upon our novel object representation, we propose a tailored merging strategy to recover coherent object instances. Specifically, we leverage maximum mean discrepancy on kernel density estimates to enable robust merging of object candidates during online exploration while minimizing added computational complexity. The key is to maintain a fixed particle set per object. Furthermore, to rectify the relation loss caused by misclassified objects, NoPA propagates relationships between objects with high affinity. Experiments show that NoPA substantially outperforms current methods without sacrificing real-time inference speed.