NoPA：非參數在線3D場景圖生成

摘要

經典的3D場景圖生成方法因環境映射計算成本高昂，且需生成中介點雲表示，而無法實現即時運作。為緩解此問題，近期研究捨棄點雲，改用輕量級高斯分布來表示每個物體。此近似方法大幅加速推論，使即時3D場景圖生成成為可能。然而，該表示存在兩項關鍵缺陷：1) 每個物體僅以單一3D高斯近似，導致3D幾何細節嚴重損失；2) 此近似與真實物體幾何間的差異，加劇了在線推論過程中物件候選項合併不準確的問題。為解決上述問題，我們提出NoPA，將每個物體表示為獨立的非參數分布。此表述保留3D幾何資訊，同時維持參數化高斯表述的即時推論能力。為奠基於新穎的物件表示，我們提出客製化合併策略，以恢復連貫的物件實例。具體而言，我們利用核密度估計的最大均值差異，在最小化額外計算複雜度的前提下，實現線上探索中物件候選項的穩健合併。關鍵在於為每個物件維持固定粒子集合。此外，為修正因錯誤分類導致的關係缺失，NoPA透過高親和性在物件間傳播關係。實驗顯示，NoPA在不犧牲即時推論速度的情況下，顯著優於現有方法。

English

Classic 3D scene graph generation approaches fail to work in real-time due to the heavy computational cost of environment mapping and the need to generate intermediate point-cloud representations. To alleviate this issue, a recent work eschews point clouds in favor of a lightweight Gaussian distribution for each object. This approximation drastically speeds up inference and enables real-time 3D scene graph generation. However, the representation has two key weaknesses. 1) Each object is approximated by a single 3D Gaussian, which causes a severe loss of 3D geometric detail. 2) The discrepancy between this approximation and the true object geometry exacerbates the inaccurate merging of object candidates during online inference. To address these issues, we propose NoPA, which represents each object as a separate non-parametric distribution. This formulation retains 3D geometric information while preserving real-time inference of the parametric Gaussian formulation. To build upon our novel object representation, we propose a tailored merging strategy to recover coherent object instances. Specifically, we leverage maximum mean discrepancy on kernel density estimates to enable robust merging of object candidates during online exploration while minimizing added computational complexity. The key is to maintain a fixed particle set per object. Furthermore, to rectify the relation loss caused by misclassified objects, NoPA propagates relationships between objects with high affinity. Experiments show that NoPA substantially outperforms current methods without sacrificing real-time inference speed.