生成式語意場景補全

摘要

戶外光達語意場景完成（SSC）從僅觀測目標體積 1% 的掃描中還原稠密語意體素網格，且類別不平衡超過 7,000 倍。我們將 SSC 重新定義為生成式語意場景完成（GSSC）：一個以三種角色運作的單一離散擴散架構。第一，成對稀疏-稠密場景合成（PS^3）生成匹配的稀疏光達觀測及其稠密語意完成，從源頭處理長尾問題，並產出我們與 SemanticKITTI 一同訓練所用的 PS^3-SemanticKITTI 資料集。第二，語意引導生成式場景完成（SGSC）使用多項式離散擴散從雜訊生成場景，並以稀疏掃描為條件，透過鳥瞰圖語意地圖與稀疏 3D 特徵流傳遞條件。第三，同一架構改以單一流匹配步驟精化現有的完成結果：結構化來源離散擴散（S^2D^2）。S^2D^2 提升了 SGSC 自身輸出以及所有受測外部 SSC 基礎模型的 mIoU，且無需基礎模型重新訓練或測試時適應。在最強的基礎模型上，單一步驟且無測試時資料增強在 SemanticKITTI 隱藏測試集上達到 38.8% 的 mIoU。據我們所知，這是該排行榜上最佳的因果、單次掃描、單樣本結果，在相同限制下比先前最佳公開分數高出 2.1 個百分點。四次修正步驟搭配八視圖測試時資料增強則達到 39.2%，此結果不在此限制內。

English

Outdoor LiDAR semantic scene completion (SSC) recovers a dense semantic voxel grid from a scan observing 1% of the target volume, under class imbalance beyond 7,000x. We recast SSC as generative semantic scene completion (GSSC): a single discrete-diffusion formulation in three roles. First, paired sparse-dense scene synthesis (PS^3) generates matched sparse LiDAR observations with their dense semantic completions, addressing the long tail at its source and yielding the PS^3-SemanticKITTI corpus we train on alongside SemanticKITTI. Second, semantic-guided generative scene completion (SGSC) generates the scene from noise with multinomial discrete diffusion, conditioned on the sparse scan through a bird's-eye-view semantic map and a sparse 3D feature stream. Third, the same framework instead refines an existing completion in one flow-matching step: structured source discrete diffusion (S^2D^2). S^2D^2 improves the mIoU of SGSC's own output and every external SSC base tested, without base retraining or test-time adaptation. On the strongest base, one step without test-time augmentation reaches 38.8% mIoU on the SemanticKITTI hidden test. To our knowledge that is the best causal, single-sweep, single-sample result on that leaderboard, +2.1 pp over the previous best published score under the same restriction. Four correction steps with eight-view test-time augmentation reach 39.2%, outside that restriction.