ScenePainter：基於概念關係對齊的語義一致永續3D場景生成

摘要

永續3D場景生成旨在產生長範圍且連貫的3D視圖序列，這適用於長期視頻合成與3D場景重建。現有方法遵循「導航與想像」的模式，並依賴於外推技術來實現連續視圖的擴展。然而，生成視圖序列存在語意漂移問題，這源於外推模塊累積的偏差。為應對這一挑戰，我們提出了ScenePainter，一個用於語意一致3D場景生成的新框架，它將外推器的場景特定先驗與當前場景的理解對齊。具體而言，我們引入了一種名為SceneConceptGraph的層次圖結構，來構建多層次場景概念之間的關係，這指導外推器生成一致的新視圖，並可動態精煉以增強多樣性。大量實驗證明，我們的框架克服了語意漂移問題，並生成了更加一致且沉浸的3D視圖序列。項目頁面：https://xiac20.github.io/ScenePainter/。

English

Perpetual 3D scene generation aims to produce long-range and coherent 3D view sequences, which is applicable for long-term video synthesis and 3D scene reconstruction. Existing methods follow a "navigate-and-imagine" fashion and rely on outpainting for successive view expansion. However, the generated view sequences suffer from semantic drift issue derived from the accumulated deviation of the outpainting module. To tackle this challenge, we propose ScenePainter, a new framework for semantically consistent 3D scene generation, which aligns the outpainter's scene-specific prior with the comprehension of the current scene. To be specific, we introduce a hierarchical graph structure dubbed SceneConceptGraph to construct relations among multi-level scene concepts, which directs the outpainter for consistent novel views and can be dynamically refined to enhance diversity. Extensive experiments demonstrate that our framework overcomes the semantic drift issue and generates more consistent and immersive 3D view sequences. Project Page: https://xiac20.github.io/ScenePainter/.

ScenePainter：基於概念關係對齊的語義一致永續3D場景生成

ScenePainter: Semantically Consistent Perpetual 3D Scene Generation with Concept Relation Alignment

摘要

Support