SpaceBlender：透過生成式3D場景混合創建富上下文的協作空間

摘要

目前越來越多人對使用生成式人工智慧來創建三維空間，用於虛擬實境（VR）應用感興趣。然而，如今的模型生成人工環境，無法支援需要整合使用者物理背景的協作任務。為了生成支援VR遠程存在的環境，我們引入了SpaceBlender，這是一個新穎的流程，利用生成式人工智慧技術將使用者的物理環境融入統一的虛擬空間中。該流程通過深度估計、網格對齊和擴散式空間完成的迭代過程，引導幾何先驗和適應性文本提示，將使用者提供的二維圖像轉換為充滿上下文的三維環境。在一項初步的受試者內研究中，20名參與者成對執行協作的VR親和圖表任務，我們將SpaceBlender與通用虛擬環境和最先進的場景生成框架進行比較，評估其創建適合協作的虛擬空間的能力。參與者讚賞SpaceBlender提供的增強熟悉感和上下文，但也指出生成環境中的複雜性可能會分散任務的注意力。根據參與者的反饋，我們提出改進流程的方向，並討論混合空間對不同情境的價值和設計。

English

There is increased interest in using generative AI to create 3D spaces for Virtual Reality (VR) applications. However, today's models produce artificial environments, falling short of supporting collaborative tasks that benefit from incorporating the user's physical context. To generate environments that support VR telepresence, we introduce SpaceBlender, a novel pipeline that utilizes generative AI techniques to blend users' physical surroundings into unified virtual spaces. This pipeline transforms user-provided 2D images into context-rich 3D environments through an iterative process consisting of depth estimation, mesh alignment, and diffusion-based space completion guided by geometric priors and adaptive text prompts. In a preliminary within-subjects study, where 20 participants performed a collaborative VR affinity diagramming task in pairs, we compared SpaceBlender with a generic virtual environment and a state-of-the-art scene generation framework, evaluating its ability to create virtual spaces suitable for collaboration. Participants appreciated the enhanced familiarity and context provided by SpaceBlender but also noted complexities in the generative environments that could detract from task focus. Drawing on participant feedback, we propose directions for improving the pipeline and discuss the value and design of blended spaces for different scenarios.

SpaceBlender：透過生成式3D場景混合創建富上下文的協作空間

SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending

摘要

Summary

Support

Support