SpaceBlender：通过生成式3D场景混合创建富有上下文的协作空间

摘要

目前越来越多的人对使用生成式人工智能来为虚拟现实（VR）应用程序创建3D空间表现出兴趣。然而，如今的模型生成人工环境，无法支持那些需要融入用户物理环境背景的协作任务。为了生成支持VR远程存在的环境，我们引入了SpaceBlender，这是一个新颖的流程，利用生成式人工智能技术将用户的物理环境融合到统一的虚拟空间中。该流程通过深度估计、网格对齐和基于扩散的空间补全的迭代过程，利用几何先验和自适应文本提示，将用户提供的2D图像转换为富有上下文的3D环境。在一项初步的被试研究中，20名参与者成对执行了协作的VR亲和图表任务，我们将SpaceBlender与通用虚拟环境和最先进的场景生成框架进行了比较，评估其创建适合协作的虚拟空间的能力。参与者赞赏SpaceBlender提供的增强熟悉感和上下文，但也指出生成环境中的复杂性可能会分散任务焦点。根据参与者的反馈，我们提出了改进流程的方向，并讨论了融合空间在不同场景中的价值和设计。

English

There is increased interest in using generative AI to create 3D spaces for Virtual Reality (VR) applications. However, today's models produce artificial environments, falling short of supporting collaborative tasks that benefit from incorporating the user's physical context. To generate environments that support VR telepresence, we introduce SpaceBlender, a novel pipeline that utilizes generative AI techniques to blend users' physical surroundings into unified virtual spaces. This pipeline transforms user-provided 2D images into context-rich 3D environments through an iterative process consisting of depth estimation, mesh alignment, and diffusion-based space completion guided by geometric priors and adaptive text prompts. In a preliminary within-subjects study, where 20 participants performed a collaborative VR affinity diagramming task in pairs, we compared SpaceBlender with a generic virtual environment and a state-of-the-art scene generation framework, evaluating its ability to create virtual spaces suitable for collaboration. Participants appreciated the enhanced familiarity and context provided by SpaceBlender but also noted complexities in the generative environments that could detract from task focus. Drawing on participant feedback, we propose directions for improving the pipeline and discuss the value and design of blended spaces for different scenarios.

SpaceBlender：通过生成式3D场景混合创建富有上下文的协作空间

SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending

摘要

Summary

Support

Support