梦境空间：使用文本驱动的全景纹理传播梦想中的房间空间

摘要

基于扩散的方法在生成2D媒体方面取得了显著成功。然而，在3D空间应用中（如XR/VR）实现类似的场景级网格纹理技能仍受限，主要是由于3D几何的复杂性以及对沉浸式自由视点渲染的必要性。本文提出了一种新颖的室内场景纹理框架，通过文本驱动的纹理生成提供具有迷人细节和真实空间连贯性的纹理。关键见解是首先从场景的中心视点想象出一种风格化的360°全景纹理，然后通过修补和模仿技术将其传播到其余区域。为了确保纹理对场景有意义且对齐，我们开发了一种新颖的粗到细的全景纹理生成方法，具有双纹理对齐，既考虑了捕捉场景的几何形状和纹理线索。为了在纹理传播过程中避免杂乱的几何形状，我们设计了一种分离策略，该策略在机密区域进行纹理修补，然后学习隐式模仿网络，以合成遮挡和微小结构区域的纹理。对真实世界室内场景进行了大量实验和沉浸式VR应用，证明了生成纹理的高质量以及在VR头显上引人入胜的体验。项目网页：https://ybbbbt.com/publication/dreamspace

English

Diffusion-based methods have achieved prominent success in generating 2D media. However, accomplishing similar proficiencies for scene-level mesh texturing in 3D spatial applications, e.g., XR/VR, remains constrained, primarily due to the intricate nature of 3D geometry and the necessity for immersive free-viewpoint rendering. In this paper, we propose a novel indoor scene texturing framework, which delivers text-driven texture generation with enchanting details and authentic spatial coherence. The key insight is to first imagine a stylized 360{\deg} panoramic texture from the central viewpoint of the scene, and then propagate it to the rest areas with inpainting and imitating techniques. To ensure meaningful and aligned textures to the scene, we develop a novel coarse-to-fine panoramic texture generation approach with dual texture alignment, which both considers the geometry and texture cues of the captured scenes. To survive from cluttered geometries during texture propagation, we design a separated strategy, which conducts texture inpainting in confidential regions and then learns an implicit imitating network to synthesize textures in occluded and tiny structural areas. Extensive experiments and the immersive VR application on real-world indoor scenes demonstrate the high quality of the generated textures and the engaging experience on VR headsets. Project webpage: https://ybbbbt.com/publication/dreamspace

梦境空间：使用文本驱动的全景纹理传播梦想中的房间空间

DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation

摘要

Support