FlexiTex：利用视觉引导增强纹理生成

FlexiTex: Enhancing Texture Generation with Visual Guidance

September 19, 2024

作者: DaDong Jiang, Xianghui Yang, Zibo Zhao, Sheng Zhang, Jiaao Yu, Zeqiang Lai, Shaoxiong Yang, Chunchao Guo, Xiaobo Zhou, Zhihui Ke

cs.AI

摘要

最近的纹理生成方法取得了令人印象深刻的成果，这要归功于它们利用大规模文本到图像扩散模型中强大的生成先验知识。然而，抽象的文本提示在提供全局纹理或形状信息方面存在局限，导致纹理生成方法产生模糊或不一致的图案。为了解决这个问题，我们提出了FlexiTex，通过视觉引导嵌入丰富信息以生成高质量纹理。FlexiTex的核心是视觉引导增强模块，它从视觉引导中融入更具体的信息，以减少文本提示中的歧义并保留高频细节。为了进一步增强视觉引导，我们引入了一个自动设计方向提示的Direction-Aware Adaptation模块，根据不同的摄像机姿势避免了Janus问题，并保持语义上的全局一致性。受益于视觉引导，FlexiTex产生了定量和定性上令人满意的结果，展示了其推动纹理生成在实际应用中的潜力。

English

Recent texture generation methods achieve impressive results due to the powerful generative prior they leverage from large-scale text-to-image diffusion models. However, abstract textual prompts are limited in providing global textural or shape information, which results in the texture generation methods producing blurry or inconsistent patterns. To tackle this, we present FlexiTex, embedding rich information via visual guidance to generate a high-quality texture. The core of FlexiTex is the Visual Guidance Enhancement module, which incorporates more specific information from visual guidance to reduce ambiguity in the text prompt and preserve high-frequency details. To further enhance the visual guidance, we introduce a Direction-Aware Adaptation module that automatically designs direction prompts based on different camera poses, avoiding the Janus problem and maintaining semantically global consistency. Benefiting from the visual guidance, FlexiTex produces quantitatively and qualitatively sound results, demonstrating its potential to advance texture generation for real-world applications.