ChatPaper.aiChatPaper

FlexPainter:靈活且多視角一致的紋理生成

FlexPainter: Flexible and Multi-View Consistent Texture Generation

June 3, 2025
作者: Dongyu Yan, Leyi Wu, Jiantao Lin, Luozhou Wang, Tianshuo Xu, Zhifei Chen, Zhen Yang, Lie Xu, Shunsi Zhang, Yingcong Chen
cs.AI

摘要

紋理貼圖生成是3D建模中的重要環節,直接影響渲染品質。近年來,基於擴散模型的方法為紋理生成開闢了新途徑。然而,受限的控制靈活性與有限的提示模式可能阻礙創作者獲得理想結果。此外,多視角生成圖像間的不一致性常導致紋理生成品質欠佳。為解決這些問題,我們提出了FlexPainter,一種新穎的紋理生成流程,它支持靈活的多模態條件引導,並實現高度一致的紋理生成。我們構建了一個共享的條件嵌入空間,以實現不同輸入模態間的靈活聚合。利用此嵌入空間,我們提出了一種基於圖像的CFG方法,分解結構與風格信息,實現基於參考圖像的風格化。借助圖像擴散先驗中的3D知識,我們首先使用網格表示同時生成多視角圖像,以增強全局理解。同時,我們在擴散採樣過程中引入了視角同步與自適應加權模塊,進一步確保局部一致性。最後,結合3D感知的紋理補全模型與紋理增強模型,生成無縫、高分辨率的紋理貼圖。全面實驗表明,我們的框架在靈活性與生成品質上均顯著優於現有最先進方法。
English
Texture map production is an important part of 3D modeling and determines the rendering quality. Recently, diffusion-based methods have opened a new way for texture generation. However, restricted control flexibility and limited prompt modalities may prevent creators from producing desired results. Furthermore, inconsistencies between generated multi-view images often lead to poor texture generation quality. To address these issues, we introduce FlexPainter, a novel texture generation pipeline that enables flexible multi-modal conditional guidance and achieves highly consistent texture generation. A shared conditional embedding space is constructed to perform flexible aggregation between different input modalities. Utilizing such embedding space, we present an image-based CFG method to decompose structural and style information, achieving reference image-based stylization. Leveraging the 3D knowledge within the image diffusion prior, we first generate multi-view images simultaneously using a grid representation to enhance global understanding. Meanwhile, we propose a view synchronization and adaptive weighting module during diffusion sampling to further ensure local consistency. Finally, a 3D-aware texture completion model combined with a texture enhancement model is used to generate seamless, high-resolution texture maps. Comprehensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods in both flexibility and generation quality.
PDF112June 6, 2025