Show-o Turbo：朝向加速統一多模式理解與生成的方向

Show-o Turbo: Towards Accelerated Unified Multimodal Understanding and Generation

February 8, 2025

作者: Chenkai Xu, Xu Wang, Zhenyi Liao, Yishun Li, Tianqi Hou, Zhijie Deng

cs.AI

摘要

近年來，建立統一的多模態理解和生成模型引起了越來越多的研究興趣，其中Show-o被視為一個顯著的代表，展示了在文本到圖像和圖像到文本生成方面的巨大潛力。Show-o的推論包括逐步去噪圖像標記和自回歸解碼文本標記，然而，不幸的是，它在兩個方面都存在效率問題。本文介紹了Show-o Turbo 以彌合這一差距。我們首先基於文本標記的平行解碼，為Show-o中圖像和文本生成提出了統一的去噪觀點。然後，我們建議將一種適用於縮短擴散模型去噪過程的一致性蒸餾（CD）方法擴展到Show-o的多模態去噪軌跡。我們引入了一種軌跡分割策略和課程學習程序以改善訓練收斂。在實驗中，在文本到圖像生成方面，Show-o Turbo 在4個採樣步驟下展示了0.625的GenEval分數，而不使用無分類器引導（CFG），優於原始的具有8個步驟和CFG的Show-o；在圖像到文本生成方面，Show-o Turbo 顯示出1.5倍的加速，而不會明顯犧牲性能。代碼可在https://github.com/zhijie-group/Show-o-Turbo找到。

English

There has been increasing research interest in building unified multimodal understanding and generation models, among which Show-o stands as a notable representative, demonstrating great promise for both text-to-image and image-to-text generation. The inference of Show-o involves progressively denoising image tokens and autoregressively decoding text tokens, and hence, unfortunately, suffers from inefficiency issues from both sides. This paper introduces Show-o Turbo to bridge the gap. We first identify a unified denoising perspective for the generation of images and text in Show-o based on the parallel decoding of text tokens. We then propose to extend consistency distillation (CD), a qualified approach for shortening the denoising process of diffusion models, to the multimodal denoising trajectories of Show-o. We introduce a trajectory segmentation strategy and a curriculum learning procedure to improve the training convergence. Empirically, in text-to-image generation, Show-o Turbo displays a GenEval score of 0.625 at 4 sampling steps without using classifier-free guidance (CFG), outperforming that of the original Show-o with 8 steps and CFG; in image-to-text generation, Show-o Turbo exhibits a 1.5x speedup without significantly sacrificing performance. The code is available at https://github.com/zhijie-group/Show-o-Turbo.