FastVoiceGrad：一步擴散式語音轉換，具有對抗條件擴散蒸餾

FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillation

September 3, 2024

作者: Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka, Yuto Kondo

cs.AI

摘要

基於擴散的語音轉換（VC）技術，如VoiceGrad，因其在語音質量和說話者相似性方面的高VC性能而引起關注。然而，一個顯著的限制是多步反向擴散導致的緩慢推斷。因此，我們提出了FastVoiceGrad，一種新穎的一步擴散式VC，將迭代次數從幾十次減少到一次，同時繼承多步擴散式VC的高VC性能。我們使用對抗條件擴散蒸餾（ACDD）獲得模型，利用生成對抗網絡和擴散模型的能力，同時重新考慮抽樣中的初始狀態。一次任意到任意VC的評估表明，FastVoiceGrad實現了優於或與先前多步擴散式VC相當的VC性能，同時提高了推斷速度。音頻樣本可在以下網址找到：https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/fastvoicegrad/。

English

Diffusion-based voice conversion (VC) techniques such as VoiceGrad have attracted interest because of their high VC performance in terms of speech quality and speaker similarity. However, a notable limitation is the slow inference caused by the multi-step reverse diffusion. Therefore, we propose FastVoiceGrad, a novel one-step diffusion-based VC that reduces the number of iterations from dozens to one while inheriting the high VC performance of the multi-step diffusion-based VC. We obtain the model using adversarial conditional diffusion distillation (ACDD), leveraging the ability of generative adversarial networks and diffusion models while reconsidering the initial states in sampling. Evaluations of one-shot any-to-any VC demonstrate that FastVoiceGrad achieves VC performance superior to or comparable to that of previous multi-step diffusion-based VC while enhancing the inference speed. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/fastvoicegrad/.