FastVoiceGrad：基于一步扩散的语音转换与对抗条件扩散蒸馏

FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillation

September 3, 2024

作者: Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka, Yuto Kondo

cs.AI

摘要

基于扩散的语音转换（VC）技术，如VoiceGrad，因其在语音质量和说话人相似度方面的高VC性能而备受关注。然而，一个显著的局限是多步反向扩散导致的推断速度缓慢。因此，我们提出了FastVoiceGrad，一种新颖的一步扩散型VC，将迭代次数从几十次减少到一次，同时继承多步扩散型VC的高VC性能。我们利用对抗条件扩散蒸馏（ACDD）获得模型，利用生成对抗网络和扩散模型的能力，同时重新考虑采样中的初始状态。一次任意到任意VC的评估表明，FastVoiceGrad实现了优于或与先前多步扩散型VC相媲美的VC性能，同时提高了推断速度。音频样本可在以下网址找到：https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/fastvoicegrad/.

English

Diffusion-based voice conversion (VC) techniques such as VoiceGrad have attracted interest because of their high VC performance in terms of speech quality and speaker similarity. However, a notable limitation is the slow inference caused by the multi-step reverse diffusion. Therefore, we propose FastVoiceGrad, a novel one-step diffusion-based VC that reduces the number of iterations from dozens to one while inheriting the high VC performance of the multi-step diffusion-based VC. We obtain the model using adversarial conditional diffusion distillation (ACDD), leveraging the ability of generative adversarial networks and diffusion models while reconsidering the initial states in sampling. Evaluations of one-shot any-to-any VC demonstrate that FastVoiceGrad achieves VC performance superior to or comparable to that of previous multi-step diffusion-based VC while enhancing the inference speed. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/fastvoicegrad/.