薛定論橋梁在文本轉語音合成方面勝過擴散模型

摘要

在文本轉語音（TTS）合成中，擴散模型已經取得了令人期待的生成質量。然而，由於預定義的數據到噪聲擴散過程，它們的先驗分佈被限制在一個嘈雜的表示中，這提供了很少有關生成目標的信息。在這項工作中，我們提出了一種新穎的 TTS 系統，名為 Bridge-TTS，首次嘗試用乾淨且確定性的先驗替代已建立的基於擴散的 TTS 方法中的嘈雜高斯先驗，這提供了目標的強結構信息。具體來說，我們利用從文本輸入獲得的潛在表示作為我們的先驗，並在它與地面真實的 mel-頻譜圖之間建立一個完全可追踪的薛定輪橋，從而實現數據到數據的過程。此外，我們公式的可追踪性和靈活性使我們能夠在實驗中研究設計空間，例如噪聲時間表，並開發隨機和確定性取樣器。在 LJ-Speech 數據集上的實驗結果顯示了我們的方法在合成質量和取樣效率方面的有效性，明顯優於我們的擴散對應物 Grad-TTS 在 50 步 / 1000 步合成以及強快速 TTS 模型在少步驟情況下的表現。項目頁面：https://bridge-tts.github.io/

English

In text-to-speech (TTS) synthesis, diffusion models have achieved promising generation quality. However, because of the pre-defined data-to-noise diffusion process, their prior distribution is restricted to a noisy representation, which provides little information of the generation target. In this work, we present a novel TTS system, Bridge-TTS, making the first attempt to substitute the noisy Gaussian prior in established diffusion-based TTS methods with a clean and deterministic one, which provides strong structural information of the target. Specifically, we leverage the latent representation obtained from text input as our prior, and build a fully tractable Schrodinger bridge between it and the ground-truth mel-spectrogram, leading to a data-to-data process. Moreover, the tractability and flexibility of our formulation allow us to empirically study the design spaces such as noise schedules, as well as to develop stochastic and deterministic samplers. Experimental results on the LJ-Speech dataset illustrate the effectiveness of our method in terms of both synthesis quality and sampling efficiency, significantly outperforming our diffusion counterpart Grad-TTS in 50-step/1000-step synthesis and strong fast TTS models in few-step scenarios. Project page: https://bridge-tts.github.io/

薛定論橋梁在文本轉語音合成方面勝過擴散模型

Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesis

摘要

Support