快速定時條件潛在音頻擴散

摘要

從文字提示生成長形式44.1kHz立體聲音頻可能需要大量計算。此外，大多數先前的研究並未處理音樂和音效在持續時間上自然變化的問題。我們的研究專注於使用生成模型，以有效方式生成長形式、可變長度的44.1kHz立體音樂和音效，並以文字提示作為基礎。穩定音頻基於潛在擴散，其潛在性由完全卷積變分自編碼器定義。它受文字提示和時間嵌入的條件限制，允許對生成的音樂和音效的內容和長度進行精細控制。穩定音頻能夠在A100 GPU上以8秒的速度在44.1kHz下渲染長達95秒的立體信號。儘管它具有計算效率和快速推論的特點，但在兩個公開的文本轉音樂和音頻基準測試中，它仍然是最佳之一，與最先進的模型不同，它能夠生成具有結構和立體音效的音樂。

English

Generating long-form 44.1kHz stereo audio from text prompts can be computationally demanding. Further, most previous works do not tackle that music and sound effects naturally vary in their duration. Our research focuses on the efficient generation of long-form, variable-length stereo music and sounds at 44.1kHz using text prompts with a generative model. Stable Audio is based on latent diffusion, with its latent defined by a fully-convolutional variational autoencoder. It is conditioned on text prompts as well as timing embeddings, allowing for fine control over both the content and length of the generated music and sounds. Stable Audio is capable of rendering stereo signals of up to 95 sec at 44.1kHz in 8 sec on an A100 GPU. Despite its compute efficiency and fast inference, it is one of the best in two public text-to-music and -audio benchmarks and, differently from state-of-the-art models, can generate music with structure and stereo sounds.

快速定時條件潛在音頻擴散

Fast Timing-Conditioned Latent Audio Diffusion

摘要

Support