MagiCodec：基於簡單遮罩高斯注入的編解碼器，實現高保真重建與生成

摘要

神經音頻編解碼器在將原始音頻波形高效映射為離散標記表示方面取得了顯著進展，這為當代音頻生成模型奠定了基礎。然而，現有的大多數編解碼器主要針對重建質量進行優化，往往以犧牲編碼標記的下游可建模性為代價。為克服這一瓶頸，我們引入了MagiCodec，一種基於單層流式Transformer的新型音頻編解碼器。MagiCodec設計了一個多階段訓練管道，結合了高斯噪聲注入和潛在正則化，明確旨在增強生成代碼的語義表達能力，同時保持高重建保真度。我們從頻域角度分析了噪聲注入的效果，證明了其在衰減高頻成分和促進穩健標記化方面的有效性。大量實驗評估表明，MagiCodec在重建質量和下游任務上均超越了最先進的編解碼器。值得注意的是，MagiCodec生成的標記呈現出類似自然語言的Zipf分佈，從而提高了與基於語言模型的生成架構的兼容性。代碼和預訓練模型可在https://github.com/Ereboas/MagiCodec獲取。

English

Neural audio codecs have made significant strides in efficiently mapping raw audio waveforms into discrete token representations, which are foundational for contemporary audio generative models. However, most existing codecs are optimized primarily for reconstruction quality, often at the expense of the downstream modelability of the encoded tokens. Motivated by the need to overcome this bottleneck, we introduce MagiCodec, a novel single-layer, streaming Transformer-based audio codec. MagiCodec is designed with a multistage training pipeline that incorporates Gaussian noise injection and latent regularization, explicitly targeting the enhancement of semantic expressiveness in the generated codes while preserving high reconstruction fidelity. We analytically derive the effect of noise injection in the frequency domain, demonstrating its efficacy in attenuating high-frequency components and fostering robust tokenization. Extensive experimental evaluations show that MagiCodec surpasses state-of-the-art codecs in both reconstruction quality and downstream tasks. Notably, the tokens produced by MagiCodec exhibit Zipf-like distributions, as observed in natural languages, thereby improving compatibility with language-model-based generative architectures. The code and pre-trained models are available at https://github.com/Ereboas/MagiCodec.

MagiCodec：基於簡單遮罩高斯注入的編解碼器，實現高保真重建與生成

MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation

摘要

Support