FlexiSLM：一種動態且可控幀率的語音語言模型

摘要

口語語言模型（SLM）將大型語言模型（LLM）擴展至語音輸入與輸出。現有的SLM以固定幀率（例如25或12.5 Hz）表示語音，忽略了語音中隨時間變化的資訊密度，也無法在推理時靈活地在品質與速度之間進行取捨。近期音訊編碼器研究提出了動態幀率語音編碼技術，利用此非均勻特性實現了兩項新功能：極低的平均幀率以及幀率可控性。然而，此技術尚未應用於SLM。我們提出Flexible Spoken Language Model（FlexiSLM），這是首個支援語音輸入與輸出均具動態且可控幀率的SLM。利用動態幀率表示，FlexiSLM在其高品質運作點上優於包括Qwen2.5-Omni與Kimi-Audio在內的固定幀率7B模型。我們進一步驗證，FlexiSLM可準確地降至4.0 Hz；在6.25 Hz時，相較於12.5 Hz，推理時間約縮減一半，同時維持強勁的語音對語音品質。音訊樣本請見 https://flexislm.github.io 。

English

Spoken language models (SLMs) extend LLMs to speech input and output. Existing SLMs represent speech at fixed frame rates (e.g., 25 or 12.5 Hz), ignoring the time-varying information density of speech and offering no flexibility to trade off quality for speed at inference time. Recent audio tokenizer research has proposed dynamic frame rate speech coding, which exploits this non-uniformity and enables two new capabilities: very low average frame rates and frame rate controllability. However, this technique has not yet been applied to SLMs. We introduce Flexible Spoken Language Model (FlexiSLM), the first SLM that supports dynamic and controllable frame rates on both speech input and output. Using dynamic frame rate representations, FlexiSLM outperforms fixed-frame-rate 7B models including Qwen2.5-Omni and Kimi-Audio at its high-quality operating points. We further verify that FlexiSLM can be accurately steered down to 4.0 Hz; at 6.25 Hz, it roughly halves inference time relative to 12.5 Hz while retaining strong speech-to-speech quality. Audio samples are available at https://flexislm.github.io .