VibeVoice-ASR-串流技術報告

摘要

傳統的說話者歸屬語音辨識（ASR）系統將語音辨識與說話者分段視為兩項獨立的任務。近來，VibeVoice-ASR 等端到端模型已將這兩項任務統一到單一模型之中。然而，現有的統一模型多半仍以離線辨識為主，難以滿足即時語音助理與代理程式對低延遲的需求。為了解決這個問題，我們提出 VibeVoice-ASR-Streaming，這是首批以 LLM 為基礎、針對串流式說話者歸屬 ASR 的端到端方法之一。它將固定大小的音訊區塊、少量的前瞻音訊與先前的文字交錯處理，使模型能在語音到達時即時產生「誰說了什麼」，無需額外的說話者分段階段。在轉錄準確度方面，我們的 7B 模型在五個評估集上達到最低的平均 WER/CER；在說話者歸屬方面，它在 13 個評估設定中的 12 個達到最佳或並列最佳的表現。我們釋出 1.5B 與 7B 模型的權重，以及推論程式碼。

English

Traditional speaker-attributed ASR systems treated ASR and speaker diarization as two separate tasks. Recently, end-to-end models such as VibeVoice-ASR have unified the two tasks within a single model. However, existing unified models still mainly support offline recognition, making it difficult to meet the low-latency requirements of real-time voice assistants and agents. To tackle this issue, we present VibeVoice-ASR-Streaming, one of the first LLM-based end-to-end approaches to streaming speaker-attributed ASR. It interleaves fixed-size audio chunks, a small amount of lookahead audio and previous text. This allows the model to produce ''who said what'' as speech arrives, without a separate diarization stage. For transcription accuracy, our 7B model achieves the lowest average WER/CER across five evaluation sets. For speaker attribution, it achieves the best or tied-best on 12 of 13 evaluation settings. We release the 1.5B and 7B model weights together with inference code.