一個模型，多種延遲：針對多樣化即時應用的通用語音增強

摘要

不同的即時語音應用各有其嚴格的延遲預算，通常需要針對每種場景分別訓練增強模型。本文提出一種一統通用的即時通用語音增強模型，可對演算法延遲與計算延遲進行精確控制。演算法延遲透過可配置的前向幀靈活調整；為避免因不同填補配置導致的學習效率低下，我們引入對應多種前向設定的平行卷積層。計算延遲則經由早退機制控制，允許在不同網路深度進行推理。為縮小專用模型與彈性模型之間的性能差距，我們提出結合共享到多重解碼器過渡的兩階段訓練策略。總體而言，本框架使單一模型能夠在無需重新訓練的情況下，部署於各種不同的延遲預算場景。

English

Different real-time speech applications impose distinct latency budgets, often requiring separately trained enhancement models for each scenario. In this paper, we propose a one-for-all, real-time universal speech enhancement model that provides explicit control over both algorithmic and computational latency. Algorithmic latency is flexibly adjusted via configurable look-ahead frames. To avoid learning inefficiency caused by varying padding configurations, we introduce parallel convolutional layers corresponding to different look-ahead settings. Computational latency is controlled through an early-exit mechanism, enabling inference at different network depths. To narrow the performance gap between specialized and flexible models, we propose a two-stage training strategy with a shared-to-multiple decoder transition. Overall, the proposed framework enables a single model to be deployed across diverse latency budgets without retraining separate models.