代理式語音識別的語音記憶

摘要

我們提出 Voice Memory，一種專為代理語音識別設計的純推理方案：在串流處理時，一個凍結的校正器讀取單一領域記憶檔，並針對每個語句決定要採用假設結果，或放棄該結果而保留最佳辨識結果。非同步地，一個以分數為門檻的優化器會透過有界編輯來修訂該檔案，僅在編輯能確實改善保留分數時才予以接受。此架構延伸自傳統的 ASR-LM 框架，我們將此拆分稱為「聽者-思考者」架構；兩個角色僅透過記憶耦合，因此無需改變權重，且所學技能保持可審計與可移植性。約束被證明是此循環所發現的關鍵技能：在金融新聞領域中，無約束的生成式錯誤校正（GER）會過度校正，在其多達 64% 的編輯中破壞正確的詞元，而 Voice Memory 則將此比例降至 35%。在十個 HyPoradise 領域中，搭配開放校正器使用 Voice Memory，加權詞錯誤率從 8.36% 降至 7.52%（加入三個情境範例後更降至 7.47%），且沒有任何資料集的表現低於其最佳單一辨識基準線；收益集中在可恢復空間最大的領域，包括航空旅行指令（從 8.40% 降至 3.40%）與嘈雜的遠場語音（CHiME-4，從 12.69% 降至 10.46%）。此記憶可跨校正器家族遷移，且推理路徑不增加任何參數。我們提供示範與範例程式碼供未來研究使用。

English

We present Voice Memory, a inference-only scheme for agentic speech recognition: at stream time, a frozen corrector reads a single per-domain memory.md and decides per utterance whether to act on the hypothesis or abstain and keep the 1-best. Asynchronously, a score-gated optimizer revises that file through bounded edits, accepting an edit only when it strictly improves a held-out score. Extended from classical ASR-LM framework, we refer this split the listener-thinker architecture; the two roles are coupled only through the memory, so no weights change and the learned skill stays auditable and portable. Restraint turns out to be the operative skill this loop discovers: unconstrained generative error correction (GER) over-corrects, breaking correct tokens on up to 64% of its edits on financial news, and Voice Memory, reduces this rate to 35%. Across ten HyPoradise domains with an open corrector, Voice Memory, lowers weighted word error rate from 8.36% to 7.52% (7.47% with three added in-context examples) without regressing any dataset below its 1-best baseline; gains concentrate where recoverable headroom is largest, including air-travel commands (8.40% to 3.40%) and noisy far-field speech (CHiME-4, 12.69% to 10.46%). The memory transfers across corrector families and adds zero parameters to the inference path. A demo and example code are provided for future studies.