面向智能代理语音识别的语音记忆

摘要

我们提出Voice Memory，这是一种仅推理方案，专为智能语音识别设计：在流式处理过程中，一个冻结的校正器读取单个领域相关的memory.md文件，并针对每个语句决定是采纳假设还是放弃并保留最优结果。与此同时，一个基于分数门控的优化器通过有限编辑异步修改该文件，仅当编辑能严格提升留存分数时才予以接受。这一架构从经典的ASR-LM框架扩展而来，我们称之为“监听-思考”架构；两个角色仅通过记忆文件耦合，因此无需改变权重，所学技能保持可审计和可移植。事实证明，约束是该循环所发现的关键操作技能：无约束的生成式错误校正（GER）会过度校正，在财经新闻中高达64%的编辑破坏了正确词元，而Voice Memory将这一比率降至35%。在十个HyPoradise领域中使用开放校正器，Voice Memory将加权词错误率从8.36%降至7.52%（增加三个上下文示例后为7.47%），且没有任何数据集的性能低于其1-best基线；改进主要集中在可恢复空间最大的领域，包括航空旅行命令（从8.40%降至3.40%）和嘈杂远场语音（CHiME-4，从12.69%降至10.46%）。记忆文件可跨校正器家族迁移，且推理路径中不增加任何参数。我们提供了演示和示例代码以供未来研究。

English

We present Voice Memory, a inference-only scheme for agentic speech recognition: at stream time, a frozen corrector reads a single per-domain memory.md and decides per utterance whether to act on the hypothesis or abstain and keep the 1-best. Asynchronously, a score-gated optimizer revises that file through bounded edits, accepting an edit only when it strictly improves a held-out score. Extended from classical ASR-LM framework, we refer this split the listener-thinker architecture; the two roles are coupled only through the memory, so no weights change and the learned skill stays auditable and portable. Restraint turns out to be the operative skill this loop discovers: unconstrained generative error correction (GER) over-corrects, breaking correct tokens on up to 64% of its edits on financial news, and Voice Memory, reduces this rate to 35%. Across ten HyPoradise domains with an open corrector, Voice Memory, lowers weighted word error rate from 8.36% to 7.52% (7.47% with three added in-context examples) without regressing any dataset below its 1-best baseline; gains concentrate where recoverable headroom is largest, including air-travel commands (8.40% to 3.40%) and noisy far-field speech (CHiME-4, 12.69% to 10.46%). The memory transfers across corrector families and adds zero parameters to the inference path. A demo and example code are provided for future studies.