AMRD：面向轻量化语音情感识别的自适应多教师关系蒸馏

摘要

设备端语音情感识别（SER）对实时应用至关重要，然而在SER任务中表现优异的大型自监督模型对边缘设备而言成本过高。多教师知识蒸馏可将其压缩为轻量级学生模型，但仍面临两个挑战：教师可靠性在不同批次间存在差异，且logit级蒸馏忽略了样本间的关系结构。我们提出自适应多教师关系蒸馏（AMRD）以同时解决这两个问题。在每个教师的logit相似度矩阵上应用单类支持向量机，分配逐批次权重以偏向更一致的教师。关系蒸馏损失对齐教师与学生模型的相似度矩阵，捕获logit匹配所遗漏的结构信息。在IEMOCAP和CREMA-D数据集上，跨四种学生架构的实验表明，AMRD在大多数设置下优于单教师蒸馏基线，且消融实验证实两个组件能带来互补的性能提升。

English

On-device speech emotion recognition (SER) is critical for real-time applications, yet large self-supervised models that excel at SER are too costly for edge devices. Multi-teacher knowledge distillation can compress them into a lightweight student, but two challenges remain: teacher reliability varies across batches, and logit-level distillation ignores inter-sample relational structure. We propose Adaptive Multi-teacher Relational Distillation (AMRD) to address both. A one-class SVM on each teacher's logit similarity matrix assigns per-batch weights favoring more coherent teachers. A relational distillation loss aligns teacher and student similarity matrices, capturing structure that logit matching misses. On IEMOCAP and CREMA-D datasets across four student architectures, AMRD outperforms single-teacher distillation baselines in most settings, and ablations confirm both components yield complementary gains.