DriftMoE：一种基于专家混合模型的概念漂移处理方法

DriftMoE: A Mixture of Experts Approach to Handle Concept Drifts

July 24, 2025

作者: Miguel Aspis, Sebastián A. Cajas Ordónez, Andrés L. Suárez-Cetrulo, Ricardo Simón Carbajo

cs.AI

摘要

面对概念漂移的非稳态数据流学习，需要模型能够在保持资源效率的同时实现即时适应。现有的自适应集成方法通常依赖于粗粒度的适应机制或简单的投票方案，未能充分利用专业知识。本文提出了DriftMoE，一种在线专家混合（MoE）架构，通过新颖的协同训练框架解决了这些局限。DriftMoE配备了一个紧凑的神经路由器，与一组增量式霍夫丁树专家共同训练。其核心创新在于一个促进专家专业化的共生学习循环：路由器选择最合适的专家进行预测，相关专家根据真实标签进行增量更新，而路由器则利用一个多热正确性掩码优化其参数，该掩码强化了每位准确专家的表现。这一反馈循环为路由器提供了清晰的训练信号，同时加速了专家的专业化进程。我们在涵盖突变、渐变及现实世界漂移的九大数据流学习基准上评估了DriftMoE的性能，测试了两种配置：一种是专家专注于数据区域（多类别变体），另一种是专家聚焦于单类别专业化（任务导向变体）。结果表明，DriftMoE与最先进的流学习自适应集成方法相比具有竞争力，为概念漂移适应提供了一种原则性强且高效的途径。所有代码、数据管道及可复现性脚本均已公开于我们的GitHub仓库：https://github.com/miguel-ceadar/drift-moe。

English

Learning from non-stationary data streams subject to concept drift requires models that can adapt on-the-fly while remaining resource-efficient. Existing adaptive ensemble methods often rely on coarse-grained adaptation mechanisms or simple voting schemes that fail to optimally leverage specialized knowledge. This paper introduces DriftMoE, an online Mixture-of-Experts (MoE) architecture that addresses these limitations through a novel co-training framework. DriftMoE features a compact neural router that is co-trained alongside a pool of incremental Hoeffding tree experts. The key innovation lies in a symbiotic learning loop that enables expert specialization: the router selects the most suitable expert for prediction, the relevant experts update incrementally with the true label, and the router refines its parameters using a multi-hot correctness mask that reinforces every accurate expert. This feedback loop provides the router with a clear training signal while accelerating expert specialization. We evaluate DriftMoE's performance across nine state-of-the-art data stream learning benchmarks spanning abrupt, gradual, and real-world drifts testing two distinct configurations: one where experts specialize on data regimes (multi-class variant), and another where they focus on single-class specialization (task-based variant). Our results demonstrate that DriftMoE achieves competitive results with state-of-the-art stream learning adaptive ensembles, offering a principled and efficient approach to concept drift adaptation. All code, data pipelines, and reproducibility scripts are available in our public GitHub repository: https://github.com/miguel-ceadar/drift-moe.