DriftMoE：一種基於專家混合模型處理概念漂移的方法

DriftMoE: A Mixture of Experts Approach to Handle Concept Drifts

July 24, 2025

作者: Miguel Aspis, Sebastián A. Cajas Ordónez, Andrés L. Suárez-Cetrulo, Ricardo Simón Carbajo

cs.AI

摘要

從非平穩數據流中學習並應對概念漂移，需要模型能夠在保持資源效率的同時進行即時適應。現有的自適應集成方法通常依賴於粗粒度的適應機制或簡單的投票方案，這些方法未能充分利用專業知識。本文介紹了DriftMoE，一種在線的專家混合（MoE）架構，通過新穎的協同訓練框架解決了這些限制。DriftMoE配備了一個緊湊的神經路由網絡，該網絡與一組增量霍夫丁樹專家共同訓練。其關鍵創新在於一個促進專家專業化的共生學習循環：路由網絡選擇最適合的專家進行預測，相關專家根據真實標籤進行增量更新，而路由網絡則使用一個多熱正確性掩碼來強化每個準確的專家，從而精煉其參數。這一反饋循環為路由網絡提供了清晰的訓練信號，同時加速了專家的專業化。我們在九個最先進的數據流學習基準上評估了DriftMoE的性能，這些基準涵蓋了突變、漸變和現實世界的漂移，並測試了兩種不同的配置：一種是專家專注於數據區域（多類變體），另一種是專家專注於單類專業化（基於任務的變體）。我們的結果表明，DriftMoE在與最先進的流學習自適應集成方法相比時，取得了競爭性的結果，為概念漂移適應提供了一種原則性和高效的方法。所有代碼、數據管道和可重現性腳本均可在我們的公共GitHub倉庫中獲取：https://github.com/miguel-ceadar/drift-moe。

English

Learning from non-stationary data streams subject to concept drift requires models that can adapt on-the-fly while remaining resource-efficient. Existing adaptive ensemble methods often rely on coarse-grained adaptation mechanisms or simple voting schemes that fail to optimally leverage specialized knowledge. This paper introduces DriftMoE, an online Mixture-of-Experts (MoE) architecture that addresses these limitations through a novel co-training framework. DriftMoE features a compact neural router that is co-trained alongside a pool of incremental Hoeffding tree experts. The key innovation lies in a symbiotic learning loop that enables expert specialization: the router selects the most suitable expert for prediction, the relevant experts update incrementally with the true label, and the router refines its parameters using a multi-hot correctness mask that reinforces every accurate expert. This feedback loop provides the router with a clear training signal while accelerating expert specialization. We evaluate DriftMoE's performance across nine state-of-the-art data stream learning benchmarks spanning abrupt, gradual, and real-world drifts testing two distinct configurations: one where experts specialize on data regimes (multi-class variant), and another where they focus on single-class specialization (task-based variant). Our results demonstrate that DriftMoE achieves competitive results with state-of-the-art stream learning adaptive ensembles, offering a principled and efficient approach to concept drift adaptation. All code, data pipelines, and reproducibility scripts are available in our public GitHub repository: https://github.com/miguel-ceadar/drift-moe.