持續學習的過渡

摘要

經典的持續學習（CL）主要聚焦於透過以參數為中心的機制（例如訓練策略、架構設計與權重調適）來使模型能夠更新並保留知識。然而，新興的研究範式正重新形塑持續學習的範疇，使其超越傳統的模型調適觀點。舉例而言，在策略學習（on-policy learning）拓寬了更新機制的空間；測試時訓練（test-time training）將持續學習從訓練階段延伸至推論階段；而外部輔助元件（如記憶、技能庫與互動協定）則將模型能力的演化邊界拓展至遠超靜態參數空間之外。整體而言，這些發展顯示持續學習正從以參數為中心的學習轉向系統層級的調適。為了刻劃此一轉變，我們透過三個維度審視持續學習的演化歷程：學習發生的「何時」（When）、「如何」（How）與「何處」（Where）。「如何」維度涵蓋離策略（off-policy）、在策略（on-policy）與超越梯度的最佳化機制；「何時」維度涵蓋預訓練、後訓練與推論階段等不同時期的演化；「何處」維度則區分內部參數更新與外部結構約束。以此三軸框架為基礎，我們系統性地回顧具代表性的方法，追蹤持續學習正在發生的轉變，並探討此範式轉移所帶來的關鍵挑戰、更深層的意涵與未來研究方向。

English

Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.g., training strategies, architectural designs, and weight adaptation. However, emerging paradigms are reshaping the scope of CL beyond this traditional model adaptation view. For instance, on-policy learning broadens the space of update mechanisms; test-time training extends CL from the training phase to inference; and external harness components such as memory, skill libraries, and interaction protocols extend the evolutionary boundaries of model capabilities far beyond the static parameter space. Collectively, these developments indicate a transition from parameter-centric learning toward system-level adaptation. To characterize this transition, we examine the evolution of continual learning through three dimensions: When, How, and Where learning occurs. The How dimension encompasses off-policy, on-policy, and beyond-gradient optimization mechanics. The When dimension captures evolution across pre-training, post-training, and inference-time stages. The Where dimension delineates updates occurring within internal parameters versus external structural constraints. Anchored by this tri-axial framework, we systematically survey representative methods, trace the ongoing transition of continual learning, and discuss the key challenges, broader implications, and future directions arising from this paradigm shift.