Continual Learning in Transition: From Parameters to Systems

Continual learning has hit an inflection point. The field is moving away from tweaking model weights during training toward a broader vision where learning happens across multiple stages, through diverse mechanisms, and via external components that act as scaffolding around the model. This paper argues that classical continual learning—focused on how to update parameters without forgetting—is only one slice of a much larger phenomenon now unfolding.

The Shifting Landscape

Traditional continual learning asks a narrow question: How do we train a model so it learns new tasks without catastrophically forgetting old ones? Researchers tackled this through architectural tricks, regularization strategies, and clever parameter updates. But the field has expanded in three unexpected directions.

First, learning now happens beyond training. Test-time training lets models adapt at inference—refining themselves on each new input they encounter. Second, the mechanisms of learning have diversified. On-policy learning (where the model learns from data it generates itself) opens doors beyond the gradient-based optimization that dominates supervised learning. Third, learning is escaping the parameter space entirely. Models now augment themselves with external memory banks, skill libraries, and interaction protocols that store and retrieve knowledge independently of weights.

These shifts reflect a deeper transition: continual learning is becoming system-level rather than parameter-centric. A model is no longer a static function defined by its weights; it's a dynamic system that evolves through multiple channels.

A Framework to Make Sense of Chaos

The authors propose three orthogonal dimensions to map this transition:

When: Learning can occur during pre-training, post-training (fine-tuning), or at inference time. Classical continual learning lived entirely in the post-training phase. Now, inference-time adaptation is viable.

How: The optimization mechanism ranges from off-policy (learning from logged data) to on-policy (learning from self-generated trajectories) to beyond-gradient methods (evolutionary algorithms, reinforcement learning updates, or symbolic reasoning).

Where: Updates happen either inside the model (parameter changes) or outside it (memory writes, skill bank expansions, protocol modifications).

Each method in the continual learning literature occupies a different position in this 3D space. A replay-buffer approach sits at post-training, off-policy, internal. A system combining a language model with an external memory retriever lives at inference, beyond-gradient, external. This tri-axial view exposes previously hidden connections and reveals what the field is collectively building toward.

Why It Matters

The shift has practical implications. External components often scale better than parameter retuning—a memory bank can grow indefinitely without retraining. Test-time adaptation sidesteps the distribution shift problem by letting models self-correct. On-policy learning enables active exploration rather than passive consumption of fixed datasets.

More fundamentally, the framework clarifies what "continual learning" actually means in 2024. It's no longer about preventing forgetting in neural networks. It's about systems that can evolve, learn, and adapt across the entire pipeline—from initial training through deployment and beyond. The paper surveys representative methods across this expanded space, traces how the field moved from one corner of the 3D space to another, and flags unresolved challenges: How do we measure progress when learning is system-wide? How do external components and internal parameters interact? What guarantees can we offer?

By naming the transition and providing a map, the authors give the community a shared language for the work ahead.