Modern Continual Learning with Foundation Models, Evaluation Challenges, and Future Directions

Overview

Artificial intelligence models are traditionally trained on static datasets under the assumption of stable data distributions. However, in dynamic real-world environments, models must continuously learn and adapt without forgetting prior knowledge. Continual Learning (CL), or lifelong learning, addresses this by balancing the stability-plasticity dilemma: acquiring new knowledge (plasticity) while preserving existing memories (stability) to mitigate catastrophic forgetting.

This comprehensive review synthesizes the vast and rapidly evolving landscape of CL. It covers fundamental paradigms, theoretical foundations, methodological families (including modern parameter-efficient and prompt-based methods for foundation models), critical evaluation challenges, key applications, and future research directions.

CL Paradigms & Methodological Taxonomy

The paper structures the CL landscape across two main dimensions:

1. Continual Learning Paradigms:
   • Task-Incremental (TIL): Clear task boundaries with task identity provided during both training and inference (e.g., separate output heads).
   • Domain-Incremental (DIL): Task objective remains consistent, but input data distributions/domains shift over time; task identity is unknown during inference.
   • Class-Incremental (CIL): Sequential introduction of new classes without task identity during inference, requiring the model to classify across all learned classes simultaneously (the most challenging setting).
   • Emerging Paradigms: Online/streaming CL, few-shot CL, unsupervised CL, federated CL, and multi-agent CL.
2. Methodological Taxonomy:
   • Regularization-based: Imposes constraints on parameter updates to protect prior knowledge (e.g., EWC using Fisher Information Matrix, Synaptic Intelligence).
   • Replay-based: Mitigates forgetting by replaying a subset of raw samples (Experience Replay) or synthetically generated data (Generative Replay).
   • Architecture-based: Allocates task-specific parameters, subnetworks, or expandable modules to isolate knowledge (e.g., Progressive Neural Networks).
   • Optimization-based: Directs gradient updates to minimize destructive interference (e.g., GEM, A-GEM).
   • Representation-Learning: Focuses on learning stable, transferable, and domain-invariant features (e.g., via self-supervised pretraining).
   • Parameter-Efficient & Prompt-based: Adapts large pretrained foundation models by tuning only a tiny fraction of parameters (e.g., LoRA, prompt tuning like L2P, DualPrompt).

Key Applications

Continual learning is highly transformative across critical real-world domains:

• Healthcare & Medical Imaging: Adapting diagnostic systems to new scanners, imaging protocols, and patient populations (managing domain shifts).
• Robotics & Autonomous Systems: Enabling robots to learn new manipulation or navigation skills incrementally in unpredictable physical spaces.
• Natural Language Processing (NLP): Keeping chatbots and virtual assistants updated with evolving language patterns, slang, and user preferences.
• Recommender Systems: Dynamically updating recommendation catalogs and user preferences without forgetting historical interests.
• Cybersecurity: Learning new attack patterns and threat vectors in real-time to proactively defend network architectures.

Open Challenges & Evaluation Gaps

The review critically analyzes the primary bottlenecks preventing practical CL deployment:

• Benchmark Fragmentation: Lack of standardized dataset partitioning and task construction makes fair comparison across papers highly difficult.
• Evaluation Inconsistencies: Inconsistent replay memory budgets, lack of statistical variance reporting across random seeds, and over-reliance on artificial task splits.
• Computational & Memory Constraints: The high storage cost of replay buffers and the parameter growth of architecture-based methods limit on-device edge AI.
• Foundation Model Adaptation: Fully fine-tuning massive models is computationally prohibitive and risks damaging general knowledge; prompt-based methods are promising but still suffer from prompt interference.

Contributions

• Unified Comparative Framework: Provides a structured, multi-dimensional taxonomy categorizing CL paradigms, theoretical concepts, and methods.
• Comprehensive Survey of Modern Trends: Incorporates recent breakthroughs in transformers, prompt learning, parameter-efficient fine-tuning (PEFT), and multimodal foundation models.
• Analysis of Real-World Deployment Obstacles: Discusses practical constraints like privacy-preserving federated CL, memory limitations, and domain shifts in healthcare.
• Strategic Roadmap for Future Work: Identifies critical future directions, prioritizing standardized evaluation protocols, machine unlearning, and multimodal lifelong adaptation.

Conclusion

This review demonstrates that while continual learning has made significant strides, transitioning from controlled academic setups to robust, deployment-oriented lifelong systems remains an open frontier. Future progress will heavily depend on shifting toward standardized evaluation protocols, scalable parameter-efficient architectures, and multimodal adaptation frameworks. Developing AI systems that can safely and reliably adapt in continuously evolving environments is essential for the next generation of artificial intelligence.

We have submitted this paper to Engineering Applications of Artificial Intelligence, and we look forward to good results! 🙏

You can find our preprint at this link: 10.20944/preprints202605.1234.v1

Download Slides