Skip to content
Proyectos

optimass/continual_learning_papers

Ver en GitHub

optimass/continual_learning_papers has 745 GitHub stars and 82 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-15 11:26 UTC. Last push 2023-07-25.

Cómo se miden estas cifras

745 estrellas82 Bifurcaciones

Crecimiento de estrellas medido

in 24h
0
in 7 days
Recopilando historial
in 15 days
Recopilando historial
in 1 month
Recopilando historial

Cambios netos en períodos que se solapan, no totales diarios. Los datos no disponibles se dejan en blanco. Las estrellas indican interés, no calidad.

Acerca de

Relevant papers in Continual Learning

Aprendizaje

README.md

Original

Continual Learning Literature

This repository is maintained by Massimo Caccia and Timothée Lesort don't hesitate to send us an email to collaborate or fix some entries ({massimo.p.caccia , t.lesort} at gmail.com). The automation script of this repo is adapted from Automatic_Awesome_Bibliography.

For contributing to the repository please follow the process here

You can directly use our bib.tex in overleaf with this link

Outline

Classics

Argues knowledge transfer is essential if robots are to learn control with moderate learning times

Introduces CL and reveals the catastrophic forgetting problem

Empirical Study

introduces a super simple methods that outperforms almost all methods in all of the CL benchmarks. We need new better benchamrks

Extensive empirical study of CL methods (in the multi-head setting)

An extensive review of CL methods in three different scenarios (task-, domain-, and class-incremental learning)

Proposes desideratas and reexamines the evaluation protocol

Investigates CF in neural networks

Surveys

  • An Investigation of Replay-based Approaches for Continual Learning , (IJCNN 2021) by Bagus, Benedikt and Gepperth, Alexander [bib]
  • Embracing Change: Continual Learning in Deep Neural Networks, (2020) by Hadsell, Raia, Rao, Dushyant, Rusu, Andrei and Pascanu, Razvan [bib]
  • Towards Continual Reinforcement Learning: A Review and Perspectives, (2020) by Khimya Khetarpal, Matthew Riemer, Irina Rish and Doina Precup [bib]

A review on continual reinforcement learning

propose a consolidated view to bridge continual learning, active learning and open set recognition in DNNs

An extensive review of CL in Natural Language Processing (NLP)

An extensive review of CL

A survey on incremental learning and the various applications fields

Influentials

More efficient GEM; Introduces online continual learning

Proposes desideratas and reexamines the evaluation protocol

Proposes a reference architecture for a continual learning system

A model that alliviates CF via constrained optimization

Introduces generative replay

Investigates CF in neural networks

New Settings or Metrics

A setup and benchmark to evaluate lifelong learning models in more real-life aligned scenarios.

A library that unifies Continual Supervised and Continual Reinforcement Learning research

proposes a new continual few-shot setting where spacial and temporal context can be leveraged to and unseen classes need to be predicted

(title is a good enough summary)

Proposes a new approach to CL evaluation more aligned with real-life applications, bringing CL closer to Online Learning and Open-World learning

method for compositional continual learning of sequence-to-sequence models

propose a consolidated view to bridge continual learning, active learning and open set recognition in DNNs

  • Don't forget, there is more than forgetting: new metrics for Continual Learning, (arXiv 2018) by D{'\i}az-Rodr{'\i}guez, Natalia, Lomonaco, Vincenzo, Filliat, David and Maltoni, Davide [bib]

introduces a CL score that takes more than just forgetting into account

General Continual Learning Methods (SL and RL)

Task-Agnostic Continual Learning

Uses Bayes by Backprop for variational Continual Learning.

Proposes a new approach to CL evaluation more aligned with real-life applications, bringing CL closer to Online Learning and Open-World learning

  • iTAML: An Incremental Task-Agnostic Meta-learning Approach, (CVPR 2020) by Rajasegaran, Jathushan, Khan, Salman, Hayat, Munawar, Khan, Fahad Shahbaz and Shah, Mubarak [bib]
  • Continual Unsupervised Representation Learning , (2019) by Dushyant Rao, Francesco Visin, Andrei A. Rusu, Yee Whye Teh, Razvan Pascanu and Raia Hadsell [bib]

Introduces unsupervised continual learning (no task label and no task boundaries)

This paper introduces expansion-based approach for task-free continual learning

Introduces an optimizer for CL that relies on closed form updates of mu and sigma of BNN; introduce label trick for class learning (single-head) but warning: it isn't really task-agnostic

Regularization Methods

continual learning for non-stationary data using Bayesian neural networks and memory-based online variational Bayes

Improved results and interpretation of VCL.

Introduces VCL with uncertainty measured for neurons instead of weights.

functional regularisation for Continual Learning: avoids forgetting a previous task by constructing and memorising an approximate posterior belief over the underlying task-specific function

Introduces an optimizer for CL that relies on closed form updates of mu and sigma of BNN; introduce label trick for class learning (single-head) but warning: it isn't really task-agnostic

Conceptor-Aided Backprop (CAB): gradients are shielded by conceptors against degradation of previously learned tasks

Introducing a hard attention idea with binary masks

Formalizes the shortcomings of multi-head evaluation, as well as the importance of replay in single-head setup. Presenting an improved version of EWC.

A new P\&C architecture; online EWC for keeping the knowledge about the previous task, knowledge for keeping the knowledge about the current task (Multi-head setting, RL)

Improves on VCL

Importance of parameter measured based on their contribution to change in the learned prediction function

Synaptic Intelligence (SI). Importance of parameter measured based on their contribution to change in the loss.

  • Overcoming catastrophic forgetting by incremental moment matching, (NeurIPS 2017) by Lee, Sang-Woo, Kim, Jin-Hwa, Jun, Jaehyun, Ha, Jung-Woo and Zhang, Byoung-Tak [bib]

Distillation Methods

Explores Continual Self-Supervised Learning and proposes a simple and effective feature distillation method

Introduces Memory-Constrained Online Continual Learning, a setting where no information can be transferred between tasks, and proposes a distillation-based solution (Batch-level Distillation)

Novel knowledge distillation that trades efficiently rigidity and plasticity to learn large amount of small tasks

Introducing global distillation loss and balanced finetuning; leveraging unlabeled data in the open world setting (Single-head setting)

Introducing bias parameters to the last fully connected layer to resolve the data imbalance issue (Single-head setting)

  • Continual Reinforcement Learning deployed in Real-life using PolicyDistillation and Sim2Real Transfer, (ICML Workshop 2019) by *Kalifou, René Traoré, Caselles-Dupré, Hugo, Lesort, Timothée, Sun, Te, Diaz-Rodriguez, Natalia and Filliat, David * [bib]
  • Lifelong learning via progressive distillation and retrospection , (ECCV 2018) by Hou, Saihui, Pan, Xinyu, Change Loy, Chen, Wang, Zilei and Lin, Dahua [bib]

Introducing an expert of the current task in the knowledge distillation method (Multi-head setting)

Finetuning the last fully connected layer with a balanced dataset to resolve the data imbalance issue (Single-head setting)

Functional regularization through distillation (keeping the output of the updated network on the new data close to the output of the old network on the new data)

Binary cross-entropy loss for representation learning \& exemplar memory (or coreset) for replay (Single-head setting)

Rehearsal Methods

Use graphs to link saved samples and improve the memory quality.

Use Shapley Value adversarially to select which samples to relay

introduces a super simple methods that outperforms almost all methods in all of the CL benchmarks. We need new better benchamrks

Novel knowledge distillation that trades efficiently rigidity and plasticity to learn large amount of small tasks

More efficient GEM; Introduces online continual learning

projecting the gradients from new tasks onto a subspace in which the neural network output on previous task does not change and the projected gradient is still in a useful direction for learning the new task

sample selection as a constraint reduction problem based on the constrained optimization view of continual learning

Controlled sampling of memories for replay to automatically rehearse on tasks currently undergoing the most forgetting

Uses stacks of VQ-VAE modules to progressively compress the data stream, enabling better rehearsal

Introducing bias parameters to the last fully connected layer to resolve the data imbalance issue (Single-head setting)

  • Learning a Unified Classifier Incrementally via Rebalancing, (CVPR 2019) by Hou, Saihui, Pan, Xinyu, Loy, Chen Change, Wang, Zilei and Lin, Dahua [bib]
  • Continual Reinforcement Learning deployed in Real-life using PolicyDistillation and Sim2Real Transfer, (ICML Workshop 2019) by *Kalifou, René Traoré, Caselles-Dupré, Hugo, Lesort, Timothée, Sun, Te, Diaz-Rodriguez, Natalia and Filliat, David * [bib]
  • Experience replay for continual learning , (NeurIPS 2019) by Rolnick, David, Ahuja, Arun, Schwarz, Jonathan, Lillicrap, Timothy and Wayne, Gregory [bib]
  • Gradient Episodic Memory for Continual Learning , (NeurIPS 2017) by Lopez-Paz, David and Ranzato, Marc-Aurelio [bib]

A model that alliviates CF via constrained optimization

Binary cross-entropy loss for representation learning \& exemplar memory (or coreset) for replay (Single-head setting)

Generative Replay Methods

introdudes Dynamic generative memory (DGM) which relies on conditional generative adversarial networks with learnable connection plasticity realized with neural masking

Extensive evaluation of CL methods for generative modeling

Extensive evaluation of generative replay methods

smarter Generative Replay

Introduces generative replay

Dynamic Architectures or Routing Methods

bio-inspired method which dynamically restrict and route information in a context-specific manner

a binary mask over the network is inferred based on the input, and only the masked part of the network is used to train/infer

a network trained to do CL where select subnetworks are used to learn each task; these subnetworks are chosen a priori

  • Incremental Learning through Deep Adaptation , (2018) by Amir Rosenfeld and John K. Tsotsos [bib]
  • Packnet: Adding multiple tasks to a single network by iterative pruning, (CVPR 2018) by Mallya, Arun and Lazebnik, Svetlana [bib]
  • Piggyback: Adapting a single network to multiple tasks by learning to mask weights, (ECCV 2018) by Mallya, Arun, Davis, Dillon and Lazebnik, Svetlana [bib]
  • Continual Learning in Practice , (NeurIPS Workshop 2018) by Diethe, Tom, Borchert, Tom, Thereska, Eno, Pigem, Borja de Balle and Lawrence, Neil [bib]

Proposes a reference architecture for a continual learning system

  • Growing a brain: Fine-tuning by increasing model capacity, (CVPR 2017) by Wang, Yu-Xiong, Ramanan, Deva and Hebert, Martial [bib]
  • PathNet: Evolution Channels Gradient Descent in Super Neural Networks , (2017) by Chrisantha Fernando and Dylan Banarse and Charles Blundell and Yori Zwols and David Ha and Andrei A. Rusu and Alexander Pritzel and Daan Wierstra [bib]
  • Lifelong learning with dynamically expandable networks, (arXiv 2017) by Yoon, Jaehong, Yang, Eunho, Lee, Jeongtae and Hwang, Sung Ju [bib]
  • Progressive Neural Networks , (2016) by Rusu, A.~A., Rabinowitz, N.~C., Desjardins, G. and Soyer, H., Kirkpatrick, J., Kavukcuoglu, K. and Pascanu, R. and Hadsell, R. [bib]

Each task have a specific model connected to the previous ones

Hybrid Methods

Learning task-conditioned hypernetworks for continual learning as well as task embeddings; hypernetwors offers good model compression.

Approach leverages the principles of deep model compression, critical weights selection, and progressive networks expansion. All enforced in an iterative manner

This paper introduces expansion-based approach for task-free continual learning

Continual Few-Shot Learning

proposes a new continual few-shot setting where spacial and temporal context can be leveraged to and unseen classes need to be predicted

(title is a good enough summary)

Proposes a new approach to CL evaluation more aligned with real-life applications, bringing CL closer to Online Learning and Open-World learning

defines Online Meta-learning; propsoses Follow the Meta Leader (FTML) (~ Online MAML)

Meta-learns a tasks structure; continual adaptation via non-parametric prior

Formulates an online learning procedure that uses SGD to update model parameters, and an EM with a Chinese restaurant process prior to develop and maintain a mixture of models to handle non-stationary task distribution

Introduces What \& How framework; enables Task Agnostic CL with meta learned task inference

Meta-Continual Learning

Proposes an online replay-based meta-continual learning algorithm with learning-rate modulation to mitigate catastrophic forgetting

Follow-up of OML. Meta-learns an activation-gating function instead.

Introduces Learns how to continually learn (OML) i.e. learns how to do online updates without forgetting.

Learning MAML in a Meta continual learning way slows down forgetting

Lifelong Reinforcement Learning

Studies Q-Learning methods in CRL environments. When there's no task interference, (A-)GEM can outperform Experience Replay

Co{MPS} is a novel meta-policy search algorithm for task-agnostic continual RL

combines replay and an RNN to set a simple baseline for TACRL: shows that the baselines matches and surpasses previously thought upper bounds

Detects changes and explores when and where they happen to recover from non-stationarity.

learns multiple policies and cast policy-retrieval as a multi-arm bandit problem

uses an infinite mixture of Gaussian Processes to learn a task-agnostic policy

A review on continual reinforcement learning

Formulates an online learning procedure that uses SGD to update model parameters, and an EM with a Chinese restaurant process prior to develop and maintain a mixture of models to handle non-stationary task distribution

  • Continual Reinforcement Learning deployed in Real-life using PolicyDistillation and Sim2Real Transfer, (ICML Workshop 2019) by *Kalifou, René Traoré, Caselles-Dupré, Hugo, Lesort, Timothée, Sun, Te, Diaz-Rodriguez, Natalia and Filliat, David * [bib]
  • Experience replay for continual learning , (NeurIPS 2019) by Rolnick, David, Ahuja, Arun, Schwarz, Jonathan, Lillicrap, Timothy and Wayne, Gregory [bib]
  • PathNet: Evolution Channels Gradient Descent in Super Neural Networks , (2017) by Chrisantha Fernando and Dylan Banarse and Charles Blundell and Yori Zwols and David Ha and Andrei A. Rusu and Alexander Pritzel and Daan Wierstra [bib]
  • Incremental robot learning of new objects with fixed update time, (2017) by R. {Camoriano}, G. {Pasquale}, C. {Ciliberto}, L. {Natale}, L. {Rosasco} and G. {Metta} [bib]

Task-Agnostic Lifelong Reinforcement Learning

Co{MPS} is a novel meta-policy search algorithm for task-agnostic continual RL

combines replay and an RNN to set a simple baseline for TACRL: shows that the baselines matches and surpasses previously thought upper bounds

Detects changes and explores when and where they happen to recover from non-stationarity.

learns multiple policies and cast policy-retrieval as a multi-arm bandit problem

uses an infinite mixture of Gaussian Processes to learn a task-agnostic policy

Formulates an online learning procedure that uses SGD to update model parameters, and an EM with a Chinese restaurant process prior to develop and maintain a mixture of models to handle non-stationary task distribution

Continual Generative Modeling

Introduces unsupervised continual learning (no task label and no task boundaries)

Extensive evaluation of CL methods for generative modeling

Biologically-Inspired

bio-inspired method which dynamically restrict and route information in a context-specific manner

a network trained to do CL where select subnetworks are used to learn each task; these subnetworks are chosen a priori

Miscellaneous

Applications

This paper investigates overcoming catastrophic forgetting for malware classification

a healthcare-specific replay-based method to mitigate destructive interference during continual learning

method for compositional continual learning of sequence-to-sequence models

HTM applied to real-world anomaly detection problem

HTM applied to a prediction problem of taxi passenger demand

Thesis

Libraries

A library for real-world continual learning with integrated hyperparameter tuning.

A library that unifies Continual Supervised and Continual Reinforcement Learning research

A library for Continual Supervised Learning

a multi-agent lifelong learning testbed that supports both zero-shot and few-shot settings.

  • River: machine learning for streaming data in Python, (2020) by Jacob Montiel, Max Halford, Saulo Martiello Mastelini and Geoffrey Bolmier, Raphael Sourty, Robin Vaysse and Adil Zouitine, Heitor Murilo Gomes, Jesse Read and Talel Abdessalem and Albert Bifet [bib]

A library for online learning.

  • Continuum, Data Loaders for Continual Learning, (2020) by Douillard, Arthur and Lesort, Timothée [bib]

A library proposing continual learning scenarios and metrics.

A library for Continual Class-Incremental Learning

Workshops

  • Workshop on Continual Learning at ICML 2020 , (2020) by Rahaf Aljundi, Haytham Fayek, Eugene Belilovsky, David Lopez-Paz, Arslan Chaudhry, Marc Pickett, Puneet Dokania, Jonathan Schwarz and Sayna Ebrahimi [bib]
  • 4th Lifelong Machine Learning Workshop at ICML 2020 , (2020) by Shagun Sodhani, Sarath Chandar, Balaraman Ravindran and Doina Precup [bib]
  • CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future Directions, (arXiv 2020) by Lomonaco, Vincenzo, Pellegrini, Lorenzo, Rodriguez, Pau, Caccia, Massimo, She, Qi, Chen, Yu, Jodelet, Quentin, Wang, Ruiping, Mai, Zheda, Vazquez, David and others [bib]

surveys the results of the first CL competition at CVPR