# Learning without Forgetting

**Authors:** Zhizhong Li, Derek Hoiem  
**arXiv:** [1606.09282](https://arxiv.org/abs/1606.09282)  
**Date:** 2016-06-29  
**Summary page:** https://www.summarizepaper.com/en/paper/1606.09282v1/

## Key points

- Authors Zhizhong Li and Derek Hoiem address the challenge of expanding CNN capabilities without access to training data for existing tasks
- Traditional assumption of readily available training data for all tasks becomes impractical as the number of tasks increases
- Introducing new capabilities to the CNN requires training solely on new task data while preserving original abilities
- Li and Hoiem propose the Learning without Forgetting method to address this issue
- The method focuses on training the network using only new task data while retaining original capabilities
- Outperforms feature extraction and fine-tuning adaptation methods
- Shows comparable performance to multitask learning utilizing original task data assumed to be unavailable
- Potential to replace fine-tuning as a standard practice for enhancing performance on new tasks
- Offers a promising solution for efficiently expanding CNN capabilities without compromising existing functionalities

## Summary

In their paper titled "Learning without Forgetting," authors Zhizhong Li and Derek Hoiem address the challenge of continuously expanding the capabilities of a Convolutional Neural Network (CNN) without access to training data for its existing tasks. Traditionally, it is assumed that training data for all tasks is readily available. However, as the number of tasks increases, storing and retraining on such data becomes impractical. This leads to a new problem where adding new capabilities to the CNN requires training solely on new task data while preserving the network's original abilities. To tackle this issue, Li and Hoiem propose the Learning without Forgetting method. This method focuses on training the network using only new task data while ensuring that the original capabilities are retained. Their approach outperforms commonly used techniques such as feature extraction and fine-tuning adaptation methods. Surprisingly, Learning without Forgetting also demonstrates comparable performance to multitask learning that utilizes original task data assumed to be unavailable. The authors' findings suggest that Learning without Forgetting has the potential to replace fine-tuning as a standard practice for enhancing performance on new tasks. By addressing the limitations associated with accessing training data for all tasks in a unified vision system, this method offers a promising solution for efficiently expanding the capabilities of CNNs without compromising their existing functionalities. This research was submitted to ECCV 2016 and falls under categories such as computer vision (cs.CV), machine learning (cs.LG), and statistical machine learning (stat.ML).