# On the Limitations of Continual Learning for Malware Classification

**Authors:** Mohammad Saidur Rahman, Scott E. Coull, Matthew Wright  
**arXiv:** [2208.06568](https://arxiv.org/abs/2208.06568)  
**Date:** 2022-08-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2208.06568v1/

## Key points

- Malware classification is challenging due to the large volume of new samples received daily and the evolution of malware
- Sequential training using CL techniques could reduce training and storage overhead
- 11 CL techniques were investigated in three scenarios using two real-world malware datasets
- Performance was evaluated on binary and multi-class malware family classification tasks
- Continual learning methods underperformed naive Joint replay in almost all settings, reducing accuracy by more than 70 percentage points in some cases
- Selectively replaying 20% of stored data achieved better performance with only 50% of the training time compared to Joint replay
- The study discusses potential reasons for poor performance and hopes to spur further research on developing more effective techniques for malware classification using CL.
- This work highlights the limitations and challenges associated with applying CL techniques to address catastrophic forgetting in malware classification tasks.

## Summary

The classification of malicious software (malware) poses a unique challenge for continual learning (CL) regimes due to the large volume of new samples received on a daily basis and the evolution of malware to exploit new vulnerabilities. Sequential training using CL techniques could provide substantial benefits in reducing training and storage overhead. However, there has been no exploration of CL applied to malware classification tasks. In this study, 11 CL techniques were investigated in three scenarios using two large real-world malware datasets. The performance of the CL methods was evaluated on both binary malware classification (Domain-IL) and multi-class malware family classification (Task-IL and Class-IL) tasks. Surprisingly, the results showed that continual learning methods significantly underperformed naive Joint replay of the training data in almost all settings, reducing accuracy by more than 70 percentage points in some cases. A simple approach of selectively replaying 20% of the stored data achieved better performance with only 50% of the training time compared to Joint replay. The study discusses potential reasons for the unexpectedly poor performance of the CL techniques and hopes that it spurs further research on developing more effective techniques for malware classification using CL. Overall, this work highlights the limitations and challenges associated with applying CL techniques to address catastrophic forgetting in malware classification tasks.