# Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks

**Authors:** Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, Yi Yang  
**arXiv:** [1808.06866](https://arxiv.org/abs/1808.06866)  
**Date:** 2018-08-21  
**Summary page:** https://www.summarizepaper.com/en/paper/1808.06866v1/

## Key points

- Soft Filter Pruning (SFP) accelerates the inference process of deep CNNs
- SFP allows pruned filters to be updated during model training after pruning
- SFP enables larger model capacity and provides a larger optimization space compared to previous methods
- SFP exhibits less dependence on pre-trained models and allows for simultaneous training and pruning from scratch
- SFP outperforms previous filter pruning methods when trained from scratch across various advanced CNN architectures
- On ILSCRC-2012 dataset, SFP achieves over 42% reduction in FLOPs on ResNet-101 with a 0.2% improvement in top-5 accuracy
- Code implementation of SFP is publicly available on GitHub at https://github.com/he-y/soft-filter-pruning

## Summary

This paper introduces Soft Filter Pruning (SFP) for accelerating the inference process of deep Convolutional Neural Networks (CNNs). The proposed SFP allows pruned filters to be updated during model training after pruning. Compared to previous methods, SFP offers two key advantages. Firstly, it enables larger model capacity by updating previously pruned filters and provides a larger optimization space than fixing the filters to zero. Consequently, the network trained using SFP has greater capacity to learn from the training data. Secondly, SFP exhibits less dependence on pre-trained models as its large capacity allows for simultaneous training and pruning of the model from scratch. Previous filter pruning methods require a pre-trained model as a basis for performance guarantee. Empirical results demonstrate that SFP outperforms previous filter pruning methods when trained from scratch and is effective across various advanced CNN architectures. Notably, on ILSCRC-2012 dataset, SFP achieves over 42% reduction in FLOPs (floating point operations) on ResNet-101 with even a 0.2% improvement in top-5 accuracy. The code implementation of SFP is publicly available on GitHub at https://github.com/he-y/soft-filter-pruning.