# Signal Collapse in One-Shot Pruning: When Sparse Models Fail to Distinguish Neural Representations

**Authors:** Dhananjay Saikumar, Blesson Varghese  
**arXiv:** [2502.15790](https://arxiv.org/abs/2502.15790)  
**Date:** 2025-02-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.15790v1/

## Key points

- Neural network pruning is crucial for reducing model complexity for deployment on resource-constrained hardware.
- Signal collapse, rather than the removal of critical parameters, is identified as the primary cause of performance decline in pruned networks.
- Traditional one-shot pruning methods often rely on computationally expensive second-order approximations and weight selection strategies.
- Addressing signal collapse is essential for enhancing the accuracy of pruned networks.
- REFLOW is introduced as a novel approach to mitigate signal collapse without updating trainable weights, unveiling high-quality sparse sub-networks within the original parameter space.
- REFLOW achieves state-of-the-art performance results by enabling magnitude pruning, restoring ResNeXt101 accuracy from under 4.1% to 78.9% on ImageNet with only 20% of the weights retained, surpassing existing approaches.
- These findings challenge conventional beliefs about neural network pruning for large-scale models and emphasize considering signal collapse when optimizing pruned networks based on their absolute values.
- This research offers promising insights for improving model efficiency and performance in resource-constrained settings.

## Summary

Neural network pruning plays a crucial role in reducing model complexity for deployment on resource-constrained hardware. This study identifies signal collapse as the primary cause of performance decline in pruned networks, rather than the removal of critical parameters. Traditional one-shot pruning methods often rely on computationally expensive second-order approximations and weight selection strategies. However, this research emphasizes that addressing signal collapse is essential for enhancing the accuracy of pruned networks. Introducing REFLOW as a novel approach, this study demonstrates how mitigating signal collapse without updating trainable weights can unveil high-quality sparse sub-networks within the original parameter space. By enabling magnitude pruning, REFLOW achieves state-of-the-art performance results and restores ResNeXt101 accuracy from under 4.1% to an impressive 78.9% on ImageNet with only 20% of the weights retained, surpassing existing approaches. These findings challenge conventional beliefs about neural network pruning for large-scale models and highlight the importance of considering signal collapse when optimizing pruned networks based on their absolute values. This research sheds light on a new perspective in neural network optimization and offers promising insights for improving model efficiency and performance in resource-constrained settings.