# Neural Network Quantization for Efficient Inference: A Survey

**Authors:** Olivia Weng  
**arXiv:** [2112.06126](https://arxiv.org/abs/2112.06126)  
**Date:** 2021-12-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2112.06126v2/

## Key points

- Challenges of deploying neural networks in resource-constrained devices
- Neural network quantization as a solution to reduce size and complexity
- Overview of various quantization techniques: weight quantization, activation quantization, ternary or binary weight representations, low-rank factorizations, and knowledge distillation
- Explanation of each technique's advantages and limitations
- Discussion of evaluation metrics for comparing quantization methods
- Proposed future research directions: more efficient algorithms for deep neural network quantization, combining multiple techniques for better results
- Valuable insights into state-of-the-art techniques in neural network quantization
- Contribution towards enabling efficient inference in real-world applications such as edge computing.

## Summary

The paper titled "Neural Network Quantization for Efficient Inference: A Survey" by Olivia Weng explores the challenges of deploying neural networks in resource-constrained devices and presents a comprehensive survey of neural network quantization techniques. As neural networks have become more powerful, there is a growing desire to deploy them in real-world applications. However, the depth and complexity of these networks make it difficult to run them efficiently on devices with limited resources. Neural network quantization has emerged as a solution to reduce the size and complexity of neural networks by reducing the precision of network parameters. By using smaller and simpler networks, it becomes possible to meet the constraints of target hardware and enable efficient inference. The paper provides an overview of various quantization techniques that have been developed over the last decade. The survey covers different approaches to quantizing neural networks, including weight quantization, activation quantization, ternary or binary weight representations, low-rank factorizations, and knowledge distillation. Each technique is explained in detail, highlighting its advantages and limitations. The paper also discusses evaluation metrics used for comparing different quantization methods. Based on this survey and comparison of these techniques, the authors propose future research directions in the field of neural network quantization. They emphasize the need for developing more efficient algorithms for quantizing deep neural networks while maintaining high accuracy. Additionally, they suggest exploring novel ways to combine multiple quantization techniques to achieve even better results. Overall, this paper provides valuable insights into state-of-the-art techniques in neural network quantization and offers guidance for future research in this area. By addressing the challenges associated with deploying complex neural networks on resource-constrained devices, this work contributes towards enabling efficient inference in real-world applications such as edge computing.