# Fast Implementation of 4-bit Convolutional Neural Networks for Mobile Devices

**Authors:** Anton Trusov, Elena Limonova, Dmitry Slugin, Dmitry Nikolaev, Vladimir V. Arlazarov  
**arXiv:** [2009.06488](https://arxiv.org/abs/2009.06488)  
**Date:** 2020-09-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2009.06488v2/

## Key points

- Quantized low-precision neural networks are popular for high performance with less computational resources
- Advantages for FPGA and ASIC devices, but general-purpose processors may not efficiently perform low-bit integer computations
- Typically, 8-bit quantized network used for mobile central processors, but in some cases fewer bits can be used
- Challenge is implementing this efficiently
- Authors introduce efficient implementation of 4-bit matrix multiplication specifically designed for quantized neural networks
- Time measurements on mobile ARM processor showed 2.9 times speedup compared to standard floating-point multiplication and 1.5 times faster than an 8-bit quantized network
- Presented a 4-bit quantized neural network designed for OCR recognition on the MIDV-500 dataset achieving an accuracy of 95% with overall inference speedup of 48%
- Using fewer bits in neural networks can reduce inference time without significantly sacrificing accuracy
- Efficient implementation of 4-bit matrix multiplication provides promising results that could benefit mobile devices with limited computational resources

## Summary

The use of quantized low-precision neural networks has become increasingly popular due to their ability to provide high performance while requiring less computational resources for inference. This is particularly important for real-time and embedded recognition systems. However, while these networks are advantageous for FPGA and ASIC devices, general-purpose processor architectures may not always be able to efficiently perform low-bit integer computations. Typically, the most frequently used low-precision neural network model for mobile central processors is an 8-bit quantized network. However, in certain cases, it may be possible to use fewer bits for weights and activations. The main challenge then becomes implementing this efficiently. In this study, the authors introduce an efficient implementation of 4-bit matrix multiplication specifically designed for quantized neural networks. Time measurements were performed on a mobile ARM processor which showed a 2.9 times speedup compared to standard floating-point multiplication and a 1.5 times faster speed than an 8-bit quantized network. To demonstrate the effectiveness of their approach, the authors also present a 4-bit quantized neural network designed for OCR recognition on the MIDV-500 dataset. This network achieved an accuracy of 95%, with an overall inference speedup of 48%. In comparison, an 8-bit quantized network achieved slightly higher accuracy at 95.4% but only had a speedup of 39%. Overall, these results suggest that using fewer bits in neural networks can be effective in reducing inference time without significantly sacrificing accuracy. The authors' efficient implementation of 4-bit matrix multiplication provides promising results that could benefit mobile devices with limited computational resources by providing improved performance and reduced power consumption compared to conventional methods.