# Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Models

**Authors:** Chen Feng, Yicheng Lin, Shaojie Zhuo, Chenzheng Su, Ramchalam Kinattinkara Ramakrishnan, Zhaocong Yuan, Xiaopeng Zhang  
**arXiv:** [2507.07877](https://arxiv.org/abs/2507.07877)  
**Date:** 2025-07-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2507.07877v2/

## Key points

- Recent advancements in ASR have shown high accuracy and reliability in applications like live transcription and voice command processing.
- Deploying ASR models on resource-constrained edge devices poses challenges due to limitations in memory, computing power, and energy consumption.
- Post-training quantization (PTQ) is highlighted as a solution for reducing model size and inference costs without retraining.
- The performance implications of different advanced quantization methods and bit-width configurations on ASR models are not fully understood.
- Researchers conducted a thorough evaluation by benchmarking eight PTQ techniques on Whisper and Moonshine edge-ASR model families across seven diverse datasets.
- Analysis focused on the impact of quantization on both weights and activations within the models to understand efficiency versus accuracy trade-offs.
- Results showed that even 3-bit quantization can be successful with advanced PTQ techniques on high-capacity models for low-power edge devices.
- The study contributes significantly to enhancing the efficiency and effectiveness of speech recognition systems in real-world applications.

## Summary

In their paper titled "Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Models," authors Chen Feng, Yicheng Lin, Shaojie Zhuo, Chenzheng Su, Ramchalam Kinattinkara Ramakrishnan, Zhaocong Yuan, and Xiaopeng Zhang delve into the realm of  technology. They highlight recent advancements in ASR that have showcased impressive accuracy and reliability across various audio applications like live transcription and voice command processing. Despite these achievements, deploying ASR models on resource-constrained edge devices such as IoT devices and wearables remains challenging due to limitations in memory, computing power, and energy consumption. The authors emphasize the significance of  as a viable solution for reducing model size and inference costs without the need for retraining. However, the performance implications of different advanced quantization methods and bit-width configurations on ASR models are not fully understood. To address this gap in knowledge, the researchers conduct a thorough evaluation by benchmarking eight state-of-the-art PTQ techniques on two prominent edge-ASR model families known as Whisper and Moonshine. Their study involves a systematic assessment of model performances in terms of accuracy, memory I/O operations, and bit operations across seven diverse datasets sourced from the open ASR leaderboard. By analyzing the impact of quantization on both weights and activations within the models, the authors aim to shed light on the trade-offs between efficiency and accuracy. Leveraging an extension of the LLM compression toolkit, along with a unified calibration and evaluation data pipeline equipped with detailed analysis tools. The results obtained from their research showcase how even 3-bit quantization can yield successful outcomes on high-capacity models when coupled with advanced PTQ techniques. These findings offer valuable insights for optimizing ASR models specifically tailored for low-power always-on edge devices. By providing a comprehensive exploration of cutting-edge quantization methods applied to ASR technology, this study contributes significantly to enhancing the efficiency and effectiveness of speech recognition systems in real-world applications.