Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4
Authors: Billal Boumedine
Abstract
We run Kroko, a 65.7M-parameter streaming Zipformer2 transducer for English, on the ESP32-P4, a microcontroller with two 400 MHz RISC-V cores, a 128-bit SIMD extension (PIE) and 32 MB of PSRAM, but no neural accelerator. Without any training, the encoder's 290 matrices are stored with 2 or 4 bits per weight in Hadamard-rotated groups: the width of each matrix comes from its measured sensitivity, and each matrix is first corrected toward the original model's output, then rounded with GPTQ and a search over the group norms. The released 71 MB of int8 weights become a 26.65 MB file. Activations are int16 between the matrix products and int8 into them. A C engine with hand-written PIE kernels processes each 1.28 s chunk of audio in 0.90 s on both cores, beam search included, and its PC build computes the same bits as the chip, so accuracy is measured on whole test sets. LibriSpeech WER is 4.34% on test-clean and 10.26% on test-other, against 3.94% and 9.17% for the released model; on FLEURS and Common Voice the gap grows to 2.3 and 1.1 points. A GPU simulator shows that the integer arithmetic costs nothing and that the whole gap comes from the weights, and an ablation on all of test-clean shows that GPTQ does most of the work. We also report the port's speed history, from 51.3 s to 0.62 s of encoder time per chunk, each design step measured again in one build, an attempt to reach 16 MB that stopped at 19.63 MB, and what did not work. We did not measure energy. The inference engine, the firmware and the 2/4-bit weights are available at https://github.com/BoumedineBillal/esp32-pie-asr.
The papers around it, by date and closeness
Higher is closer to this paper; left is older. Lines join papers close to each other: groups of lines are sub-topics. Select a point to see why it is there. Back to the summary of this paper
Papers around this one
Ranked by how close their title and abstract are to this paper’s. Select some to read them together.
Closest papers
-
O\'ido: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural AcceleratorDaniel Varela Sánchez, Guillermo Martín-Coello Juárez · 2026 · cited 1 time · Not summarized yet
O’ido runs open-vocabulary English speech recognition on a microcontroller without a neural accelerator, using an int8 Conformer on the ESP32-S3.
Abstract
Speech recognition on low-cost microcontrollers is, in practice, limited to keyword spotting or a fixed list of commands. We present O\'ido, an inference engine that runs open-vocabulary English speech recognition entirely on an ESP32-S3 (240 MHz dual-core Xtensa LX7, 512 KB of SRAM, 8 MB of PSRAM, 16 MB of flash, no neural accelerator). O\'ido executes a 13M-parameter Conformer-CTC model with int8 weights, per-frame dynamic int8 activations and int8 attention on the chip's SIMD instructions, with a schedule designed around flash and PSRAM bandwidth. On LibriSpeech test-clean/test-other the on-device arithmetic reaches 3.70%/8.23% word error rate (WER), within 0.12 points of full precision, …
-
Realizing Low-Precision Storage and Sparse Execution in a Folded FPGA Speech EnhancerClément Laroche · 2026 · Not summarized yet
Realizing Low-Precision Storage and Sparse Execution compresses a streaming speech-enhancement model for FPGA deployment, rather than a speech recognizer.
Abstract
Hardware-aware training often optimizes FPGA resources, latency, or bit operations, but folding changes what compression saves: shared arithmetic decouples precision, weight storage, and scheduled work. We derive training objectives from the measured costs of a folded streaming speech-enhancement accelerator. Mixed-precision weights reduce on-chip storage; structured channel sparsity removes entire operations from the engine schedule. We deploy a bit-exact implementation on the Kria KV260. The best sparse model eliminates 47.3% of convolution MACs, cutting frame time by 45% (1.86 to 1.03 ms) and dynamic energy by 60% (83 to 33 $\mu$J/frame) against a storage-priced model measured in the same…
-
How Far Can the Gemma 4 Speech Encoder Go Without Changing Its Weights? Saudi and Gulf Arabic Speech Recognition with a Frozen and an Adapter-Augmented EncoderAmr Abdelkhalik · 2026 · Not summarized yet
How Far Can the Gemma 4 Speech Encoder Go studies Arabic recognition with a frozen pretrained encoder, not low-bit microcontroller inference.
Abstract
We study the 305M-parameter audio encoder shared by Gemma 4 and EmbeddingGemma 2 (bit-identical to that of Gemma 4 E2B except for the final audio-embedding projection) for Saudi and Gulf Arabic speech recognition, without changing any of its pretrained weights; the adapters of our adapted system are additive and kept separate. A frozen system trains only a 22.6M-parameter CTC reader over a learned mix of four intermediate encoder layers and the encoder output; an adapted system adds 3.1M low-rank adapter parameters, a 41.5M-parameter hybrid CTC/attention reader and joint decoding with a sub-word language model. Against Cohere Transcribe Arabic (2.07B parameters), re-run on identical audio, t…
-
Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition ModelsChen Feng, Yicheng Lin, Shaojie Zhuo et al. · 2025 · Sound · preprint, not peer reviewed · cited 12 times · AI summary available
Edge-ASR evaluates post-training low-bit quantization methods for automatic speech recognition models without retraining.
Abstract
Recent advances in Automatic Speech Recognition (ASR) have demonstrated remarkable accuracy and robustness in diverse audio applications, such as live transcription and voice command processing. However, deploying these models on resource-constrained edge devices (e.g., IoT device, wearables) still presents substantial challenges due to strict limits on memory, compute and power. Quantization, particularly Post-Training Quantization (PTQ), offers an effective way to reduce model size and inference cost without retraining. Despite its importance, the performance implications of various advanced quantization methods and bit-width configurations on ASR models remain unclear. In this work, we pr…
-
Real-Time Music Source Separation on a Low-Power Audio DSPJia-Nan Li, Li Liu, K. Malsky et al. · 2026 · Not summarized yet
Real-Time Music Source Separation tests real-time audio-model deployment on a constrained DSP, focusing on source separation rather than recognition.
Abstract
Real-time music source separation is validated on desktop CPUs and GPUs. Does any published system fit the embedded audio hardware it targets? On a commercial audio DSP (2 MB SRAM, 2.07 GMAC/s measured), none does, and the constraints eliminate different models: memory rules out the 16-51 M parameter TasNet/X-UMX family, per-frame compute rules out RT-STT, needing 5.5x the available MAC rate. Parameter count predicts neither: weight reuse spans 1x to 345x. We then build one that fits. Training on continuous rather than block-padded convolution context proves essential: a model scoring 3.93 dB block-wise otherwise collapses to silence within 2 s frame-by-frame. A gated complex FIR deep filter…
-
Paradee: Distilling Kokoro-82M into an 8M-Parameter Single-Voice Text-to-Speech ModelSahil Mahendrakar · 2026 · Not summarized yet
Paradee compresses a text-to-speech model through distillation and quantization, rather than deploying a speech recognizer.
Abstract
We distill Kokoro-82M, a widely used open text-to-speech model with 54 voices, into Paradee, an 8.07M-parameter model that speaks one of them. Paradee keeps Kokoro's architecture with much narrower layers, and each of its two halves is trained separately against the frozen teacher. It has 10x fewer parameters and needs 15x less compute. We first synthesize a corpus with the teacher and keep its durations, pitch, energy and phoneme features. We then train a small text side to predict these values, and a small decoder to turn the teacher's saved values into the teacher's audio, first with spectral losses and then adversarially. Finally, we connect the two halves and quantize the weights to int…
-
Beyond Model Size: Redesigning LiSenNet for embedded speech enhancementClément Laroche, R. Olsson · 2026 · Not summarized yet
Beyond Model Size redesigns a speech-enhancement network for a microcontroller NPU, whereas this paper targets recognition without a neural accelerator.
Abstract
Deploying real-time speech enhancement on resource-constrained devices requires meeting strict latency, memory, and energy constraints. Microcontroller NPUs can accelerate neural inference under these constraints, but only through a restricted set of operators in static, integer-quantized graphs. Recent speech-enhancement networks have reduced parameter counts and MACs to levels nominally suitable for microcontrollers, but their operators and execution patterns often remain incompatible with restricted NPUs. We address this gap by redesigning LiSenNet, a 37k parameter sub-band dual-path model, for the STM32N6570-DK Neural-ART accelerator. We replace its recurrent bottleneck with convolutiona…
-
2-bit Conformer quantization for automatic speech recognitionOleg Rybakov, Phoenix Meadowlark, Shaojin Ding et al. · 2023 · Interspeech · cited 23 times · Not summarized yet
2-bit Conformer quantization studies two-bit weight quantization for ASR, though it evaluates Conformer models rather than microcontroller deployment.
Abstract
Large speech models are rapidly gaining traction in research community. As a result, model compression has become an important topic, so that these models can fit in memory and be served with reduced cost. Practical approaches for compressing automatic speech recognition (ASR) model use int8 or int4 weight quantization. In this study, we propose to develop 2-bit ASR models. We explore the impact of symmetric and asymmetric quantization combined with sub-channel quantization and clipping on both LibriSpeech dataset and large-scale training data. We obtain a lossless 2-bit Conformer model with 32% model size reduction when compared to state of the art 4-bit Conformer model for LibriSpeech. Wit…
Prior work it builds on
-
Zipformer: A faster and better encoder for automatic speech recognitionZengwei Yao, Liyong Guo, Xiao-Yu Yang et al. · 2023 · International Conference on Learning Representations · cited 184 times · Not summarized yet
Zipformer introduces the encoder architecture used by Kroko, the recognizer quantized and deployed in this paper.
Abstract
The Conformer has become the most popular encoder model for automatic speech recognition (ASR). It adds convolution modules to a transformer to learn both local and global dependencies. In this work we describe a faster, more memory-efficient, and better-performing transformer, called Zipformer. Modeling changes include: 1) a U-Net-like encoder structure where middle stacks operate at lower frame rates; 2) reorganized block structure with more modules, within which we re-use attention weights for efficiency; 3) a modified form of LayerNorm called BiasNorm allows us to retain some length information; 4) new activation functions SwooshR and SwooshL work better than Swish. We also propose a new…
-
Moonshine: Speech Recognition for Live Transcription and Voice CommandsNat Jeffries, Evan King, Manjunath Kudlur et al. · 2024 · arXiv.org · preprint, not peer reviewed · cited 31 times · Not summarized yet
Moonshine develops efficient speech-recognition models for live transcription and voice commands, but does not address low-bit microcontroller deployment.
Abstract
This paper introduces Moonshine, a family of speech recognition models optimized for live transcription and voice command processing. Moonshine is based on an encoder-decoder transformer architecture and employs Rotary Position Embedding (RoPE) instead of traditional absolute position embeddings. The model is trained on speech segments of various lengths, but without using zero-padding, leading to greater efficiency for the encoder during inference time. When benchmarked against OpenAI's Whisper tiny-en, Moonshine Tiny demonstrates a 5x reduction in compute requirements for transcribing a 10-second speech segment while incurring no increase in word error rates across standard evaluation data…
-
Librispeech: An ASR corpus based on public domain audio booksVassil Panayotov, Guo-Guo Chen, D. Povey et al. · 2015 · IEEE International Conference on Acoustics, Speech, and Signal Processing · cited 8,710 times · Not summarized yet
Librispeech provides the audiobook speech corpus used to evaluate the recognizer’s word error rate.
-
Robust Speech Recognition via Large-Scale Weak SupervisionA. Radford, Jong Wook Kim, Tao Xu et al. · 2022 · International Conference on Machine Learning · cited 9,370 times · Not summarized yet
Robust Speech Recognition studies large-scale weakly supervised models and their benchmark accuracy, rather than low-bit embedded inference.
Abstract
We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standard benchmarks and are often competitive with prior fully supervised results but in a zero-shot transfer setting without the need for any fine-tuning. When compared to humans, the models approach their accuracy and robustness. We are releasing models and inference code to serve as a foundation for further work on robust speech processing.
-
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained TransformersElias Frantar, Saleh Ashkboos, T. Hoefler et al. · 2022 · arXiv.org · preprint, not peer reviewed · cited 2,863 times · AI summary available
GPTQ presents the post-training weight-quantization method used in this paper to round the encoder’s quantized weights.
Abstract
Generative Pre-trained Transformer models, known as GPT or OPT, set themselves apart through breakthrough performance across complex language modelling tasks, but also by their extremely high computational and storage costs. Specifically, due to their massive size, even inference for large, highly-accurate GPT models may require multiple performant GPUs, which limits the usability of such models. While there is emerging work on relieving this pressure via model compression, the applicability and performance of existing compression techniques is limited by the scale and complexity of GPT models. In this paper, we address this challenge, and propose GPTQ, a new one-shot weight quantization met…
Go further
How are these papers found?
We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.
Updated on 2 November 2026.