Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Proche Même question Cité par cet article Utilisé en profondeur par cet article Pas pertinent

   O\'ido: Open-Vocabulary Speech Recognition on a $5 Microcontroller without a Neural Accelerator

   Daniel Varela Sánchez, Guillermo Martín-Coello Juárez · 2026 · cité 1 fois · Pas encore résumé

   O’ido exécute une reconnaissance vocale anglaise à vocabulaire ouvert sur microcontrôleur sans accélérateur, comme l’étude, mais avec un modèle Conformer-CTC int8 sur ESP32-S3.

   Résumé scientifique

   Speech recognition on low-cost microcontrollers is, in practice, limited to keyword spotting or a fixed list of commands. We present O\'ido, an inference engine that runs open-vocabulary English speech recognition entirely on an ESP32-S3 (240 MHz dual-core Xtensa LX7, 512 KB of SRAM, 8 MB of PSRAM, 16 MB of flash, no neural accelerator). O\'ido executes a 13M-parameter Conformer-CTC model with int8 weights, per-frame dynamic int8 activations and int8 attention on the chip's SIMD instructions, with a schedule designed around flash and PSRAM bandwidth. On LibriSpeech test-clean/test-other the on-device arithmetic reaches 3.70%/8.23% word error rate (WER), within 0.12 points of full precision, …

2. Proche Contexte plus large Pas pertinent

   Realizing Low-Precision Storage and Sparse Execution in a Folded FPGA Speech Enhancer

   Clément Laroche · 2026 · Pas encore résumé

   Cette étude réduit stockage et calcul d’un système FPGA de restauration de parole par précision mixte et sparsité structurée, plutôt que de reconnaître la parole sur microcontrôleur.

   Résumé scientifique

   Hardware-aware training often optimizes FPGA resources, latency, or bit operations, but folding changes what compression saves: shared arithmetic decouples precision, weight storage, and scheduled work. We derive training objectives from the measured costs of a folded streaming speech-enhancement accelerator. Mixed-precision weights reduce on-chip storage; structured channel sparsity removes entire operations from the engine schedule. We deploy a bit-exact implementation on the Kria KV260. The best sparse model eliminates 47.3% of convolution MACs, cutting frame time by 45% (1.86 to 1.03 ms) and dynamic energy by 60% (83 to 33 $\mu$J/frame) against a storage-priced model measured in the same…

3. Apparenté Contexte plus large Pas pertinent

   How Far Can the Gemma 4 Speech Encoder Go Without Changing Its Weights? Saudi and Gulf Arabic Speech Recognition with a Frozen and an Adapter-Augmented Encoder

   Amr Abdelkhalik · 2026 · Pas encore résumé

   Cette étude adapte un encodeur audio Gemma gelé à la reconnaissance de l’arabe du Golfe, tandis que l’étude déploie un transducteur anglais quantifié sur microcontrôleur.

   Résumé scientifique

   We study the 305M-parameter audio encoder shared by Gemma 4 and EmbeddingGemma 2 (bit-identical to that of Gemma 4 E2B except for the final audio-embedding projection) for Saudi and Gulf Arabic speech recognition, without changing any of its pretrained weights; the adapters of our adapted system are additive and kept separate. A frozen system trains only a 22.6M-parameter CTC reader over a learned mix of four intermediate encoder layers and the encoder output; an adapted system adds 3.1M low-rank adapter parameters, a 41.5M-parameter hybrid CTC/attention reader and joint decoding with a sub-word language model. Against Cohere Transcribe Arabic (2.07B parameters), re-run on identical audio, t…

4. Apparenté Même question Cité par cet article Pas pertinent

   Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Models

   Chen Feng, Yicheng Lin, Shaojie Zhuo et al. · 2025 · Sound · preprint, non relu par les pairs · cité 12 fois · Résumé IA disponible

   Edge-ASR examine la quantification après entraînement et les compromis de précision des modèles de reconnaissance vocale, comme l’étude, qui déploie un modèle à poids 2 et 4 bits.

   Résumé scientifique

   Recent advances in Automatic Speech Recognition (ASR) have demonstrated remarkable accuracy and robustness in diverse audio applications, such as live transcription and voice command processing. However, deploying these models on resource-constrained edge devices (e.g., IoT device, wearables) still presents substantial challenges due to strict limits on memory, compute and power. Quantization, particularly Post-Training Quantization (PTQ), offers an effective way to reduce model size and inference cost without retraining. Despite its importance, the performance implications of various advanced quantization methods and bit-width configurations on ASR models remain unclear. In this work, we pr…

5. Apparenté Contexte plus large Pas pertinent

   Real-Time Music Source Separation on a Low-Power Audio DSP

   Jia-Nan Li, Li Liu, K. Malsky et al. · 2026 · Pas encore résumé

   Cette étude conçoit une séparation de sources musicales pour DSP à faible puissance, en tenant compte des contraintes mémoire et calcul, plutôt qu’une reconnaissance vocale embarquée.

   Résumé scientifique

   Real-time music source separation is validated on desktop CPUs and GPUs. Does any published system fit the embedded audio hardware it targets? On a commercial audio DSP (2 MB SRAM, 2.07 GMAC/s measured), none does, and the constraints eliminate different models: memory rules out the 16-51 M parameter TasNet/X-UMX family, per-frame compute rules out RT-STT, needing 5.5x the available MAC rate. Parameter count predicts neither: weight reuse spans 1x to 345x. We then build one that fits. Training on continuous rather than block-padded convolution context proves essential: a model scoring 3.93 dB block-wise otherwise collapses to silence within 2 s frame-by-frame. A gated complex FIR deep filter…

6. Apparenté Contexte plus large Pas pertinent

   Paradee: Distilling Kokoro-82M into an 8M-Parameter Single-Voice Text-to-Speech Model

   Sahil Mahendrakar · 2026 · Pas encore résumé

   Paradee compresse un modèle de synthèse vocale par distillation et quantification, alors que l’étude compresse un modèle de reconnaissance vocale sans réentraînement.

   Résumé scientifique

   We distill Kokoro-82M, a widely used open text-to-speech model with 54 voices, into Paradee, an 8.07M-parameter model that speaks one of them. Paradee keeps Kokoro's architecture with much narrower layers, and each of its two halves is trained separately against the frozen teacher. It has 10x fewer parameters and needs 15x less compute. We first synthesize a corpus with the teacher and keep its durations, pitch, energy and phoneme features. We then train a small text side to predict these values, and a small decoder to turn the teacher's saved values into the teacher's audio, first with spectral losses and then adversarially. Finally, we connect the two halves and quantize the weights to int…

7. Apparenté Contexte plus large Pas pertinent

   Beyond Model Size: Redesigning LiSenNet for embedded speech enhancement

   Clément Laroche, R. Olsson · 2026 · Pas encore résumé

   Cette étude redessine un réseau d’amélioration de la parole pour un accélérateur NPU à opérateurs restreints, contrairement au moteur de reconnaissance sans accélérateur de l’étude.

   Résumé scientifique

   Deploying real-time speech enhancement on resource-constrained devices requires meeting strict latency, memory, and energy constraints. Microcontroller NPUs can accelerate neural inference under these constraints, but only through a restricted set of operators in static, integer-quantized graphs. Recent speech-enhancement networks have reduced parameter counts and MACs to levels nominally suitable for microcontrollers, but their operators and execution patterns often remain incompatible with restricted NPUs. We address this gap by redesigning LiSenNet, a 37k parameter sub-band dual-path model, for the STM32N6570-DK Neural-ART accelerator. We replace its recurrent bottleneck with convolutiona…

8. Apparenté Autre approche du problème Cité par cet article Pas pertinent

   2-bit Conformer quantization for automatic speech recognition

   Oleg Rybakov, Phoenix Meadowlark, Shaojin Ding et al. · 2023 · Interspeech · cité 23 fois · Pas encore résumé

   Cette étude quantifie un Conformer de reconnaissance vocale à 2 bits et évalue la précision sur LibriSpeech, tandis que l’étude applique une quantification post-entraînement mixte à Kroko.

   Résumé scientifique

   Large speech models are rapidly gaining traction in research community. As a result, model compression has become an important topic, so that these models can fit in memory and be served with reduced cost. Practical approaches for compressing automatic speech recognition (ASR) model use int8 or int4 weight quantization. In this study, we propose to develop 2-bit ASR models. We explore the impact of symmetric and asymmetric quantization combined with sub-channel quantization and clipping on both LibriSpeech dataset and large-scale training data. We obtain a lossless 2-bit Conformer model with 32% model size reduction when compared to state of the art 4-bit Conformer model for LibriSpeech. Wit…

Les travaux antérieurs sur lesquels il s’appuie

1. Apparenté Base de cet article Pas pertinent

   Zipformer: A faster and better encoder for automatic speech recognition

   Zengwei Yao, Liyong Guo, Xiao-Yu Yang et al. · 2023 · International Conference on Learning Representations · cité 184 fois · Pas encore résumé

   Zipformer présente l’encodeur utilisé dans le transducteur Kroko que l’étude déploie après quantification sur microcontrôleur.

   Résumé scientifique

   The Conformer has become the most popular encoder model for automatic speech recognition (ASR). It adds convolution modules to a transformer to learn both local and global dependencies. In this work we describe a faster, more memory-efficient, and better-performing transformer, called Zipformer. Modeling changes include: 1) a U-Net-like encoder structure where middle stacks operate at lower frame rates; 2) reorganized block structure with more modules, within which we re-use attention weights for efficiency; 3) a modified form of LayerNorm called BiasNorm allows us to retain some length information; 4) new activation functions SwooshR and SwooshL work better than Swish. We also propose a new…

2. Apparenté Contexte plus large Pas pertinent

   Moonshine: Speech Recognition for Live Transcription and Voice Commands

   Nat Jeffries, Evan King, Manjunath Kudlur et al. · 2024 · arXiv.org · preprint, non relu par les pairs · cité 31 fois · Pas encore résumé

   Moonshine optimise des modèles de reconnaissance pour la transcription en direct et les commandes vocales, tandis que l’étude vise l’exécution continue sur microcontrôleur.

   Résumé scientifique

   This paper introduces Moonshine, a family of speech recognition models optimized for live transcription and voice command processing. Moonshine is based on an encoder-decoder transformer architecture and employs Rotary Position Embedding (RoPE) instead of traditional absolute position embeddings. The model is trained on speech segments of various lengths, but without using zero-padding, leading to greater efficiency for the encoder during inference time. When benchmarked against OpenAI's Whisper tiny-en, Moonshine Tiny demonstrates a 5x reduction in compute requirements for transcribing a 10-second speech segment while incurring no increase in word error rates across standard evaluation data…

3. Apparenté* Données, outil ou méthode en commun Utilisé en profondeur par cet article Pas pertinent

   Librispeech: An ASR corpus based on public domain audio books

   Vassil Panayotov, Guo-Guo Chen, D. Povey et al. · 2015 · IEEE International Conference on Acoustics, Speech, and Signal Processing · cité 8 710 fois · Pas encore résumé

   LibriSpeech fournit le corpus sur lequel l’étude évalue le taux d’erreur de son modèle quantifié.

4. Apparenté Contexte plus large Pas pertinent

   Robust Speech Recognition via Large-Scale Weak Supervision

   A. Radford, Jong Wook Kim, Tao Xu et al. · 2022 · International Conference on Machine Learning · cité 9 370 fois · Pas encore résumé

   Cette étude entraîne des modèles de reconnaissance robustes sur de vastes transcriptions audio multilingues, tandis que l’étude porte sur la compression et l’exécution embarquée.

   Résumé scientifique

   We study the capabilities of speech processing systems trained simply to predict large amounts of transcripts of audio on the internet. When scaled to 680,000 hours of multilingual and multitask supervision, the resulting models generalize well to standard benchmarks and are often competitive with prior fully supervised results but in a zero-shot transfer setting without the need for any fine-tuning. When compared to humans, the models approach their accuracy and robustness. We are releasing models and inference code to serve as a foundation for further work on robust speech processing.

5. Apparenté Base de cet article Utilisé en profondeur par cet article Pas pertinent

   GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers

   Elias Frantar, Saleh Ashkboos, T. Hoefler et al. · 2022 · arXiv.org · preprint, non relu par les pairs · cité 2 863 fois · Résumé IA disponible

   GPTQ fournit la méthode d’arrondi après quantification que l’étude applique aux poids du chemin encodeur de Kroko.

   Résumé scientifique

   Generative Pre-trained Transformer models, known as GPT or OPT, set themselves apart through breakthrough performance across complex language modelling tasks, but also by their extremely high computational and storage costs. Specifically, due to their massive size, even inference for large, highly-accurate GPT models may require multiple performant GPUs, which limits the usability of such models. While there is emerging work on relieving this pressure via model compression, the applicability and performance of existing compression techniques is limited by the scale and complexity of GPT models. In this paper, we address this challenge, and propose GPTQ, a new one-shot weight quantization met…