AraSpot: Arabic Spoken Command Spotting

Mahmoud Salhab; Haidar Harmanani

AI summary · arXiv: 2303.16621v1 · cs.CL

AraSpot: Arabic Spoken Command Spotting

Mahmoud Salhab, Haidar Harmanani

Original paper Submitted to arXiv on 29 Mar. 2023 A preprint License: CC BY 4.0

How to read this summary? Illustrated guide

The essentials

AraSpot is an Arabic spoken command spotting system that identifies keywords in an audio stream
The system uses the ConformerGRU model architecture and different online data augmentation techniques to enrich and enlarge the dataset
A text-to-speech model is trained for synthetic data generation to overcome data scarcity for keyword spotting in Arabic
The authors use the Arabic Speech Commands (ASC) dataset, achieving a SOTA accuracy of 99.59%
The paper highlights the scarcity of publicly available KWS datasets in languages other than English and Asian languages, despite Arabic being the fourth most used language on the internet
Previous approaches to KWS are discussed, including transfer learning from ASR systems and multi-target setups
Building a SOTA KWS model requires more than 4000 utterances per command but methods such as synthesized speech can be used to expand training data

Key points written by AI: they may contain errors, check them against the paper.

AI-generated keywords: AraSpot

Read further

Comprehensive Summary
Layman's Summary
Blog article
Original abstract

This paper presents AraSpot, a state-of-the-art Arabic spoken command spotting system that identifies keywords in an audio stream. The task is challenging as it requires high accuracy while running efficiently on low power devices. To address this challenge, the authors introduce the ConformerGRU model architecture and use different online data augmentation techniques to enrich and enlarge the dataset. They also train a text-to-speech model for synthetic data generation to overcome data scarcity for keyword spotting in Arabic. The authors use the Arabic Speech Commands (ASC) dataset, which includes 40 keywords recorded by 30 participants with each keyword recorded ten times. They achieve a SOTA accuracy of 99.59%, outperforming previous approaches such as deep CNN models and LSTM models. The paper highlights the scarcity of publicly available KWS datasets in languages other than English and Asian languages, despite Arabic being the fourth most used language on the internet. The authors' work contributes to filling this gap by presenting AraSpot for Arabic command spotting. The paper also discusses previous approaches to KWS, including transfer learning from ASR systems and multi-target setups. It emphasizes that building a SOTA KWS model requires more than 4000 utterances per command but notes that methods such as synthesized speech can be used to expand training data. Overall, this paper presents a comprehensive approach to developing an accurate and efficient Arabic spoken command spotting system using innovative techniques such as ConformerGRU architecture and synthesized speech data augmentation.

AraSpot: A State-of-the-Art Arabic Spoken Command Spotting System

In this research paper, the authors present AraSpot, a state-of-the-art Arabic spoken command spotting system that identifies keywords in an audio stream. The task is challenging as it requires high accuracy while running efficiently on low power devices. To address this challenge, the authors introduce the ConformerGRU model architecture and use different online data augmentation techniques to enrich and enlarge the dataset. They also train a text-to-speech model for synthetic data generation to overcome data scarcity for keyword spotting in Arabic.

Background

Keyword spotting (KWS) is an important technology used in many applications such as voice assistants and home automation systems. It has been studied extensively for English language but there are few publicly available datasets for other languages such as Arabic which is the fourth most used language on the internet. This makes building a SOTA KWS model difficult due to lack of training data. Previous approaches have included transfer learning from ASR systems and multi-target setups but they still require more than 4000 utterances per command to achieve good performance.

Methodology

The authors use the Arabic Speech Commands (ASC) dataset which includes 40 keywords recorded by 30 participants with each keyword recorded ten times. They propose a novel ConformerGRU architecture which combines convolutional layers with gated recurrent units (GRUs). The proposed architecture achieves 99.59% accuracy on ASC dataset outperforming previous approaches such as deep CNN models and LSTM models. To further improve performance, they apply various online data augmentation techniques including speed perturbation, pitch shifting, time warping etc., resulting in improved accuracy of 99.69%. In addition, they generate synthetic speech using Tacotron 2 text to speech synthesis model trained on LibriSpeech corpus resulting in improved accuracy of 99.78%.

Conclusion

This paper presents AraSpot for accurate and efficient keyword spotting in Arabic audio streams using innovative techniques such as ConformerGRU architecture and synthesized speech data augmentation . The authors demonstrate that their approach can achieve SOTA results with only 4000 utterances per command compared to previous methods requiring tens of thousands of utterances per command . Their work contributes towards filling the gap caused by lack of publicly available KWS datasets for languages other than English or Asian languages .

Created on 13 Apr. 2023

Was this synthesis useful?

Score: 0

Ask questions about this paper to our AI assistant

How to chat with this paper? Illustrated guide

Compare with the PDF

You can also chat with multiple papers at once here.

AI assistant instructions?

Tools for this paper

Explain it differently 3 free rewrites left today

Save to a collection Group papers in collections to compare them, chat with them and export them. Create a free account Log in

Export BibTeX · RIS (Zotero, Mendeley, EndNote) · Markdown (Notion, Obsidian)

Advanced tools Premium

For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium

Zotero

Citation alerts Citation alerts guide

Who works on it? The people and institutions who publish on its topic

Journal club Premium+ Slides to present it

Versions What changed between versions

Talk about it in public Premium+ Press kit: release, questions, hype check Press kit guide

Papers around this one

See them on a map

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

Related
AraGPT2: Pre-Trained Transformer for Arabic Language Generation
Wissam Antoun, Fady Baly, Hazem Hajj · 2020 · Computation and Language · Not summarized yet

Abstract
Recently, pre-trained transformer-based architectures have proven to be very efficient at language modeling and understanding, given that they are trained on a large enough corpus. Applications in language generation for Arabic are still lagging in comparison to other NLP advances primarily due to the lack of advanced Arabic language generation models. In this paper, we develop the first advanced Arabic language generation model, AraGPT2, trained from scratch on a large Arabic corpus of internet text and news articles. Our largest model, AraGPT2-mega, has 1.46 billion parameters, which makes it the largest Arabic language model available. The Mega model was evaluated and showed success on di…
Related
Automatic Arabic Dialect Identification Systems for Written Texts: A Survey
Maha J. Althobaiti · 2020 · Computation and Language · AI summary available

Abstract
Arabic dialect identification is a specific task of natural language processing, aiming to automatically predict the Arabic dialect of a given text. Arabic dialect identification is the first step in various natural language processing applications such as machine translation, multilingual text-to-speech synthesis, and cross-language text generation. Therefore, in the last decade, interest has increased in addressing the problem of Arabic dialect identification. In this paper, we present a comprehensive survey of Arabic dialect identification research in written texts. We first define the problem and its challenges. Then, the survey extensively discusses in a critical manner many aspects rel…
Related
ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic
Fajri Koto, Haonan Li, Sara Shatnawi et al. · 2024 · Computation and Language · Not summarized yet

Abstract
The focus of language model evaluation has transitioned towards reasoning and knowledge-intensive tasks, driven by advancements in pretraining large models. While state-of-the-art models are partially trained on large Arabic texts, evaluating their performance in Arabic remains challenging due to the limited availability of relevant datasets. To bridge this gap, we present ArabicMMLU, the first multi-task language understanding benchmark for Arabic language, sourced from school exams across diverse educational levels in different countries spanning North Africa, the Levant, and the Gulf regions. Our data comprises 40 tasks and 14,575 multiple-choice questions in Modern Standard Arabic (MSA),…
Related
Proficiency assessment of L2 spoken English using wav2vec 2.0
Stefano Bannò, Marco Matassoni · 2022 · Computation and Language · AI summary available

Abstract
The increasing demand for learning English as a second language has led to a growing interest in methods for automatically assessing spoken language proficiency. Most approaches use hand-crafted features, but their efficacy relies on their particular underlying assumptions and they risk discarding potentially salient information about proficiency. Other approaches rely on transcriptions produced by ASR systems which may not provide a faithful rendition of a learner's utterance in specific scenarios (e.g., non-native children's spontaneous speech). Furthermore, transcriptions do not yield any information about relevant aspects such as intonation, rhythm or prosody. In this paper, we investiga…
Related
GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP
Md Tawkat Islam Khondaker, Abdul Waheed, El Moatez Billah Nagoudi et al. · 2023 · Computation and Language · AI summary available

Abstract
ChatGPT's emergence heralds a transformative phase in NLP, particularly demonstrated through its excellent performance on many English benchmarks. However, the model's efficacy across diverse linguistic contexts remains largely uncharted territory. This work aims to bridge this knowledge gap, with a primary focus on assessing ChatGPT's capabilities on Arabic languages and dialectal varieties. Our comprehensive study conducts a large-scale automated and human evaluation of ChatGPT, encompassing 44 distinct language understanding and generation tasks on over 60 different datasets. To our knowledge, this marks the first extensive performance analysis of ChatGPT's deployment in Arabic NLP. Our f…
Related
Fine-Tashkeel: Finetuning Byte-Level Models for Accurate Arabic Text Diacritization
Bashar Al-Rfooh, Gheith Abandah, Rami Al-Rfou · 2023 · Computation and Language · AI summary available

Abstract
Most of previous work on learning diacritization of the Arabic language relied on training models from scratch. In this paper, we investigate how to leverage pre-trained language models to learn diacritization. We finetune token-free pre-trained multilingual models (ByT5) to learn to predict and insert missing diacritics in Arabic text, a complex task that requires understanding the sentence semantics and the morphological structure of the tokens. We show that we can achieve state-of-the-art on the diacritization task with minimal amount of training and no feature engineering, reducing WER by 40%. We release our finetuned models for the greater benefit of the researchers in the community.
Related
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
Yu Zhang, Wei Han, James Qin et al. · 2023 · Computation and Language · AI summary available

Abstract
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages. This is achieved by pre-training the encoder of the model on a large unlabeled multilingual dataset of 12 million (M) hours spanning over 300 languages, and fine-tuning on a smaller labeled dataset. We use multilingual pre-training with random-projection quantization and speech-text modality matching to achieve state-of-the-art performance on downstream multilingual ASR and speech-to-text translation tasks. We also demonstrate that despite using a labeled training set 1/7-th the size of that used for the Whisper model, our model exhibits comparable or be…
Related
Filler Word Detection and Classification: A Dataset and Benchmark
Ge Zhu, Juan-Pablo Caceres, Justin Salamon · 2022 · Computation and Language · AI summary available

Abstract
Filler words such as `uh' or `um' are sounds or words people use to signal they are pausing to think. Finding and removing filler words from recordings is a common and tedious task in media editing. Automatically detecting and classifying filler words could greatly aid in this task, but few studies have been published on this problem. A key reason is the absence of a dataset with annotated filler words for training and evaluation. In this work, we present a novel speech dataset, PodcastFillers, with 35K annotated filler words and 50K annotations of other sounds that commonly occur in podcasts such as breaths, laughter, and word repetitions. We propose a pipeline that leverages VAD and ASR to…

Go further

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

For now only among the papers summarized on this site: search again to add the prior work, the follow-ups and Semantic Scholar’s recommendations.

The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.