# Convolutional Neural Networks and Language Embeddings for End-to-End Dialect Recognition

**Authors:** Suwon Shon, Ahmed Ali, James Glass  
**arXiv:** [1803.04567](https://arxiv.org/abs/1803.04567)  
**Date:** 2018-03-12  
**Summary page:** https://www.summarizepaper.com/en/paper/1803.04567v2/

## Key points

- The paper addresses the problem of dialect identification (DID) within language identification (LID)
- Proposes an end-to-end DID system using a Siamese neural network to extract language embeddings
- Focuses on the Arabic dialectal speech dataset called MGB-3 and uses acoustic and linguistic features
- Trains the system using three types of acoustic features: MFCCs, FBANK, and spectrogram energies
- Investigates dataset augmentation approach to enhance performance with limited data resources
- Explores similarities and dissimilarities between dialects using the Siamese network for feature dimensionality reduction
- Achieves 73% accuracy with a single feature set and 78% accuracy with a fusion system combining multiple features on the MGB-3 test set
- FBANK features perform slightly better than MFCCs
- Dataset augmentation through speed perturbation enhances system robustness
- Combining Siamese network with language embeddings in a fused system shows promising synergy

## Summary

The paper titled "Convolutional Neural Networks and Language Embeddings for End-to-End Dialect Recognition" addresses the challenging problem of dialect identification (DID) within the broader field of language identification (LID). DID is particularly difficult due to the linguistic similarities between dialects. To tackle this problem, the authors propose an end-to-end DID system that utilizes a Siamese neural network to extract language embeddings. The study focuses on the Arabic dialectal speech dataset called Multi-Genre Broadcast 3 (MGB-3) and employs both acoustic and linguistic features for the task. The end-to-end system is trained using three types of acoustic features: Mel-Frequency Cepstral Coefficients (MFCCs), log Mel-scale Filter Bank energies (FBANK), and spectrogram energies. In order to enhance performance with limited data resources, the researchers investigate a dataset augmentation approach. They also conduct research on linguistic features, specifically exploring similarities and dissimilarities between dialects using the Siamese network which allows them to reduce feature dimensionality while improving DID performance. The experimental results demonstrate that using a single feature set achieves an accuracy of 73%, while a fusion system combining multiple features yields 78% accuracy on the MGB-3 dialect test set consisting of five dialects. The study reveals that FBANK features perform slightly better than MFCCs. Additionally, dataset augmentation through speed perturbation significantly enhances the robustness of the system. Although the Siamese network with language embeddings does not achieve as good results as the end-to-end DID system alone, combining these two approaches in a fused system shows promising synergy. Overall, this paper presents an innovative approach to address the challenging task of dialect identification by proposing an end-to-end system and leveraging both acoustic and linguistic features. The findings contribute to improving performance in identifying Arabic dialects, particularly in scenarios with limited data resources.