# Selective Data Augmentation for Robust Speech Translation

**Authors:** Rajul Acharya, Ashish Panda, Sunil Kumar Kopparapu  
**arXiv:** [2304.03169](https://arxiv.org/abs/2304.03169)  
**Date:** 2023-03-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.03169v1/

## Key points

- Proposal of an end-to-end architecture for English-Hindi speech translation (ST)
- Use of two imperfect machine translation (MT) services to translate Libri-trans en text into hi text
- Data augmentation strategy of noisy MT data to aid robust ST
- Selective data augmentation strategy proposed that results in better ST performance, as measured by BLEU score, compared to brute force augmentation of MT data
- Absolute improvement of 1.59 BLEU score observed with the proposed approach
- More training data is not necessarily a good strategy for building an e2e ST model, and that selective augmentation of data helps improve performance in the task
- Edit distance-based data selection improves system performance with a gain in BLEU score, even though the augmented data is less compared to using all available data
- Generation of parallel data for low-resource en-Hindi is another contribution of this paper
- Example provided showing how their approach works by selecting training data from two sets based on edit distance between translations obtained via different MT services
- Overall importance highlighted of selective augmentation strategies for improving e2e ST system performance and need for sufficiently large parallel corpora for Indian languages to promote further research in this direction.

## Summary

This paper proposes the use of an end-to-end (e2e) architecture for English-Hindi speech translation (ST), which has gained popularity over cascade systems due to its enhanced performance and ability to retain para and non-linguistic characteristics of speech. The authors use two imperfect machine translation (MT) services to translate Libri-trans en text into hi text, and propose a data augmentation strategy of noisy MT data to aid robust ST. The main contribution of this paper is the proposal of a selective data augmentation strategy that results in better ST performance, as measured by BLEU score, compared to brute force augmentation of MT data. The authors observed an absolute improvement of 1.59 BLEU score with their approach. The authors also show that more training data is not necessarily a good strategy for building an e2e ST model, and that selective augmentation of data helps improve performance in the task. They demonstrate that edit distance-based data selection improves system performance with a gain in BLEU score, even though the augmented data is less compared to using all available data. The generation of parallel data for low-resource en-Hindi is another contribution of this paper. The authors provide an example showing how their approach works by selecting training data from two sets based on edit distance between translations obtained via different MT services. They choose all the training data from one set and augment it with select data from the other set based on ED criteria, resulting in improved BLEU scores. Overall, this paper highlights the importance of selective augmentation strategies for improving e2e ST system performance and underscores the need for sufficiently large parallel corpora for Indian languages to promote further research in this direction.