# Scenario Aware Speech Recognition: Advancements for Apollo Fearless Steps & CHiME-4 Corpora

**Authors:** Szu-Jui Chen, Wei Xia, John H. L. Hansen  
**arXiv:** [2109.11086](https://arxiv.org/abs/2109.11086)  
**Date:** 2021-09-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2109.11086v1/

## Key points

- The authors present a novel approach to speech recognition using triplet loss for alternative feature representation.
- They propose a general non-semantic speech representation called TRILL, which is trained with self-supervised criteria based on triplet loss for acoustic modeling.
- The approach is applied to two corpora - CHiME-4 and CRSS-UTDallas Fearless Steps Corpus - with emphasis on the 100-hour challenge corpus consisting of five selected NASA Apollo-11 channels.
- An analysis of the extracted embeddings provides a foundation for characterizing training utterances into distinct groups based on acoustic distinguishing properties.
- Triplet-loss based embedding outperforms i-Vector in acoustic modeling, confirming that triplet loss is more effective than speaker features.
- With additional techniques such as pronunciation and silence probability modeling, plus multi-style training, they achieve significant relative word error rate (WER) improvements of +5.42% and +3.18% for the development and evaluation sets of the Fearless Steps Corpus.
- The same technique is tested on the one-channel track of CHiME-4, resulting in an impressive +11.90% relative WER improvement for real test data.
- TRILL-based representations can be used to improve speech recognition performance across different datasets and scenarios.
- This study presents a promising new approach to speech recognition using triplet loss-based feature representations that outperform traditional speaker features like i-Vectors.

## Summary

In this paper, the authors present a novel approach to speech recognition using triplet loss for alternative feature representation. They propose a general non-semantic speech representation called TRILL, which is trained with self-supervised criteria based on triplet loss for acoustic modeling. The approach is applied to two corpora - CHiME-4 and CRSS-UTDallas Fearless Steps Corpus - with emphasis on the 100-hour challenge corpus consisting of five selected NASA Apollo-11 channels. An analysis of the extracted embeddings provides a foundation for characterizing training utterances into distinct groups based on acoustic distinguishing properties. The authors demonstrate that triplet-loss based embedding outperforms i-Vector in acoustic modeling, confirming that triplet loss is more effective than speaker features. With additional techniques such as pronunciation and silence probability modeling, plus multi-style training, they achieve significant relative word error rate (WER) improvements of +5.42% and +3.18% for the development and evaluation sets of the Fearless Steps Corpus. To explore generalization, the same technique is tested on the one-channel track of CHiME-4, resulting in an impressive +11.90% relative WER improvement for real test data. These results suggest that TRILL-based representations can be used to improve speech recognition performance across different datasets and scenarios. Overall, this study presents a promising new approach to speech recognition using triplet loss-based feature representations that outperform traditional speaker features like i-Vectors. The findings have important implications for improving speech recognition accuracy in challenging environments where traditional methods may fall short.