# FeaRLESS: Feature Refinement Loss for Ensembling Self-Supervised Learning Features in Robust End-to-end Speech Recognition

**Authors:** Szu-Jui Chen, Jiamin Xie, John H. L. Hansen  
**arXiv:** [2206.15056](https://arxiv.org/abs/2206.15056)  
**Date:** 2022-06-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2206.15056v1/

## Key points

- Self-supervised learning representations (SSLR) are effective in generating robust features for various downstream tasks
- SSLRs have shown promising results in automatic speech recognition (ASR) benchmark corpora
- Previous research has only focused on the performance of solitary SSLRs as input features for ASR models
- A team of researchers proposed investigating the effectiveness of diverse SSLR combinations using various fusion methods within end-to-end (E2E) ASR models
- The team proposed a feature refinement loss for decorrelation to efficiently combine the set of input features and improve their performance in ASR models
- The proposed 'FeaRLESS learning features' outperformed systems without the proposed feature refinement loss on both WSJ and Fearless Steps Challenge (FSC) corpora
- Combining diverse SSLRs using various fusion methods within E2E ASR models and refining these features through decorrelation could significantly improve ASR performance across different domains and applications.

## Summary

The use of self-supervised learning representations (SSLR) has proven to be effective in generating robust features for various downstream tasks. In recent studies, SSLRs have shown promising results in automatic speech recognition (ASR) benchmark corpora. However, previous research has only focused on the performance of solitary SSLRs as input features for ASR models. To address this gap, a team of researchers led by Szu-Jui Chen, Jiamin Xie, and John H. L. Hansen proposed investigating the effectiveness of diverse SSLR combinations using various fusion methods within end-to-end (E2E) ASR models. The study also aimed to explore the correlations between these extracted SSLRs. The team further proposed a feature refinement loss for decorrelation to efficiently combine the set of input features. This approach aims to improve the performance of SSLRs in ASR models by refining their features and reducing their correlation with each other. For evaluation purposes, the team tested their proposed 'FeaRLESS learning features' on both the WSJ and Fearless Steps Challenge (FSC) corpora. Their findings showed that FeaRLESS outperformed systems without the proposed feature refinement loss. Overall, this study highlights the potential benefits of combining diverse SSLRs using various fusion methods within E2E ASR models and proposes an efficient method for refining these features through decorrelation. These findings could have significant implications for improving ASR performance across different domains and applications.