# w2v-SELD: A Sound Event Localization and Detection Framework for Self-Supervised Spatial Audio Pre-Training

**Authors:** Orlem Lima dos Santos, Karen Rosero, Roberto de Alencar Lotufo  
**arXiv:** [2312.06907](https://arxiv.org/abs/2312.06907)  
**Date:** 2023-12-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.06907v1/

## Key points

- Introduction of w2v-SELD: a self-supervised approach for Sound Event Detection and Localization (SELD)
- Utilization of wav2vec 2.0 pre-training methodology to learn representations directly from raw audio data without supervision
- Two main stages of the proposed method: pre-training on unlabeled 3D audio datasets and fine-tuning on labeled SELD data
- Experimental results showing effectiveness of w2v-SELD model in surpassing baseline systems and achieving competitive performance
- Notable contributions in SELD research highlighted by the authors, including SSL approach tailored for SELD, incorporation of SED and DOA estimation at frame-level, and significant improvement in SELDscore metric with pre-trained weights

## Summary

, , , , 

In the paper titled "w2v-SELD: A Sound Event Localization and Detection Framework for Self-Supervised Spatial Audio Pre-Training," Orlem Lima dos Santos, Karen Rosero, and Roberto de Alencar Lotufo introduce a novel self-supervised approach for Sound Event Detection and Localization (SELD). The approach leverages the pre-training methodology of wav2vec 2.0 to learn representations directly from raw audio data without supervision. This is achieved by utilizing unlabeled 3D audio datasets to capture robust representations of sound events and their locations. The proposed method consists of two main stages: pre-training and fine-tuning. In the pre-training phase, the w2v-SELD model is trained on unlabeled 3D audio datasets to extract intricate high-level features and contextual information from audio signals. Subsequently, in the fine-tuning stage, a smaller dataset with labeled SELD data is used to refine the pre-trained model. Experimental results on benchmark datasets demonstrate the effectiveness of this self-supervised approach for SELD. The w2v-SELD model surpasses baseline systems provided with the datasets and achieves competitive performance comparable to state-of-the-art supervised methods. The authors provide access to the code and pre-trained parameters of their model in a repository. Additionally, based on their findings, the authors present several notable contributions in SELD research. They highlight an SSL approach tailored for SELD using wav2vec 2.0 pre-training framework, showcasing robust models without heavy reliance on labeled spatial audio datasets. The fine-tuning process of the w2v-SELD model incorporates Sound Event Detection (SED) and Direction of Arrival (DOA) estimation at frame-level, enhancing prediction precision and accuracy. Furthermore, a comprehensive evaluation of the w2v-SELD model pre-trained on diverse datasets reveals insights into its adaptability and performance across different configurations. It demonstrates a significant improvement in SELDscore metric when using pre-trained weights instead of training from scratch. Overall, this paper contributes valuable advancements in SELD research through a self-supervised spatial audio pre-training framework that shows promise in improving sound event detection and localization tasks with minimal supervision requirements.