# Localization, Detection and Tracking of Multiple Moving Sound Sources with a Convolutional Recurrent Neural Network

**Authors:** Sharath Adavanne, Archontis Politis, Tuomas Virtanen  
**arXiv:** [1904.12769](https://arxiv.org/abs/1904.12769)  
**Date:** 2019-04-29  
**Summary page:** https://www.summarizepaper.com/en/paper/1904.12769v1/

## Key points

- This paper explores joint localization, detection, and tracking of sound events using a convolutional recurrent neural network (CRNN).
- The CRNN model is adapted to enable spatial tracking of moving sources when trained with dynamic scenes.
- Performance of the CRNN is compared with a stand-alone tracking method that combines a multi-source estimator and a particle filter.
- Experiments evaluate performance in various acoustic conditions including anechoic and reverberant scenarios, stationary and moving sources at different angular velocities, and varying numbers of overlapping sources.
- The CRNN consistently tracks multiple sources more effectively than the parametric method across different acoustic scenarios but has higher localization error.
- The parametric method achieves improved direction-of-arrival (DOA) estimation when combined with a temporal particle filter tracker but suffers from lower frame recall.
- Using maximum likelihood estimation instead of reference information for source number estimation reduces the overall performance of the parametric approach, particularly in reverberant and moving source scenarios.
- Both methods have trade-offs: CRNN outperforms in consistent tracking but struggles with accurate localization, while the parametric method achieves better DOA estimation but has decreased frame recall in certain scenarios.
- Consideration should be given to both tracking consistency and localization accuracy when choosing between these two methods.
- Recurrent layers within a CRNN architecture can achieve effective tracking of multiple sound sources, but further improvements are needed for localization accuracy and robustness in challenging acoustic conditions.

## Summary

This paper explores the joint localization, detection and tracking of sound events using a convolutional recurrent neural network (CRNN). The CRNN model, previously proposed for localizing and detecting stationary sources, is adapted to enable the spatial tracking of moving sources when trained with dynamic scenes. The performance of the CRNN is compared with a stand-alone tracking method that combines a multi-source estimator and a particle filter. The study evaluates the performance of both methods in various acoustic conditions including anechoic and reverberant scenarios as well as stationary and moving sources at different angular velocities. Additionally, the experiments consider scenarios with varying numbers of overlapping sources. The results show that the CRNN consistently tracks multiple sources more effectively than the parametric method across different acoustic scenarios. However, it does come at the cost of higher localization error. The parametric method exhibits improved direction-of-arrival (DOA) estimation when combined with a temporal particle filter tracker but suffers from lower frame recall. Further analysis reveals that using maximum likelihood estimation instead of reference information for source number estimation reduces the overall performance of the parametric approach. This reduction is particularly evident in reverberant and moving source scenario datasets, highlighting the need for more robust source detection and counting schemes. Overall, while the CRNN outperforms the parametric method in terms of consistent tracking performance, it struggles with accurate localization. On the other hand, although the parametric method achieves better DOA estimation when combined with a particle filter tracker its frame recall decreases significantly in certain scenarios. These findings emphasize the importance of considering both tracking consistency and localization accuracy when choosing between these two methods. In conclusion, this study demonstrates that by leveraging recurrent layers within a CRNN architecture it is possible to achieve effective tracking of multiple sound sources. However further improvements are needed to enhance localization accuracy and robustness in challenging acoustic conditions.