# SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation

**Authors:** Changsheng Quan, Xiaofei Li  
**arXiv:** [2307.16516](https://arxiv.org/abs/2307.16516)  
**Date:** 2023-07-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.16516v1/

## Key points

- SpatialNet is a neural network designed for multichannel joint speech separation, denoising, and dereverberation.
- The network operates in the STFT domain and performs end-to-end speech enhancement.
- SpatialNet consists of interleaved narrow-band and cross-band blocks to leverage spatial information.
- Narrow-band blocks use self-attention mechanisms and temporal convolutional layers for speaker clustering and smoothing/filtering.
- Cross-band blocks employ linear and frequency convolutional layers to learn correlations between frequencies.
- Experiments on simulated and real datasets show that SpatialNet achieves state-of-the-art performance.
- SpatialNet is minimally susceptible to spectral generalization problems encountered in similar systems.
- Attention maps generated by SpatialNet provide evidence of effective speaker clustering.
- Overall, SpatialNet presents a novel approach with superior performance compared to existing methods while being robust against spectral generalization issues.

## Summary

This work introduces SpatialNet, a neural network designed to exploit spatial information for multichannel joint speech separation, denoising, and dereverberation. The network operates in the short-time Fourier transform (STFT) domain and performs end-to-end speech enhancement. SpatialNet consists of interleaved narrow-band and cross-band blocks that aim to leverage narrow-band and cross-band spatial information, respectively. The narrow-band blocks process frequencies independently and utilize self-attention mechanisms and temporal convolutional layers for spatial-feature-based speaker clustering and temporal smoothing/filtering. On the other hand, the cross-band blocks process frames independently and employ a full-band linear layer and frequency convolutional layers to learn the correlation between all frequencies and adjacent frequencies. To evaluate the performance of SpatialNet, experiments are conducted on various simulated and real datasets. The results demonstrate that the proposed network achieves state-of-the-art performance across multiple tasks. Additionally, SpatialNet shows minimal susceptibility to the spectral generalization problem commonly encountered in similar systems. Furthermore, attention maps generated by SpatialNet provide evidence that the network effectively performs speaker clustering. In summary, SpatialNet presents a novel approach for exploiting spatial information in multichannel joint speech separation, denoising, and dereverberation tasks. Its effectiveness is demonstrated through superior performance compared to existing methods in various experimental scenarios while also exhibiting robustness against spectral generalization issues.