# TS3-Codec: Transformer-Based Simple Streaming Single Codec

**Authors:** Haibin Wu, Naoyuki Kanda, Sefik Emre Eskimez, Jinyu Li  
**arXiv:** [2411.18803](https://arxiv.org/abs/2411.18803)  
**Date:** 2024-11-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2411.18803v1/

## Key points

- Neural audio codecs (NACs) are essential technologies for compressing and representing audio in speech language models.
- Transformer-based NACs, such as TS3-Codec, challenge traditional approaches by utilizing only transformer layers and a few linear layers.
- TS3-Codec simplifies the model and enhances expressive power by eliminating the need for intricate hyperparameter tuning and computationally intensive convolutional layers.
- Under a streaming setup, TS3-Codec achieves comparable or superior results to state-of-the-art convolution-based codecs while consuming only 12% of the computational resources and operating at 77% of the bitrate.
- When evaluated under similar computational constraints, TS3-Codec outperforms its convolution-based counterparts significantly.
- Transformer-based NACs like TS3-Codec offer efficiency and effectiveness in handling audio data with reduced computational demands, showcasing promise for audio compression and representation tasks.

## Summary

Neural audio codecs (NACs) have become essential technologies for compressing and representing audio in speech language models. While traditional NAC models heavily rely on convolutional architectures, the potential of transformer-based NACs with a convolution-free design has been largely unexplored. In their study, Wu et al. introduce TS3-Codec - a Transformer-Based Simple Streaming Single Codec that challenges the conventional approach by utilizing only transformer layers and a few linear layers. This streamlined architecture not only simplifies the model but also enhances its expressive power by eliminating the need for intricate hyperparameter tuning and computationally intensive convolutional layers. Under a streaming setup, TS3-Codec demonstrates remarkable performance, achieving comparable or even superior results when compared to state-of-the-art convolution-based codecs while consuming only 12% of the computational resources and operating at 77% of the bitrate. Furthermore, when evaluated under similar computational constraints, TS3-Codec outperforms its convolution-based counterparts significantly. These findings suggest that transformer-based NACs like TS3-Codec offer a promising alternative for audio compression and representation tasks, showcasing efficiency and effectiveness in handling audio data with reduced computational demands. The study by Wu et al. sheds light on the potential of transformer architectures in revolutionizing audio codec technology and paves the way for further exploration in this domain.