# UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding

**Authors:** Chenpeng Du, Yiwei Guo, Feiyu Shen, Zhijun Liu, Zheng Liang, Xie Chen, Shuai Wang, Hui Zhang, Kai Yu  
**arXiv:** [2306.07547](https://arxiv.org/abs/2306.07547)  
**Date:** 2023-06-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.07547v6/

## Key points

- Authors Chenpeng Du, Yiwei Guo, Feiyu Shen, Zhijun Liu, Zheng Liang, Xie Chen, Shuai Wang, Hui Zhang, and Kai Yu introduce UniCATS framework for text-to-speech synthesis.
- UniCATS addresses limitations of existing models like VALL-E and SPEAR-TTS in speech editing due to left-to-right generation constraints and limited audio quality of acoustic tokens.
- UniCATS framework consists of two main components: CTX-txt2vec and CTX-vec2wav.
- CTX-txt2vec utilizes contextual VQ-diffusion for predicting semantic tokens from input text.
- CTX-vec2wav employs contextual vocoding to convert semantic tokens into waveforms while considering the acoustic context.
- Experimental results show that CTX-vec2wav outperforms models like HifiGAN and AudioLM in speech resynthesis from semantic tokens.
- UniCATS achieves state-of-the-art performance in speech continuation and editing tasks while leveraging contextual information.

## Summary

, , , , 

In their paper titled "UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding," authors Chenpeng Du, Yiwei Guo, Feiyu Shen, Zhijun Liu, Zheng Liang, Xie Chen, Shuai Wang, Hui Zhang, and Kai Yu introduce a novel approach to text-to-speech synthesis. They address the limitations of existing models such as VALL-E and SPEAR-TTS in terms of speech editing due to their left-to-right generation constraints and reliance on acoustic tokens with limited audio quality. The proposed UniCATS framework consists of two main components: CTX-txt2vec and CTX-vec2wav. <kw>CTX-txt2vec:</kw> This component utilizes contextual VQ-diffusion to predict semantic tokens from input text. This allows for seamless concatenation with surrounding context and incorporation of semantic information. <kw>CTX-vec2wav:</kw> On the other hand, this component employs contextual vocoding to convert these semantic tokens into waveforms while considering the acoustic context. Experimental results demonstrate that CTX-vec2wav outperforms existing models like HifiGAN and AudioLM in terms of speech resynthesis from semantic tokens. The authors' work has been accepted for presentation at AAAI 2024, showcasing the significance of their contributions to advancing text-to-speech technology. Their proposed UniCATS framework not only achieves state-of-the-art performance in both speech continuation and editing tasks but also addresses the limitations of existing models. With its unified approach and use of contextual information, UniCATS shows great potential for improving text-to-speech synthesis.