Running summarizing tools on a new article

Abstract: In recent years, speech synthesis models based on conditional flow matching have attracted increasing attention for their fast generation and high audio quality. However, in zero-shot settings, existing flow-matching models still struggle to achieve fine-grained control over emotional prosody while preserving the target speaker’s timbre, and the emotion in the reference audio often leaks into the synthesized speech. To address these issues, we propose an emotion-controllable zero-shot speech synthesis system built on Matcha-TTS. First, we design an emotion-enhanced conditioning module in the acoustic model, which introduces an explicit prosody predictor and emotion embeddings to generate a prosody-enriched conditional mean that guides vector-field prediction to synthesize highly expressive speech. Second, to mitigate the entanglement between timbre and emotion, we propose a disentangled timbre encoder based on contrastive learning and adversarial training, where a gradient reversal layer is used to remove residual emotional information from the timbre features, enabling independent control of timbre and emotion in zero-shot inference. Experimental results show that the proposed method outperforms the baseline in both objective metrics and subjective listening tests, and can synthesize speech with rich emotional expressiveness and high timbre similarity using only a few sampling steps.