DurIAN-E: Duration Informed Attention Network For Expressive Text-to-Speech Synthesis

2023-09-22 · Yu Gu, Yianrao Bian, Guangzhi Lei, Chao Weng, Dan Su

This paper introduces an improved duration informed attention neural network (DurIAN-E) for expressive and high-fidelity text-to-speech (TTS) synthesis. Inherited from the original DurIAN model, an auto-regressive model structure in which the alignments between the input linguistic information and the output acoustic features are inferred from a duration model is adopted. Meanwhile the proposed DurIAN-E utilizes multiple stacked SwishRNN-based Transformer blocks as linguistic encoders. Style-Adaptive Instance Normalization (SAIN) layers are exploited into frame-level encoders to improve the modeling ability of expressiveness. A denoiser incorporating both denoising diffusion probabilistic model (DDPM) for mel-spectrograms and SAIN modules is conducted to further improve the synthetic speech quality and expressiveness. Experimental results prove that the proposed expressive TTS model in this paper can achieve better performance than the state-of-the-art approaches in both subjective mean opinion score (MOS) and preference tests.

📄 PDF Abstract BibTeX arXiv:2309.12792

Code (0)

등록된 구현이 없습니다.

Tasks

Denoising Speech Synthesis text-to-speech Text to Speech Text-To-Speech Synthesis

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

DurIAN-E 2: Duration Informed Attention Network with Adaptive Variational Autoencoder and Adversarial Learning for Expressive Text-to-Speech Synthesis

2024-10-17 · Yu Gu, Qiushi Zhu, Guangzhi Lei, Chao Weng 외

This paper proposes an improved version of DurIAN-E (DurIAN-E 2), which is also a duration informed attention neural network for expressive and high-fidelity text-to-speech (TTS) synthesis. Similar with the DurIAN-E mode…

Speech Synthesis text-to-speech Text to Speech Text-To-Speech Synthesis

DurIAN: Duration Informed Attention Network For Multimodal Synthesis

2019-09-04 · Chengzhu Yu, Heng Lu, Na Hu, Meng Yu 외

In this paper, we present a generic and robust multimodal synthesis system that produces highly natural speech and facial expression simultaneously. The key component of this system is the Duration Informed Attention Net…

CPU Speech Synthesis

Synthesising Expressiveness in Peking Opera via Duration Informed Attention Network

2019-12-27 · Yusong Wu, Shengchen Li, Chengzhu Yu, Heng Lu 외

This paper presents a method that generates expressive singing voice of Peking opera. The synthesis of expressive opera singing usually requires pitch contours to be extracted as the training data, which relies on techni…

Peking Opera Synthesis via Duration Informed Attention Network

2020-08-07 · Yusong Wu, Shengchen Li, Chengzhu Yu, Heng Lu 외

Peking Opera has been the most dominant form of Chinese performing art since around 200 years ago. A Peking Opera singer usually exhibits a very strong personal style via introducing improvisation and expressiveness on s…

Rhythm Singing Voice Synthesis

JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment

2020-05-15 · Dan Lim, Won Jang, Gyeonghwan O, Heayoung Park 외

We propose Jointly trained Duration Informed Transformer (JDI-T), a feed-forward Transformer with a duration predictor jointly trained without explicit alignments in order to generate an acoustic feature sequence from an…

text-to-speech Text to Speech