Sequence-to-sequence Singing Voice Synthesis with Perceptual Entropy Loss
The neural network (NN) based singing voice synthesis (SVS) systems require sufficient data to train well and are prone to over-fitting due to data scarcity. However, we often encounter data limitation problem in building SVS systems because of high data acquisition and annotation costs. In this work, we propose a Perceptual Entropy (PE) loss derived from a psycho-acoustic hearing model to regularize the network. With a one-hour open-source singing voice database, we explore the impact of the PE loss on various mainstream sequence-to-sequence models, including the RNN-based, transformer-based, and conformer-based models. Our experiments show that the PE loss can mitigate the over-fitting problem and significantly improve the synthesized singing quality reflected in objective and subjective evaluations.
Code (1)
Tasks
Singing Voice SynthesisSimilar Papers 제목 키워드 기반
Singing voice synthesis based on convolutional neural networks
The present paper describes a singing voice synthesis based on convolutional neural networks (CNNs). Singing voice synthesis systems based on deep neural networks (DNNs) are currently being proposed and are improving the…
Singing Voice SynthesisControllable Singing Voice Synthesis using Phoneme-Level Energy Sequence
Controllable Singing Voice Synthesis (SVS) aims to generate expressive singing voices reflecting user intent. While recent SVS systems achieve high audio quality, most rely on probabilistic modeling, limiting precise con…
Fast and High-Quality Singing Voice Synthesis System based on Convolutional Neural Networks
The present paper describes singing voice synthesis based on convolutional neural networks (CNNs). Singing voice synthesis systems based on deep neural networks (DNNs) are currently being proposed and are improving the n…
Singing Voice SynthesisSinging Voice Synthesis Based on a Musical Note Position-Aware Attention Mechanism
This paper proposes a novel sequence-to-sequence (seq2seq) model with a musical note position-aware attention mechanism for singing voice synthesis (SVS). A seq2seq modeling approach that can simultaneously perform acous…
DecoderPositionRhythmSinging Voice SynthesisHiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis
High-fidelity singing voices usually require higher sampling rate (e.g., 48kHz) to convey expression and emotion. However, higher sampling rate causes the wider frequency band and longer waveform sequences and throws cha…
Singing Voice SynthesisVocal Bursts Intensity Prediction