Generative Modeling for Low Dimensional Speech Attributes with Neural Spline Flows
Despite recent advances in generative modeling for text-to-speech synthesis, these models do not yet have the same fine-grained adjustability of pitch-conditioned deterministic models such as FastPitch and FastSpeech2. Pitch information is not only low-dimensional, but also discontinuous, making it particularly difficult to model in a generative setting. Our work explores several techniques for handling the aforementioned issues in the context of Normalizing Flow models. We also find this problem to be very well suited for Neural Spline flows, which is a highly expressive alternative to the more common affine-coupling mechanism in Normalizing Flows.
Code (1)
Tasks
Speech Synthesistext-to-speechText to SpeechText-To-Speech SynthesisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Hierarchical Generative Modeling for Controllable Speech Synthesis
This paper proposes a neural sequence-to-sequence text-to-speech (TTS) model which can control latent attributes in the generated speech that are rarely annotated in the training data, such as speaking style, accent, bac…
AttributeSpeech Synthesistext-to-speechText to SpeechSplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Video
Synthesizing novel views from in-the-wild monocular videos is challenging due to scene dynamics and the lack of multi-view cues. To address this, we propose SplineGS, a COLMAP-free dynamic 3D Gaussian Splatting (3DGS) fr…
3DGSNovel View Synthesisparameter estimationA Variational Framework for Improving Naturalness in Generative Spoken Language Models
The success of large language models in text processing has inspired their adaptation to speech modeling. However, since speech is continuous and complex, it is often discretized for autoregressive modeling. Speech token…
Deep Random Splines for Point Process Intensity Estimation of Neural Population Data
Gaussian processes are the leading class of distributions on random functions, but they suffer from well known issues including difficulty scaling and inflexibility with respect to certain shape constraints (such as nonn…
Dimensionality ReductionGaussian ProcessesPoint ProcessesA survey of probabilistic generative frameworks for molecular simulations
Generative artificial intelligence is now a widely used tool in molecular science. Despite the popularity of probabilistic generative models, numerical experiments benchmarking their performance on molecular data are lac…
BenchmarkingDenoisingModel SelectionSurvey