Articulatory-WaveNet: Autoregressive Model For Acoustic-to-Articulatory Inversion

2020-06-22 · Narjes Bozorg, Michael T. Johnson

This paper presents Articulatory-WaveNet, a new approach for acoustic-to-articulator inversion. The proposed system uses the WaveNet speech synthesis architecture, with dilated causal convolutional layers using previous values of the predicted articulatory trajectories conditioned on acoustic features. The system was trained and evaluated on the ElectroMagnetic Articulography corpus of Mandarin Accented English (EMA-MAE),consisting of 39 speakers including both native English speakers and native Mandarin speakers speaking English. Results show significant improvement in both correlation and RMSE between the generated and true articulatory trajectories for the new method, with an average correlation of 0.83, representing a 36% relative improvement over the 0.61 correlation obtained with a baseline Hidden Markov Model (HMM)-Gaussian Mixture Model (GMM) inversion framework. To the best of our knowledge, this paper presents the first application of a point-by-point waveform synthesis approach to the problem of acoustic-to-articulatory inversion and the results show improved performance compared to previous methods for speaker dependent acoustic to articulatory inversion.

📄 PDF Abstract BibTeX arXiv:2006.12594

Code (0)

등록된 구현이 없습니다.

Tasks

Speech Synthesis

Methods 이 논문이 사용한 방법론

Mixture of Logistic Distributions 설명 없음

Dilated Causal Convolution A Dilated Causal Convolution is a causal convolution where the filter is applied over an area larger than its length by…

WaveNet WaveNet is an audio generative model based on the PixelCNN architecture. In order to deal with long-range temporal dependencies…

Acoustic-to-Articulatory Speech Inversion Features for Mispronunciation Detection of /r/ in Child Speech Sound Disorders

2023-05-25 · Nina R Benway, Yashish M Siriwardena, Jonathan L Preston, Elaine Hitchcock 외

Acoustic-to-articulatory speech inversion could enhance automated clinical mispronunciation detection to provide detailed articulatory feedback unattainable by formant-based mispronunciation detection algorithms; however…

Acoustic-to-articulatory Speech Inversion with Multi-task Learning

2022-05-27 · Yashish M. Siriwardena, Ganesh Sivaraman, Carol Espy-Wilson

Multi-task learning (MTL) frameworks have proven to be effective in diverse speech related tasks like automatic speech recognition (ASR) and speech emotion recognition. This paper proposes a MTL framework to perform acou…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Emotion Recognition Multi-Task Learning+3

Exploiting Cross Domain Acoustic-to-articulatory Inverted Features For Disordered Speech Recognition

2022-03-19 · Shujie Hu, Shansong Liu, Xurong Xie, Mengzhe Geng 외

Articulatory features are inherently invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition (ASR) systems for normal speech. Their practical application to disor…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Data Augmentation speech-recognition+1

Improving generalization of vocal tract feature reconstruction: from augmented acoustic inversion to articulatory feature reconstruction without articulatory data

2018-09-04 · Rosanna Turrisi, Raffaele Tavarone, Leonardo Badino

We address the problem of reconstructing articulatory movements, given audio and/or phonetic labels. The scarce availability of multi-speaker articulatory data makes it difficult to learn a reconstruction that generalize…

ArtBoost: Synthetic Articulatory Data Augmentation for Acoustic-to-Articulatory Inversion

2026-06-15 · Hyung Kyu Kim, Byungchan Hwang, Hak Gu Kim arxiv

Recent acoustic-to-articulatory inversion (AAI) models rely on electromagnetic articulography (EMA) data, which are costly and limited in scale. To address this limitation, we propose \textit{ArtBoost}, a novel data augm…

Data Augmentation