paper-with-me

홈 › Papers

Speech Map: A Statistical Multimodal Atlas of 4D Tongue Motion During Speech from Tagged and Cine MR Images

2017-01-24 · Jonghye Woo, Fangxu Xing, Maureen Stone, Jordan Green, Timothy G. Reese, Thomas J. Brady, Van J. Wedeen, Jerry L. Prince, Georges El Fakhri

Quantitative measurement of functional and anatomical traits of 4D tongue motion in the course of speech or other lingual behaviors remains a major challenge in scientific research and clinical applications. Here, we introduce a statistical multimodal atlas of 4D tongue motion using healthy subjects, which enables a combined quantitative characterization of tongue motion in a reference anatomical configuration. This atlas framework, termed Speech Map, combines cine- and tagged-MRI in order to provide both the anatomic reference and motion information during speech. Our approach involves a series of steps including (1) construction of a common reference anatomical configuration from cine-MRI, (2) motion estimation from tagged-MRI, (3) transformation of the motion estimations to the reference anatomical configuration, and (4) computation of motion quantities such as Lagrangian strain. Using this framework, the anatomic configuration of the tongue appears motionless, while the motion fields and associated strain measurements change over the time course of speech. In addition, to form a succinct representation of the high-dimensional and complex motion fields, principal component analysis is carried out to characterize the central tendencies and variations of motion fields of our speech tasks. Our proposed method provides a platform to quantitatively and objectively explain the differences and variability of tongue motion by illuminating internal motion and strain that have so far been intractable. The findings are used to understand how tongue function for speech is limited by abnormal internal motion and strain in glossectomy patients.

📄 PDF Abstract BibTeX arXiv:1701.06708

Code (0)

등록된 구현이 없습니다.

Tasks

Motion Estimation

Similar Papers 제목 키워드 기반

A Multilinear Tongue Model Derived from Speech Related MRI Data of the Human Vocal Tract

2016-12-15 · Alexander Hewer, Stefanie Wuhrer, Ingmar Steiner, Korin Richmond

We present a multilinear statistical model of the human tongue that captures anatomical and tongue pose related shape variations separately. The model is derived from 3D magnetic resonance imaging data of 11 speakers sus…

DenoisingImage DenoisingImage SegmentationSemantic Segmentation

Contour-based 3d tongue motion visualization using ultrasound image sequences

2016-05-19 · Kele Xu, Yin Yang, Clémence Leboullenger, Pierre Roussel 외

This article describes a contour-based 3D tongue deformation visualization framework using B-mode ultrasound image sequences. A robust, automatic tracking algorithm characterizes tongue motion via a contour, which is the…

Silent Speech Recognitionspeech-recognitionSpeech Recognition

Improving Ultrasound Tongue Image Reconstruction from Lip Images Using Self-supervised Learning and Attention Mechanism

2021-06-20 · Haiyang Liu, Jihan Zhang

Speech production is a dynamic procedure, which involved multi human organs including the tongue, jaw and lips. Modeling the dynamics of the vocal tract deformation is a fundamental problem to understand the speech, whic…

Image ReconstructionSelf-Supervised Learning

The MMASCS multi-modal annotated synchronous corpus of audio, video, facial motion and tongue motion data of normal, fast and slow speech

2014-05-01 · LREC 2014 5 · Dietmar Schabus, Michael Pucher, Phil Hoole

In this paper, we describe and analyze a corpus of speech data that we have recorded in multiple modalities simultaneously: facial motion via optical motion capturing, tongue motion via electro-magnetic articulography, a…

Speech Synthesis

Speech Driven Tongue Animation

2022-01-01 · CVPR 2022 1 · Salvador Medina, Denis Tome, Carsten Stoll, Mark Tiede 외

Advances in speech driven animation techniques allow the creation of convincing animations for virtual characters solely from audio data. Many existing approaches focus on facial and lip motion and they often do not …

Decoder