Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment
Automatic pronunciation assessment is an important technology to help self-directed language learners. While pronunciation quality has multiple aspects including accuracy, fluency, completeness, and prosody, previous efforts typically only model one aspect (e.g., accuracy) at one granularity (e.g., at the phoneme-level). In this work, we explore modeling multi-aspect pronunciation assessment at multiple granularities. Specifically, we train a Goodness Of Pronunciation feature-based Transformer (GOPT) with multi-task learning. Experiments show that GOPT achieves the best results on speechocean762 with a public automatic speech recognition (ASR) acoustic model trained on Librispeech.
Code (1)
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Multi-Task LearningPhone-level pronunciation scoringspeech-recognitionSpeech RecognitionUtterance-level pronounciation scoringWord-level pronunciation scoringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multilevel Transformer For Multimodal Emotion Recognition
Multimodal emotion recognition has attracted much attention recently. Fusing multiple modalities effectively with limited labeled data is a challenging task. Considering the success of pre-trained model and fine-grained …
Emotion RecognitionMultimodal Emotion RecognitionHierarchical Pronunciation Assessment with Multi-Aspect Attention
Automatic pronunciation assessment is a major component of a computer-assisted pronunciation training system. To provide in-depth feedback, scoring pronunciation at various levels of granularity such as phoneme, word, an…
Multi-Task LearningPhone-level pronunciation scoringUtterance-level pronounciation scoringWord-level pronunciation scoringA Multi-Granularity-Aware Aspect Learning Model for Multi-Aspect Dense Retrieval
Dense retrieval methods have been mostly focused on unstructured text and less attention has been drawn to structured data with various aspects, e.g., products with aspects such as category and brand. Recent work has pro…
Language ModellingRetrievalValue predictionExploring the Potential of Large Multimodal Models as Effective Alternatives for Pronunciation Assessment
Large Multimodal Models (LMMs) have demonstrated exceptional performance across a wide range of domains. This paper explores their potential in pronunciation assessment tasks, with a particular focus on evaluating the ca…
Multi-Granularity Framework for Unsupervised Representation Learning of Time Series
Representation learning plays a critical role in the analysis of time series data and has high practical value across a wide range of applications. including trend analysis, time series data retrieval and forecasting. In…
Decision MakingRepresentation LearningRetrievalTime Series