paper-with-me

홈 › Papers

Score-balanced Loss for Multi-aspect Pronunciation Assessment

2023-05-26 · Heejin Do, Yunsu Kim, Gary Geunbae Lee

With rapid technological growth, automatic pronunciation assessment has transitioned toward systems that evaluate pronunciation in various aspects, such as fluency and stress. However, despite the highly imbalanced score labels within each aspect, existing studies have rarely tackled the data imbalance problem. In this paper, we suggest a novel loss function, score-balanced loss, to address the problem caused by uneven data, such as bias toward the majority scores. As a re-weighting approach, we assign higher costs when the predicted score is of the minority class, thus, guiding the model to gain positive feedback for sparse score prediction. Specifically, we design two weighting factors by leveraging the concept of an effective number of samples and using the ranks of scores. We evaluate our method on the speechocean762 dataset, which has noticeably imbalanced scores for several aspects. Improved results particularly on such uneven aspects prove the effectiveness of our method.

📄 PDF Abstract BibTeX arXiv:2305.16664

Code (1)

doheejin/sb_loss_pa 공식 구현 pytorch

Similar Papers 제목 키워드 기반

Acoustic Feature Mixup for Balanced Multi-aspect Pronunciation Assessment

2024-06-22 · Heejin Do, Wonjun Lee, Gary Geunbae Lee

In automated pronunciation assessment, recent emphasis progressively lies on evaluating multiple aspects to provide enriched feedback. However, acquiring multi-aspect-score labeled data for non-native language learners' …

speech-recognitionSpeech Recognition

Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Loss

2025-02-11 · Fu-An Chao, Berlin Chen

Prior efforts in building computer-assisted pronunciation training (CAPT) systems often treat automatic pronunciation assessment (APA) and mispronunciation detection and diagnosis (MDD) as separate fronts: the former aim…

Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment

2022-05-06 · Yuan Gong, Ziyi Chen, Iek-Heng Chu, Peng Chang 외

Automatic pronunciation assessment is an important technology to help self-directed language learners. While pronunciation quality has multiple aspects including accuracy, fluency, completeness, and prosody, previous eff…

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Multi-Task LearningPhone-level pronunciation scoring+4

A Hierarchical Context-aware Modeling Approach for Multi-aspect and Multi-granular Pronunciation Assessment

2023-05-29 · Fu-An Chao, Tien-Hong Lo, Tzu-I Wu, Yao-Ting Sung 외

Automatic Pronunciation Assessment (APA) plays a vital role in Computer-assisted Pronunciation Training (CAPT) when evaluating a second language (L2) learner's speaking proficiency. However, an apparent downside of most …

Automatic Speech RecognitionMulti-Task LearningPhone-level pronunciation scoringSentence+2

Hierarchical Pronunciation Assessment with Multi-Aspect Attention

2022-11-15 · Heejin Do, Yunsu Kim, Gary Geunbae Lee

Automatic pronunciation assessment is a major component of a computer-assisted pronunciation training system. To provide in-depth feedback, scoring pronunciation at various levels of granularity such as phoneme, word, an…

Multi-Task LearningPhone-level pronunciation scoringUtterance-level pronounciation scoringWord-level pronunciation scoring