Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Prediction
Predicting audio quality in voice synthesis and conversion systems is a critical yet challenging task, especially when traditional methods like Mean Opinion Scores (MOS) are cumbersome to collect at scale. This paper addresses the gap in efficient audio quality prediction, especially in low-resource settings where extensive MOS data from large-scale listening tests may be unavailable. We demonstrate that uncertainty measures derived from out-of-the-box pretrained self-supervised learning (SSL) models, such as wav2vec, correlate with MOS scores. These findings are based on data from the 2022 and 2023 VoiceMOS challenges. We explore the extent of this correlation across different models and language contexts, revealing insights into how inherent uncertainties in SSL models can serve as effective proxies for audio quality assessment. In particular, we show that the contrastive wav2vec models are the most performant in all settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Audio Quality AssessmentSelf-Supervised LearningSimilar Papers 제목 키워드 기반
Uncertainty-aware Self-supervised 3D Data Association
3D object trackers usually require training on large amounts of annotated data that is expensive and time-consuming to collect. Instead, we propose leveraging vast unlabeled datasets by self-supervised metric learning of…
Metric LearningObjectobject-detectionObject DetectionSEMF: Supervised Expectation-Maximization Framework for Predicting Intervals
This work introduces the Supervised Expectation-Maximization Framework (SEMF), a versatile and model-agnostic approach for generating prediction intervals with any ML model. SEMF extends the Expectation-Maximization algo…
PredictionPrediction Intervalsquantile regressionregression+1Zero-shot text-to-speech synthesis conditioned using self-supervised speech representation model
This paper proposes a zero-shot text-to-speech (TTS) conditioned by a self-supervised speech-representation model acquired through self-supervised learning (SSL). Conventional methods with embedding vectors from x-vector…
RhythmSelf-Supervised LearningSpeech Synthesistext-to-speech+2Uncertainty-aware Self-training for Few-shot Text Classification
Recent success of pre-trained language models crucially hinges on fine-tuning them on large amounts of labeled data for the downstream task, that are typically expensive to acquire or difficult to access for many applica…
ClassificationFew-Shot Text ClassificationGeneral Classificationtext-classification+1Uncertainty-aware Self-training for Text Classification with Few Labels
Recent success of large-scale pre-trained language models crucially hinge on fine-tuning them on large amounts of labeled data for the downstream task, that are typically expensive to acquire. In this work, we study self…
General ClassificationPseudo Labeltext-classificationText Classification