SpeechLMScore: Evaluating speech generation using speech language model
While human evaluation is the most reliable metric for evaluating speech generation systems, it is generally costly and time-consuming. Previous studies on automatic speech quality assessment address the problem by predicting human evaluation scores with machine learning models. However, they rely on supervised learning and thus suffer from high annotation costs and domain-shift problems. We propose SpeechLMScore, an unsupervised metric to evaluate generated speech using a speech-language model. SpeechLMScore computes the average log-probability of a speech signal by mapping it into discrete tokens and measures the average probability of generating the sequence of tokens. Therefore, it does not require human annotation and is a highly scalable framework. Evaluation results demonstrate that the proposed metric shows a promising correlation with human evaluation scores on different speech generation tasks including voice conversion, text-to-speech, and speech enhancement.
Code (2)
Tasks
Language ModelingLanguage ModellingmodelSpeech Enhancementtext-to-speechText to SpeechVoice ConversionSimilar Papers 제목 키워드 기반
A Study on Zero-shot Non-intrusive Speech Assessment using Large Language Models
This work investigates two strategies for zero-shot non-intrusive speech assessment leveraging large language models. First, we explore the audio analysis capabilities of GPT-4o. Second, we propose GPT-Whisper, which use…
Automatic Speech RecognitionPrompt Engineeringspeech-recognitionSpeech RecognitionEvaluating Speech Synthesis by Training Recognizers on Synthetic Speech
Modern speech synthesis systems have improved significantly, with synthetic speech being indistinguishable from real speech. However, efficient and holistic evaluation of synthetic speech still remains a significant chal…
speech-recognitionSpeech RecognitionSpeech Synthesistext-to-speech+1LE-SSL-MOS: Self-Supervised Learning MOS Prediction with Listener Enhancement
Recently, researchers have shown an increasing interest in automatically predicting the subjective evaluation for speech synthesis systems. This prediction is a challenging task, especially on the out-of-domain test set.…
Ensemble LearningLanguage ModellingPredictionSelf-Supervised Learning+1Speech Codec Probing from Semantic and Phonetic Perspectives
Speech tokenizers are essential for connecting speech to large language models (LLMs) in multimodal systems. Speech tokenizers are expected to preserve both semantic and acoustic information for downstream understanding …
S2S-Arena, Evaluating Speech2Speech Protocols on Instruction Following with Paralinguistic Information
The rapid development of large language models (LLMs) has brought significant attention to speech models, particularly recent progress in speech2speech protocols supporting speech input and output. However, the existing …
Instruction Following