paper-with-me

홈 › Papers

Multi-dimensional Speech Quality Assessment in Crowdsourcing

2023-09-14 · Babak Naderi, Ross Cutler, Nicolae-Catalin Ristea

Subjective speech quality assessment is the gold standard for evaluating speech enhancement processing and telecommunication systems. The commonly used standard ITU-T Rec. P.800 defines how to measure speech quality in lab environments, and ITU-T Rec.~P.808 extended it for crowdsourcing. ITU-T Rec. P.835 extends P.800 to measure the quality of speech in the presence of noise. ITU-T Rec. P.804 targets the conversation test and introduces perceptual speech quality dimensions which are measured during the listening phase of the conversation. The perceptual dimensions are noisiness, coloration, discontinuity, and loudness. We create a crowdsourcing implementation of a multi-dimensional subjective test following the scales from P.804 and extend it to include reverberation, the speech signal, and overall quality. We show the tool is both accurate and reproducible. The tool has been used in the ICASSP 2023 Speech Signal Improvement challenge and we show the utility of these speech quality dimensions in this challenge. The tool will be publicly available as open-source at https://github.com/microsoft/P.808.

📄 PDF Abstract BibTeX arXiv:2309.07385

Code (2)

microsoft/P.808 공식 구현
microsoft/sig-challenge

Tasks

Speech Enhancement

Similar Papers 제목 키워드 기반

The Many Dimensions of Truthfulness: Crowdsourcing Misinformation Assessments on a Multidimensional Scale

2021-08-03 · Michael Soprano, Kevin Roitero, David La Barbera, Davide Ceolin 외

Recent work has demonstrated the viability of using crowdsourcing as a tool for evaluating the truthfulness of public statements. Under certain conditions such as: (1) having a balanced set of workers with different back…

InformativenessMisinformation

On Crowdsourcing-design with Comparison Category Rating for Evaluating Speech Enhancement Algorithms

2023-06-02 · Angélica S. Z. Suárez, Clément Laroche, Line H. Clemmensen, Sneha Das

Speech enhancement techniques improve the quality or the intelligibility of an audio signal by removing unwanted noise. It is used as preprocessing in numerous applications such as speech recognition, hearing aids, broad…

Speech Enhancementspeech-recognitionSpeech Recognition

An Open source Implementation of ITU-T Recommendation P.808 with Validation

2020-05-17 · Babak Naderi, Ross Cutler

The ITU-T Recommendation P.808 provides a crowdsourcing approach for conducting a subjective assessment of speech quality using the Absolute Category Rating (ACR) method. We provide an open-source implementation of the I…

Mitigating Cognitive Biases in Multi-Criteria Crowd Assessment

2024-07-10 · Shun Ito, Hisashi Kashima

Crowdsourcing is an easy, cheap, and fast way to perform large scale quality assessment; however, human judgments are often influenced by cognitive biases, which lowers their credibility. In this study, we focus on cogni…

Calibration-Reasoning Framework for Descriptive Speech Quality Assessment

2026-03-10 · Elizaveta Kostenok, Mathieu Salzmann, Milos Cernak arxiv

Explainable speech quality assessment requires moving beyond Mean Opinion Scores (MOS) to analyze underlying perceptual dimensions. To address this, we introduce a novel post-training method that tailors the foundational…

Reinforcement Learning