A Comparative Study of Perceptual Quality Metrics for Audio-driven Talking Head Videos
The rapid advancement of Artificial Intelligence Generated Content (AIGC) technology has propelled audio-driven talking head generation, gaining considerable research attention for practical applications. However, performance evaluation research lags behind the development of talking head generation techniques. Existing literature relies on heuristic quantitative metrics without human validation, hindering accurate progress assessment. To address this gap, we collect talking head videos generated from four generative methods and conduct controlled psychophysical experiments on visual quality, lip-audio synchronization, and head movement naturalness. Our experiments validate consistency between model predictions and human annotations, identifying metrics that align better with human opinions than widely-used measures. We believe our work will facilitate performance evaluation and model development, providing insights into AIGC in a broader context. Code and data will be made available at https://github.com/zwx8981/ADTH-QA.
Code (1)
Tasks
Talking Head GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
From Aesthetics to Human Preferences: Comparative Perspectives of Evaluating Text-to-Music Systems
Evaluating generative models remains a fundamental challenge, particularly when the goal is to reflect human preferences. In this paper, we use music generation as a case study to investigate the gap between automatic ev…
Music GenerationEvaluating generative audio systems and their metrics
Recent years have seen considerable advances in audio synthesis with deep generative models. However, the state-of-the-art is very difficult to quantify; different studies often use different evaluation methodologies and…
Audio SynthesisWhat You Hear Is What You See: Audio Quality Metrics From Image Quality Metrics
In this study, we investigate the feasibility of utilizing state-of-the-art image perceptual metrics for evaluating audio signals by representing them as spectrograms. The encouraging outcome of the proposed approach is …
A Comparative Evaluation of Deep Learning Models for Speech Enhancement in Real-World Noisy Environments
Speech enhancement, particularly denoising, is vital in improving the intelligibility and quality of speech signals for real-world applications, especially in noisy environments. While prior research has introduced vario…
DenoisingSpeaker RecognitionSpeaker VerificationSpeech EnhancementData is Overrated: Perceptual Metrics Can Lead Learning in the Absence of Training Data
Perceptual metrics are traditionally used to evaluate the quality of natural signals, such as images and audio. They are designed to mimic the perceptual behaviour of human observers and usually reflect structures found …