Papers Active Speaker Detection
“Active Speaker Detection” 태그가 달린 논문 66편 · 필터 해제
$C^3$ASD: Multi-Level Consistency-Driven Representation Learning
Active Speaker Detection determines whether a visible person in a video is speaking at each moment. While recent audio-visual fusion methods perform well on clean data, they degrade under real-world corruptions such as b…
Active Speaker DetectionRepresentation LearningKnowledge DistillationContrastive LearningGateFusion: Hierarchical Gated Cross-Modal Fusion for Active Speaker Detection
Active Speaker Detection (ASD) aims to identify who is currently speaking in each frame of a video. Most state-of-the-art approaches rely on late fusion to combine visual and audio features, but late fusion often fails t…
Active Speaker DetectionReal-Time Audio-Visual Speech Enhancement Using Pre-trained Visual Representations
Speech enhancement in audio-only settings remains challenging, particularly in the presence of interfering speakers. This paper presents a simple yet effective real-time audio-visual speech enhancement (AVSE) system, RAV…
Audio-Visual Speech RecognitionActive Speaker DetectionSpeech EnhancementUniTalk: Towards Universal Active Speaker Detection in Real World Scenarios
We present UniTalk, a novel dataset specifically designed for the task of active speaker detection, emphasizing challenging scenarios to enhance model generalization. Unlike previously established benchmarks such as AVA,…
Active Speaker DetectionCoGenAV: Versatile Audio-Visual Representation Learning via Contrastive-Generative Synchronization
The inherent synchronization between a speaker's lip movements, voice, and the underlying linguistic content offers a rich source of information for improving speech processing tasks, especially in challenging conditions…
Active Speaker DetectionAudio-Visual Speech RecognitionAudio-Visual SynchronizationRepresentation Learning+4Understanding Co-speech Gestures in-the-wild
Co-speech gestures play a vital role in non-verbal communication. In this paper, we introduce a new framework for co-speech gesture understanding in the wild. Specifically, we propose three new tasks and benchmarks to ev…
Active Speaker DetectionLASER: Lip Landmark Assisted Speaker Detection for Robustness
Active Speaker Detection (ASD) aims to identify speaking individuals in complex visual scenes. While humans can easily detect speech by matching lip movements to audio, current ASD models struggle to establish this corre…
Active Speaker DetectionAudio-Visual Active Speaker DetectionASDnB: Merging Face with Body Cues For Robust Active Speaker Detection
State-of-the-art Active Speaker Detection (ASD) approaches mainly use audio and facial features as input. However, the main hypothesis in this paper is that body dynamics is also highly correlated to "speaking" (and "lis…
Active Speaker DetectionFeature ImportanceBIAS: A Body-based Interpretable Active Speaker Approach
State-of-the-art Active Speaker Detection (ASD) approaches heavily rely on audio and facial features to perform, which is not a sustainable approach in wild scenarios. Although these methods achieve good results in the s…
Active Speaker DetectionFeature ImportanceHow to Squeeze An Explanation Out of Your Model
Deep learning models are widely used nowadays for their reliability in performing various tasks. However, they do not typically provide the reasoning behind their decision, which is a significant drawback, particularly f…
Active Speaker DetectionFabuLight-ASD: Unveiling Speech Activity via Body Language
Active speaker detection (ASD) in multimodal environments is crucial for various applications, from video conferencing to human-robot interaction. This paper introduces FabuLight-ASD, an advanced ASD model that integrate…
Active Speaker DetectionAn Efficient and Streaming Audio Visual Active Speaker Detection System
This paper delves into the challenging task of Active Speaker Detection (ASD), where the system needs to determine in real-time whether a person is speaking or not in a series of video frames. While previous works have m…
Active Speaker DetectionAudio-Visual Active Speaker DetectionCPUImitation of human motion achieves natural head movements for humanoid robots in an active-speaker detection task
Head movements are crucial for social human-human interaction. They can transmit important cues (e.g., joint attention, speaker detection) that cannot be achieved with verbal interaction alone. This advantage also holds …
Active Speaker DetectionAudio-Visual Talker Localization in Video for Spatial Sound Reproduction
Object-based audio production requires the positional metadata to be defined for each point-source object, including the key elements in the foreground of the sound scene. In many media production use cases, both cameras…
Active Speaker DetectionEnhancing Real-World Active Speaker Detection with Multi-Modal Extraction Pre-Training
Audio-visual active speaker detection (AV-ASD) aims to identify which visible face is speaking in a scene with one or more persons. Most existing AV-ASD methods prioritize capturing speech-lip correspondence. However, th…
Active Speaker DetectionAudio-Visual Active Speaker DetectionDenoisingTarget Speaker ExtractionRobust Active Speaker Detection in Noisy Environments
This paper addresses the issue of active speaker detection (ASD) in noisy environments and formulates a robust active speaker detection (rASD) problem. Existing ASD approaches leverage both audio and visual modalities, b…
Active Speaker DetectionSpeech SeparationAnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech Technologies
More than 7,000 known languages are spoken around the world. However, due to the lack of annotated resources, only a small fraction of them are currently covered by speech technologies. Albeit self-supervised speech repr…
Active Speaker DetectionLeveraging Visual Supervision for Array-based Active Speaker Detection and Localization
Conventional audio-visual approaches for active speaker detection (ASD) typically rely on visually pre-extracted face tracks and the corresponding single-channel audio to find the speaker in a video. Therefore, they tend…
Active Speaker DetectionSelf-Supervised LearningGestSync: Determining who is speaking without a talking head
In this paper we introduce a new synchronisation task, Gesture-Sync: determining if a person's gestures are correlated with their speech or not. In comparison to Lip-Sync, Gesture-Sync is far more challenging as there is…
Active Speaker DetectionGesture SynchronizationSelf-Supervised LearningAudio-visual child-adult speaker classification in dyadic interactions
Interactions involving children span a wide range of important domains from learning to clinical diagnostic and therapeutic contexts. Automated analyses of such interactions are motivated by the need to seek accurate ins…
Active Speaker DetectionClassificationDiagnostic