Papers Audio-Visual Synchronization
“Audio-Visual Synchronization” 태그가 달린 논문 32편 · 필터 해제
SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memory
The challenge of talking face generation from speech lies in aligning two different modal information, audio and video, such that the mouth region corresponds to input audio. Previous methods either exploit audio-visual …
Audio-Visual SynchronizationFace GenerationRepresentation LearningTalking Face GenerationMultimodal Transformer Distillation for Audio-Visual Synchronization
Audio-visual synchronization aims to determine whether the mouth movements and speech in the video are synchronized. VocaLiST reaches state-of-the-art performance by incorporating multimodal Transformers to model audio-v…
Audio-Visual SynchronizationSparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors
The objective of this paper is audio-visual synchronisation of general videos 'in the wild'. For such videos, the events that may be harnessed for synchronisation cues may be spatially small and may occur only infrequent…
Audio-Visual SynchronizationRethinking Audio-visual Synchronization for Active Speaker Detection
Active speaker detection (ASD) systems are important modules for analyzing multi-talker conversations. They aim to detect which speakers or none are talking in a visual scene at any given time. Existing research on ASD d…
Active Speaker DetectionAudio-Visual SynchronizationContrastive LearningVocaLiST: An Audio-Visual Synchronisation Model for Lips and Voices
In this paper, we address the problem of lip-voice synchronisation in videos containing human face and voice. Our approach is based on determining if the lips motion and the voice in a video are synchronised or not, depe…
Audio-Visual SynchronizationMusic Source SeparationNeural Pitch-Shifting and Time-Stretching with Controllable LPCNet
Modifying the pitch and timing of an audio signal are fundamental audio editing operations with applications in speech manipulation, audio-visual synchronization, and singing voice editing and synthesis. Thus far, method…
Audio-Visual SynchronizationLooking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separation
In this paper, we address the problem of separating individual speech signals from videos using audio-visual neural processing. Most conventional approaches utilize frame-wise matching criteria to extract shared informat…
Audio-Visual SynchronizationSpeech SeparationLook, Listen, and Attend: Co-Attention Network for Self-Supervised Audio-Visual Representation Learning
When watching videos, the occurrence of a visual event is often accompanied by an audio event, e.g., the voice of lip motion, the music of playing instruments. There is an underlying correlation between audio and visual …
Action RecognitionAudio-Visual SynchronizationRepresentation LearningSound Source LocalizationIdentity-Preserving Realistic Talking Face Generation
Speech-driven facial animation is useful for a variety of applications such as telepresence, chatbots, etc. The necessary attributes of having a realistic face animation are 1) audio-visual synchronization (2) identity p…
Audio-Visual SynchronizationFace GenerationImage ReconstructionTalking Face GenerationRealistic Speech-Driven Facial Animation with GANs
Speech-driven facial animation is the process that automatically synthesizes talking characters based on speech signals. The majority of work in this domain creates a mapping from audio features to visual features. This …
Audio-Visual SynchronizationLip ReadingOn Attention Modules for Audio-Visual Synchronization
With the development of media and networking technologies, multimedia applications ranging from feature presentation in a cinema setting to video on demand to interactive video conferencing are in great demand. Good sync…
Audio-Visual SynchronizationFaceDirector: Continuous Control of Facial Performance in Video
We present a method to continuously blend between multiple facial performances of an actor, which can contain different facial expressions or emotional states. As an example, given sad and angry video takes of a scene, o…
Audio-Visual Synchronizationcontinuous-controlContinuous Control