Automatic Video Dubbing at AppTek
Video dubbing is the activity of revoicing a video while offering a viewing experience equivalent to the original video. The revoicing usually comes with a changed script, mostly in a different language, and the revoicing should reproduce the original emotions, coherent with the body language, and lip synchronized. In this project, we aim to build an AD system in three phases: (1) voice-over; (2) emotional voice-over; (3) full dubbing, while enhancing the system with human-in-the-loop capabilities for a higher quality.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbing
We investigate how humans perform the task of dubbing video content from one language into another, leveraging a novel corpus of 319.57 hours of video from 54 professionally produced titles. This is the first such large-…
TranslationProsodic Alignment for off-screen automatic dubbing
The goal of automatic dubbing is to perform speech-to-speech translation while achieving audiovisual coherence. This entails isochrony, i.e., translating the original speech by also matching its prosodic structure into p…
Speech-to-Speech TranslationTranslationSyncVoice: Simple and Effective Automatic Video Dubbing with Vision-Augmented TTS
Automatic video dubbing aims to generate high-fidelity speech that is temporally aligned with visual content. However, existing methods still suffer from limited speech naturalness, insufficient audio-visual synchronizat…
Speech SynthesisMachine Translation Verbosity Control for Automatic Dubbing
Automatic dubbing aims at seamlessly replacing the speech in a video document with synthetic speech in a different language. The task implies many challenges, one of which is generating translations that not only convey …
Machine TranslationTranslationMCDubber: Multimodal Context-Aware Expressive Video Dubbing
Automatic Video Dubbing (AVD) aims to take the given script and generate speech that aligns with lip motion and prosody expressiveness. Current AVD models mainly utilize visual information of the current sentence to enha…
Sentence