Enhancing Facial Consistency in Conditional Video Generation via Facial Landmark Transformation
Landmark-guided character animation generation is an important field. Generating character animations with facial features consistent with a reference image remains a significant challenge in conditional video generation, especially involving complex motions like dancing. Existing methods often fail to maintain facial feature consistency due to mismatches between the facial landmarks extracted from source videos and the target facial features in the reference image. To address this problem, we propose a facial landmark transformation method based on the 3D Morphable Model (3DMM). We obtain transformed landmarks that align with the target facial features by reconstructing 3D faces from the source landmarks and adjusting the 3DMM parameters to match the reference image. Our method improves the facial consistency between the generated videos and the reference images, effectively improving the facial feature mismatch problem.
Code (0)
등록된 구현이 없습니다.
Tasks
Video GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FacEnhance: Facial Expression Enhancing with Recurrent DDPMs
Facial expressions, vital in non-verbal human communication, have found applications in various computer vision fields like virtual reality, gaming, and emotional AI assistants. Despite advancements, many facial expressi…
Computational EfficiencyDenoisingFacial expression generationFAAC: Facial Animation Generation with Anchor Frame and Conditional Control for Superior Fidelity and Editability
Over recent years, diffusion models have facilitated significant advancements in video generation. Yet, the creation of face-related videos still confronts issues such as low facial fidelity, lack of frame consistency, l…
Face ModelVideo GenerationSelf-Supervised 3D Face Reconstruction via Conditional Estimation
We present a conditional estimation (CEST) framework to learn 3D facial parameters from 2D single-view images by self-supervised training from videos. CEST is based on the process of analysis by synthesis, where the 3D f…
3D Face ReconstructionDisentanglementFace ReconstructionRefDecoder: Enhancing Visual Generation with Conditional Video Decoding
Video generation powers a vast array of downstream applications. However, while the de facto standard, i.e., latent diffusion models, typically employ heavily conditioned denoising networks, their decoders often remain u…
Video GenerationStyle TransferT2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design
In this paper, we focus on enhancing a diffusion-based text-to-video (T2V) model during the post-training phase by distilling a highly capable consistency model from a pretrained T2V model. Our proposed method, T2V-Turbo…
Video AlignmentVideo Generation