Pose-Guided Sign Language Video GAN with Dynamic Lambda
We propose a novel approach for the synthesis of sign language videos using GANs. We extend the previous work of Stoll et al. by using the human semantic parser of the Soft-Gated Warping-GAN from to produce photorealistic videos guided by region-level spatial layouts. Synthesizing target poses improves performance on independent and contrasting signers. Therefore, we have evaluated our system with the highly heterogeneous MS-ASL dataset with over 200 signers resulting in a SSIM of 0.893. Furthermore, we introduce a periodic weighting approach to the generator that reactivates the training and leads to quantitatively better results.
Code (0)
등록된 구현이 없습니다.
Tasks
SSIMSimilar Papers 제목 키워드 기반
Learning Question-Guided Video Representation for Multi-Turn Video Question Answering
Understanding and conversing about dynamic scenes is one of the key capabilities of AI agents that navigate the environment and convey useful information to humans. Video question answering is a specific scenario of such…
NavigateQuestion AnsweringText GenerationVideo Question AnsweringAsymmetric Cross-Guided Attention Network for Actor and Action Video Segmentation From Natural Language Query
Actor and action video segmentation from natural language query aims to selectively segment the actor and its action in a video based on an input textual description. Previous works mostly focus on learning simple correl…
Referring Expression SegmentationSegmentationVideo SegmentationVideo Semantic SegmentationPhyVLLM: Physics-Guided Video Language Model with Motion-Appearance Disentanglement
Video Large Language Models (Video LLMs) have shown impressive performance across a wide range of video-language tasks. However, they often fail in scenarios requiring a deeper understanding of physical dynamics. This li…
Frame-Difference Guided Dynamic Region Perception for CLIP Adaptation in Text-Video Retrieval
With the rapid growth of video data, text-video retrieval technology has become increasingly important in numerous application scenarios such as recommendation and search. Early text-video retrieval methods suffer from t…
Video AlignmentVideo RetrievalRobust Egocentric Visual Attention Prediction Through Language-guided Scene Context-aware Learning
As the demand for analyzing egocentric videos grows, egocentric visual attention prediction, anticipating where a camera wearer will attend, has garnered increasing attention. However, it remains challenging due to the i…