Low Bandwidth Video-Chat Compression using Deep Generative Models
To unlock video chat for hundreds of millions of people hindered by poor connectivity or unaffordable data costs, we propose to authentically reconstruct faces on the receiver's device using facial landmarks extracted at the sender's side and transmitted over the network. In this context, we discuss and evaluate the benefits and disadvantages of several deep adversarial approaches. In particular, we explore quality and bandwidth trade-offs for approaches based on static landmarks, dynamic landmarks or segmentation maps. We design a mobile-compatible architecture based on the first order animation model of Siarohin et al. In addition, we leverage SPADE blocks to refine results in important areas such as the eyes and lips. We compress the networks down to about 3MB, allowing models to run in real time on iPhone 8 (CPU). This approach enables video calling at a few kbits per second, an order of magnitude lower than currently available alternatives.
Code (0)
등록된 구현이 없습니다.
Tasks
CPUMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Generative Video Compression: Towards 0.01% Compression Rate for Video Transmission
Whether a video can be compressed at an extreme compression rate as low as 0.01%? To this end, we achieve the compression rate as 0.02% at some cases by introducing Generative Video Compression (GVC), a new framework tha…
Pleno-Generation: A Scalable Generative Face Video Compression Framework with Bandwidth Intelligence
Generative model based compact video compression is typically operated within a relative narrow range of bitrates, and often with an emphasis on ultra-low rate applications. There has been an increasing consensus in the …
Video CompressionSparse2Dense: A Keypoint-driven Generative Framework for Human Video Compression and Vertex Prediction
For bandwidth-constrained multimedia applications, simultaneously achieving ultra-low bitrate human video compression and accurate vertex prediction remains a critical challenge, as it demands the harmonization of dynami…
Multi-Task LearningVideo GenerationVideo Compression Coding via Colorization: A Generative Adversarial Network (GAN)-Based Approach
Under the limited storage, computing and network bandwidth resources, the video compression coding technology plays an important role for visual communication. To efficiently compress raw video data, a colorization-based…
ColorizationDecoderGenerative Adversarial NetworkVideo CompressionRethinking Image Compression on the Web with Generative AI
The rapid growth of the Internet, driven by social media, web browsing, and video streaming, has made images central to the Web experience, resulting in significant data transfer and increased webpage sizes. Traditional …
Image Compression