Improving Video Compression With Deep Visual-Attention Models
Recent advances in deep learning have markedly improved the quality of visual-attention modelling. In this work we apply these advances to video compression. We propose a compression method that uses a saliency model to adaptively compress frame areas in accordance with their predicted saliency. We selected three state-of-the-art saliency models, adapted them for video compression and analyzed their results. The analysis includes objective evaluation of the models as well as objective and subjective evaluation of the compressed videos. Our method, which is based on the x264 video codec, can produce videos with the same visual quality as regular x264, but it reduces the bitrate by 25% according to the objective evaluation and by 17% according to the subjective one. Also, both the subjective and objective evaluations demonstrate that saliency models can compete with gaze maps for a single observer. Our method can extend to most video bitstream formats and can improve video compression quality without requiring a switch to a new video encoding standard.
Code (0)
등록된 구현이 없습니다.
Tasks
Video CompressionSimilar Papers 제목 키워드 기반
StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding
Multimodal large language models (MLLMs) have made significant progress in visual-language reasoning, but their ability to efficiently handle long videos remains limited. Despite recent advances in long-context MLLMs, st…
Video Question AnsweringLLaVA-MLB: Mitigating and Leveraging Attention Bias for Training-Free Video LLMs
Training-free video large language models (LLMs) leverage pretrained Image LLMs to process video content without the need for further training. A key challenge in such approaches is the difficulty of retaining essential …
Video UnderstandingVideo Compression Coding via Colorization: A Generative Adversarial Network (GAN)-Based Approach
Under the limited storage, computing and network bandwidth resources, the video compression coding technology plays an important role for visual communication. To efficiently compress raw video data, a colorization-based…
ColorizationDecoderGenerative Adversarial NetworkVideo CompressionUnified Spatiotemporal Token Compression for Video-LLMs at Ultra-Low Retention
Video large language models (Video-LLMs) face high computational costs due to large volumes of visual tokens. Existing token compression methods typically adopt a two-stage spatiotemporal compression strategy, relying on…
Semantic SimilarityQuestion AnsweringTowards Scalable Neural Representation for Diverse Videos
Implicit neural representations (INR) have gained increasing attention in representing 3D scenes and images, and have been recently applied to encode videos (e.g., NeRV, E-NeRV). While achieving promising results, existi…
Action RecognitionVideo Compression