Can Learned Frame-Prediction Compete with Block-Motion Compensation for Video Coding?
Given recent advances in learned video prediction, we investigate whether a simple video codec using a pre-trained deep model for next frame prediction based on previously encoded/decoded frames without sending any motion side information can compete with standard video codecs based on block-motion compensation. Frame differences given learned frame predictions are encoded by a standard still-image (intra) codec. Experimental results show that the rate-distortion performance of the simple codec with symmetric complexity is on average better than that of x264 codec on 10 MPEG test videos, but does not yet reach the level of x265 codec. This result demonstrates the power of learned frame prediction (LFP), since unlike motion compensation, LFP does not use information from the current picture. The implications of training with L1, L2, or combined L2 and adversarial loss on prediction performance and compression efficiency are analyzed.
Code (0)
등록된 구현이 없습니다.
Tasks
Motion CompensationPredictionVideo PredictionSimilar Papers 제목 키워드 기반
Deep motion estimation for parallel inter-frame prediction in video compression
Standard video codecs rely on optical flow to guide inter-frame prediction: pixels from reference frames are moved via motion vectors to predict target video frames. We propose to learn binary motion codes that are encod…
Motion EstimationOptical Flow EstimationVideo CompressionDeep Learned Frame Prediction for Video Compression
Motion compensation is one of the most essential methods for any video compression algorithm. Video frame prediction is a task analogous to motion compensation. In recent years, the task of frame prediction is undertaken…
Motion CompensationPredictionVideo CompressionMMVC: Learned Multi-Mode Video Compression with Block-based Prediction Mode Selection and Density-Adaptive Entropy Coding
Learning-based video compression has been extensively studied over the past years, but it still has limitations in adapting to various motion patterns and entropy models. In this paper, we propose multi-mode video compre…
BenchmarkingMS-SSIMOptical Flow EstimationPrediction+3Self-Supervised Learning of Perceptually Optimized Block Motion Estimates for Video Compression
Block based motion estimation is integral to inter prediction processes performed in hybrid video codecs. Prevalent block matching based methods that are used to compute block motion vectors (MVs) rely on computationally…
Computational EfficiencyMotion EstimationMS-SSIMSelf-Supervised Learning+3DeepTAM: Deep Tracking and Mapping
We present a system for keyframe-based dense camera tracking and depth map estimation that is entirely learned. For tracking, we estimate small pose increments between the current camera image and a synthetic viewpoint. …
Depth EstimationDepth Prediction