Optical Flow and Mode Selection for Learning-based Video Coding
This paper introduces a new method for inter-frame coding based on two complementary autoencoders: MOFNet and CodecNet. MOFNet aims at computing and conveying the Optical Flow and a pixel-wise coding Mode selection. The optical flow is used to perform a prediction of the frame to code. The coding mode selection enables competition between direct copy of the prediction or transmission through CodecNet. The proposed coding scheme is assessed under the Challenge on Learned Image Compression 2020 (CLIC20) P-frame coding conditions, where it is shown to perform on par with the state-of-the-art video codec ITU/MPEG HEVC. Moreover, the possibility of copying the prediction enables to learn the optical flow in an end-to-end fashion i.e. without relying on pre-training and/or a dedicated loss term.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CompressionOptical Flow EstimationPredictionSimilar Papers 제목 키워드 기반
Predictive Coding Networks Meet Action Recognition
Action recognition is a key problem in computer vision that labels videos with a set of predefined actions. Capturing both, semantic content and motion, along the video frames is key to achieve high accuracy performance …
Action RecognitionOptical Flow EstimationMoVideo: Motion-Aware Video Generation with Diffusion Models
While recent years have witnessed great progress on using diffusion models for video generation, most of them are simple extensions of image generation frameworks, which fail to explicitly consider one of the key differe…
Image GenerationImage to Video GenerationOptical Flow EstimationVideo GenerationScale-Space Flow for End-to-End Optimized Video Compression
Despite considerable progress on end-to-end optimized deep networks for image compression, video coding remains a challenging task. Recently proposed methods for learned video compression use optical flow and bilinear wa…
Image CompressionMotion CompensationOptical Flow EstimationVideo CompressionMMVC: Learned Multi-Mode Video Compression with Block-based Prediction Mode Selection and Density-Adaptive Entropy Coding
Learning-based video compression has been extensively studied over the past years, but it still has limitations in adapting to various motion patterns and entropy models. In this paper, we propose multi-mode video compre…
BenchmarkingMS-SSIMOptical Flow EstimationPrediction+3Treating Motion as Option with Output Selection for Unsupervised Video Object Segmentation
Unsupervised video object segmentation (VOS) is a task that aims to detect the most salient object in a video without external guidance about the object. To leverage the property that salient objects usually have distinc…
ObjectOptical Flow EstimationSemantic SegmentationUnsupervised Video Object Segmentation+2