Motion Representation Using Residual Frames with 3D CNN
Recently, 3D convolutional networks (3D ConvNets) yield good performance in action recognition. However, optical flow stream is still needed to ensure better performance, the cost of which is very high. In this paper, we propose a fast but effective way to extract motion features from videos utilizing residual frames as the input data in 3D ConvNets. By replacing traditional stacked RGB frames with residual ones, 35.6% and 26.6% points improvements over top-1 accuracy can be obtained on the UCF101 and HMDB51 datasets when ResNet-18 models are trained from scratch. And we achieved the state-of-the-art results in this training mode. Analysis shows that better motion features can be extracted using residual frames compared to RGB counterpart. By combining with a simple appearance path, our proposal can be even better than some methods using optical flow streams.
Code (3)
Tasks
Action RecognitionOptical Flow EstimationSimilar Papers 제목 키워드 기반
Residual Frames with Efficient Pseudo-3D CNN for Human Action Recognition
Human action recognition is regarded as a key cornerstone in domains such as surveillance or video understanding. Despite recent progress in the development of end-to-end solutions for video-based action recognition, ach…
Action RecognitionOptical Flow EstimationTemporal Action LocalizationVideo UnderstandingRethinking Motion Representation: Residual Frames with 3D ConvNets for Better Action Recognition
Recently, 3D convolutional networks yield good performance in action recognition. However, optical flow stream is still needed to ensure better performance, the cost of which is very high. In this paper, we propose a fas…
Action RecognitionOptical Flow EstimationGenerative Motion Infilling From Imprecisely Timed Keyframes
Keyframes are a standard representation for kinematic motion specification. Recent learned motion-inbetweening methods use keyframes as a way to control generative motion models, and are trained to generate life-like mot…
End-to-end Optimized Video Compression with MV-Residual Prediction
We present an end-to-end trainable framework for P-frame compression in this paper. A joint motion vector (MV) and residual prediction network MV-Residual is designed to extract the ensembled features of motion represent…
MS-SSIMSSIMVideo CompressionEnd-to-End Compressed Video Representation Learning for Generic Event Boundary Detection
Generic event boundary detection aims to localize the generic, taxonomy-free event boundaries that segment videos into chunks. Existing methods typically require video frames to be decoded before feeding into the network…
Boundary DetectionGeneric Event Boundary DetectionRepresentation Learning