Investigation on Combining 3D Convolution of Image Data and Optical Flow to Generate Temporal Action Proposals
In this paper, several variants of two-stream architectures for temporal action proposal generation in long, untrimmed videos are presented. Inspired by the recent advances in the field of human action recognition utilizing 3D convolutions in combination with two-stream networks and based on the Single-Stream Temporal Action Proposals (SST) architecture, four different two-stream architectures utilizing sequences of images on one stream and sequences of images of optical flow on the other stream are subsequently investigated. The four architectures fuse the two separate streams at different depths in the model; for each of them, a broad range of parameters is investigated systematically as well as an optimal parametrization is empirically determined. The experiments on the THUMOS'14 dataset show that all four two-stream architectures are able to outperform the original single-stream SST and achieve state of the art results. Additional experiments revealed that the improvements are not restricted to a single method of calculating optical flow by exchanging the formerly used method of Brox with FlowNet2 and still achieving improvements.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionOptical Flow EstimationTemporal Action LocalizationTemporal Action Proposal GenerationSimilar Papers 제목 키워드 기반
What Matters in Unsupervised Optical Flow
We systematically compare and analyze a set of key components in unsupervised optical flow to identify which photometric loss, occlusion handling, and smoothness regularization is most effective. Alongside this investiga…
Occlusion HandlingOptical Flow EstimationA qualitative investigation of optical flow algorithms for video denoising
A good optical flow estimation is crucial in many video analysis and restoration algorithms employed in application fields like media industry, industrial inspection and automotive. In this work, we investigate how well …
DenoisingOptical Flow EstimationVideo DenoisingOptical Flow Estimation using a Spatial Pyramid Network
We learn to compute optical flow by combining a classical spatial-pyramid formulation with deep learning. This estimates large motions in a coarse-to-fine approach by warping one image of a pair at each pyramid level by …
Dense Pixel Correspondence EstimationOptical Flow EstimationSemantic denoising autoencoders for retinal optical coherence tomography
Noise in speckle-prone optical coherence tomography tends to obfuscate important details necessary for medical diagnosis. In this paper, a denoising approach that preserves disease characteristics on retinal optical cohe…
DenoisingGeneral ClassificationMedical DiagnosisA Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
Recent work has shown that optical flow estimation can be formulated as a supervised learning task and can be successfully solved with convolutional networks. Training of the so-called FlowNet was enabled by a large synt…
Disparity EstimationOptical Flow EstimationScene Flow Estimation