TORNADO: A Spatio-Temporal Convolutional Regression Network for Video Action Proposal
Given a video clip, action proposal aims to quickly generate a number of spatio-temporal tubes that enclose candidate human activities. Recently, the regression-based object detectors and long-term recurrent convolutional network (LRCN) have demonstrated superior performance in human action detection and recognition. However, the regression-based detectors performs inference without considering the temporal context among neighboring frames, and the LRCN using global visual percepts lacks the capability to capture local temporal dynamics. In this paper, we present a novel framework called TORNADO for human action proposal detection in un-trimmed video clips. Specifically, we propose a spatial-temporal convolutional network that combines the advantages of regression-based detector and LRCN by empowering Convolutional LSTM with regression capability. Our approach consists of a temporal convolutional regression network (T-CRN) and a spatial regression network (S-CRN) which are trained end-to-end on both RGB and OpticalFlow streams. They fuse appearance, motion and temporal contexts to regress the bounding boxes of candidate human actions simultaneously in 28 FPS. The action proposals are constructed by solving dynamic programming with peak trimming of the generated action boxes. Extensive experiments on the challenging UCF-101 and UCF-Sports datasets show that our method achieves superior performance as compared with the state-of-the-arts.
Code (0)
등록된 구현이 없습니다.
Tasks
Action DetectionregressionSimilar Papers 제목 키워드 기반
AgentCaster: Reasoning-Guided Tornado Forecasting
There is a growing need to evaluate Large Language Models (LLMs) on complex, high-impact, real-world tasks to assess their true readiness as reasoning agents. To address this gap, we introduce AgentCaster, a contaminatio…
Spatiotemporal Dilated Convolution with Uncertain Matching for Video-based Crowd Estimation
In this paper, we propose a novel SpatioTemporal convolutional Dense Network (STDNet) to address the video-based crowd counting problem, which contains the decomposition of 3D convolution and the 3D spatiotemporal dilate…
Crowd CountingInitialization Strategies of Spatio-Temporal Convolutional Neural Networks
We propose a new way of incorporating temporal information present in videos into Spatial Convolutional Neural Networks (ConvNets) trained on images, that avoids training Spatio-Temporal ConvNets from scratch. We describ…
STFCN: Spatio-Temporal FCN for Semantic Video Segmentation
This paper presents a novel method to involve both spatial and temporal features for semantic video segmentation. Current work on convolutional neural networks(CNNs) has shown that CNNs provide advanced spatial features …
SegmentationSemantic SegmentationVideo SegmentationVideo Semantic SegmentationA Novel Hybrid Approach for Tornado Prediction in the United States: Kalman-Convolutional BiLSTM with Multi-Head Attention
Tornadoes are among the most intense atmospheric vortex phenomena and pose significant challenges for detection and forecasting. Conventional methods, which heavily depend on ground-based observations and radar data, are…
State Estimation