A Benchmark Dataset and Saliency-guided Stacked Autoencoders for Video-based Salient Object Detection
Image-based salient object detection (SOD) has been extensively studied in the past decades. However, video-based SOD is much less explored since there lack large-scale video datasets within which salient objects are unambiguously defined and annotated. Toward this end, this paper proposes a video-based SOD dataset that consists of 200 videos (64 minutes). In constructing the dataset, we manually annotate all objects and regions over 7,650 uniformly sampled keyframes and collect the eye-tracking data of 23 subjects that free-view all videos. From the user data, we find salient objects in video can be defined as objects that consistently pop-out throughout the video, and objects with such attributes can be unambiguously annotated by combining manually annotated object/region masks with eye-tracking data of multiple subjects. To the best of our knowledge, it is currently the largest dataset for video-based salient object detection. Based on this dataset, this paper proposes an unsupervised baseline approach for video-based SOD by using saliency-guided stacked autoencoders. In the proposed approach, multiple spatiotemporal saliency cues are first extracted at pixel, superpixel and object levels. With these saliency cues, stacked autoencoders are unsupervisedly constructed which automatically infer a saliency score for each pixel by progressively encoding the high-dimensional saliency cues gathered from the pixel and its spatiotemporal neighbors. Experimental results show that the proposed unsupervised approach outperforms 30 state-of-the-art models on the proposed dataset, including 19 image-based & classic (unsupervised or non-deep learning), 6 image-based & deep learning, and 5 video-based & unsupervised. Moreover, benchmarking results show that the proposed dataset is very challenging and has the potential to boost the development of video-based SOD.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingObjectobject-detectionObject DetectionRGB Salient Object DetectionSalient Object DetectionSimilar Papers 제목 키워드 기반
Stacked Autoencoders for Medical Image Search
Medical images can be a valuable resource for reliable information to support medical diagnosis. However, the large volume of medical images makes it challenging to retrieve relevant information given a particular scenar…
Content-Based Image RetrievalImage RetrievalMedical DiagnosisRetrievalUC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders
In this paper, we propose the first framework (UCNet) to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection methods treat the saliency detection …
RGB-D Salient Object DetectionSaliency DetectionThermal Image SegmentationDISC: Deep Image Saliency Computing via Progressive Representation Learning
Salient object detection increasingly receives attention as an important component or step in several pattern recognition and image processing tasks. Although a variety of powerful saliency models have been intensively p…
object-detectionObject DetectionRepresentation LearningRGB Salient Object Detection+1Remote sensing framework for geological mapping via stacked autoencoders and clustering
Supervised machine learning methods for geological mapping via remote sensing face limitations due to the scarcity of accurately labelled training data that can be addressed by unsupervised learning, such as dimensionali…
ClusteringDimensionality ReductionTraining Stacked Denoising Autoencoders for Representation Learning
We implement stacked denoising autoencoders, a class of neural networks that are capable of learning powerful representations of high dimensional data. We describe stochastic gradient descent for unsupervised training of…
Denoisingimage-classificationImage ClassificationRepresentation Learning