Learning Video Saliency from Human Gaze Using Candidate Selection
During recent years remarkable progress has been made in visual saliency modeling. Our interest is in video saliency. Since videos are fundamentally different from still images, they are viewed differently by human observers. For example, the time each video frame is observed is a fraction of a second, while a still image can be viewed leisurely. Therefore, video saliency estimation methods should differ substantially from image saliency methods. In this paper we propose a novel method for video saliency estimation, which is inspired by the way people watch videos. We explicitly model the continuity of the video by predicting the saliency map of a given frame, conditioned on the map from the previous frame. Furthermore, accuracy and computation speed are improved by restricting the salient locations to a carefully selected candidate set. We validate our method using two gaze-tracked video datasets and show we outperform the state-of-the-art.
Code (0)
등록된 구현이 없습니다.
Tasks
Saliency PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Measuring the Importance of Temporal Features in Video Saliency
Where people look when watching videos is believed to be heavily influenced by temporal patterns. In this work, we test this assumption by quantifying to which extent gaze on recent video saliency benchmarks can be predi…
Infinite Gaze Generation for Videos with Autoregressive Diffusion
Predicting human gaze in video is fundamental to advancing scene understanding and multimodal interaction. While traditional saliency maps provide spatial probability distributions and scanpaths offer ordered fixations, …
Scene UnderstandingFollowing Gaze in Video
Following the gaze of people inside videos is an important signal for understanding people and their actions. In this paper, we present an approach for following gaze in video by predicting where a person (in the video) …
Gaze Prediction in Dynamic 360° Immersive Videos
This paper explores gaze prediction in dynamic $360^circ$ immersive videos, emph{i.e.}, based on the history scan path and VR contents, we predict where a viewer will look at an upcoming time. To tackle this problem, we …
Gaze PredictionPredictionImproving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts
The human prioritization of image regions can be modeled in a time invariant fashion with saliency maps or sequentially with scanpath models. However, while both types of models have steadily improved on several benchmar…
Decision MakingSequential Decision Making