ScanDMM: A Deep Markov Model of Scanpath Prediction for 360deg Images
Scanpath prediction for 360deg images aims to produce dynamic gaze behaviors based on the human visual perception mechanism. Most existing scanpath prediction methods for 360deg images do not give a complete treatment of the time-dependency when predicting human scanpath, resulting in inferior performance and poor generalizability. In this paper, we present a scanpath prediction method for 360deg images by designing a novel Deep Markov Model (DMM) architecture, namely ScanDMM. We propose a semantics-guided transition function to learn the nonlinear dynamics of time-dependent attentional landscape. Moreover, a state initialization strategy is proposed by considering the starting point of viewing, enabling the model to learn the dynamics with the correct "launcher". We further demonstrate that our model achieves state-of-the-art performance on four 360deg image databases, and exhibit its generalizability by presenting two applications of applying scanpath prediction models to other visual tasks - saliency detection and image quality assessment, expecting to provide profound insights into these fields.
Code (1)
Tasks
Image Quality AssessmentPredictionSaliency DetectionScanpath predictionSimilar Papers 제목 키워드 기반
Artificially Generated Visual Scanpath Improves Multi-label Thoracic Disease Classification in Chest X-Ray Images
Expert radiologists visually scan Chest X-Ray (CXR) images, sequentially fixating on anatomical structures to perform disease diagnosis. An automatic multi-label classifier of diseases in CXR images can benefit by incorp…
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONScanpath predictionThoracic Disease ClassificationPathformer3D: A 3D Scanpath Transformer for 360° Images
Scanpath prediction in 360{\deg} images can help realize rapid rendering and better user interaction in Virtual/Augmented Reality applications. However, existing scanpath prediction models for 360{\deg} images execute sc…
DecoderPredictionScanpath predictionPathGAN: Visual Scanpath Prediction with Generative Adversarial Networks
We introduce PathGAN, a deep neural network for visual scanpath prediction trained on adversarial examples. A visual scanpath is defined as the sequence of fixation points over an image defined by a human observer with i…
PredictionScanpath predictionState-of-the-Art in Human Scanpath Prediction
The last years have seen a surge in models predicting the scanpaths of fixations made by humans when viewing images. However, the field is lacking a principled comparison of those models with respect to their predictive …
BenchmarkingPredictionScanpath predictionSalyPath360: Saliency and Scanpath Prediction Framework for Omnidirectional Images
This paper introduces a new framework to predict visual attention of omnidirectional images. The key setup of our architecture is the simultaneous prediction of the saliency map and a corresponding scanpath for a given s…
DecoderPredictionScanpath prediction