Self Supervised Scanpath Prediction Framework for Painting Images
In our paper, we propose a novel strategy to learn distortion invariant latent representation from painting pictures for visual attention modelling downstream task. In further detail, we design an unsupervised framework that jointly maximises the mutual information over different painting styles. To show the effectiveness of our approach, we firstly propose a lightweight scanpath baseline model and compare its performance to some state-of-the-art methods. Secondly, we train the encoder of our baseline model on large-scale painting images to study the efficiency of the proposed self-supervised strategy. The lightweight decoder proves effective in learning from the self-supervised pre-trained encoder with better performances than the end-to-end fine-tuned supervised baseline on two painting datasets, including a proposed new visual attention modelling dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderPredictionScanpath predictionSimilar Papers 제목 키워드 기반
SPGen: Stochastic scanpath generation for paintings using unsupervised domain adaptation
Understanding human visual attention is key to preserving cultural heritage We introduce SPGen a novel deep learning model to predict scanpaths the sequence of eye movementswhen viewers observe paintings. Our architectur…
Unsupervised Domain AdaptationA domain adaptive deep learning solution for scanpath prediction of paintings
Cultural heritage understanding and preservation is an important issue for society as it represents a fundamental aspect of its identity. Paintings represent a significant part of cultural heritage, and are the subject o…
Scanpath predictionPathGAN: Visual Scanpath Prediction with Generative Adversarial Networks
We introduce PathGAN, a deep neural network for visual scanpath prediction trained on adversarial examples. A visual scanpath is defined as the sequence of fixation points over an image defined by a human observer with i…
PredictionScanpath predictionA Probabilistic Time-Evolving Approach to Scanpath Prediction
Human visual attention is a complex phenomenon that has been studied for decades. Within it, the particular problem of scanpath prediction poses a challenge, particularly due to the inter- and intra-observer variability,…
Dynamic Time WarpingPredictionScanpath predictionPathformer3D: A 3D Scanpath Transformer for 360° Images
Scanpath prediction in 360{\deg} images can help realize rapid rendering and better user interaction in Virtual/Augmented Reality applications. However, existing scanpath prediction models for 360{\deg} images execute sc…
DecoderPredictionScanpath prediction