Deep semantic gaze embedding and scanpath comparison for expertise classification during OPT viewing
Modeling eye movement indicative of expertise behavior is decisive in user evaluation. However, it is indisputable that task semantics affect gaze behavior. We present a novel approach to gaze scanpath comparison that incorporates convolutional neural networks (CNN) to process scene information at the fixation level. Image patches linked to respective fixations are used as input for a CNN and the resulting feature vectors provide the temporal and spatial gaze information necessary for scanpath similarity comparison.We evaluated our proposed approach on gaze data from expert and novice dentists interpreting dental radiographs using a local alignment similarity score. Our approach was capable of distinguishing experts from novices with 93% accuracy while incorporating the image semantics. Moreover, our scanpath comparison using image patch features has the potential to incorporate task semantics from a variety of tasks
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationSimilar Papers 제목 키워드 기반
GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpaths
While exploring visual scenes, humans' scanpaths are driven by their underlying attention processes. Understanding visual scanpaths is essential for various applications. Traditional scanpath models predict the where and…
DecoderPredictionScanpath predictionWhat They Saw, Not Just Where They Looked: Semantic Scanpath Similarity via VLMs and NLP metric
Scanpath similarity metrics are central to eye-movement research, yet existing methods predominantly evaluate spatial and temporal alignment while neglecting semantic equivalence between attended image regions. We presen…
Semantic SimilarityPerceived Text Relevance Estimation Using Scanpaths and GNNs
A scanpath is an important concept in eye tracking that represents a person’s eye movements in a graph-like structure. Passive gaze-based interfaces, in which users do not consciously interact using their eyes, typically…
Pre-Trained Language Models Augmented with Synthetic Scanpaths for Natural Language Understanding
Human gaze data offer cognitive information that reflects natural language comprehension. Indeed, augmenting language models with human scanpaths has proven beneficial for a range of NLP tasks, including language underst…
Language ModelingLanguage ModellingNatural Language UnderstandingUnified Dynamic Scanpath Predictors Outperform Individually Trained Neural Models
Previous research on scanpath prediction has mainly focused on group models, disregarding the fact that the scanpaths and attentional behaviors of individuals are diverse. The disregard of these differences is especially…
Saliency PredictionScanpath prediction