Image Understanding from Experts' Eyes by Modeling Perceptual Skill of Diagnostic Reasoning Processes
Eliciting and representing experts' remarkable perceptual capability of locating, identifying and categorizing objects in images specific to their domains of expertise will benefit image understanding in terms of transferring human domain knowledge and perceptual expertise into image-based computational procedures. In this paper, we present a hierarchical probabilistic framework to summarize the stereotypical and idiosyncratic eye movement patterns shared within 11 board-certified dermatologists while they are examining and diagnosing medical images. Each inferred eye movement pattern characterizes the similar temporal and spatial properties of its corresponding segments of the experts' eye movement sequences. We further discover a subset of distinctive eye movement patterns which are commonly exhibited across multiple images. Based on the combinations of the exhibitions of these eye movement patterns, we are able to categorize the images from the perspective of experts' viewing strategies. In each category, images share similar lesion distributions and configurations. The performance of our approach shows that modeling physicians' diagnostic viewing behaviors informs about medical images' understanding to correct diagnosis.
Code (0)
등록된 구현이 없습니다.
Tasks
DiagnosticSimilar Papers 제목 키워드 기반
EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecture
We propose EMMA, an efficient and unified architecture for multimodal understanding, generation and editing. Specifically, EMMA primarily consists of 1) An efficient autoencoder with a 32x compression ratio, which signif…
EyeSim-VQA: A Free-Energy-Guided Eye Simulation Framework for Video Quality Assessment
Free-energy-guided self-repair mechanisms have shown promising results in image quality assessment (IQA), but remain under-explored in video quality assessment (VQA), where temporal dynamics and model constraints pose un…
Image Quality AssessmentVideo Quality AssessmentVisual Question Answering (VQA)Measuring and modeling the perception of natural and unconstrained gaze in humans and machines
Humans are remarkably adept at interpreting the gaze direction of other individuals in their surroundings. This skill is at the core of the ability to engage in joint visual attention, which is essential for establishing…
Nonlinear Dynamics of Binocular Rivalry: A Comparative Study
When our eyes are presented with the same image, the brain processes it to view it as a single coherent one. The lateral shift in the position of our eyes, causes the two images to possess certain differences, which our …
F?D: On understanding the role of deep feature spaces on face generation evaluation
Perceptual metrics, like the Fr\'echet Inception Distance (FID), are widely used to assess the similarity between synthetically generated and ground truth (real) images. The key idea behind these metrics is to compute er…
counterfactualFace Generation