Decoding the visual attention of pathologists to reveal their level of expertise
We present a method for classifying the expertise of a pathologist based on how they allocated their attention during a cancer reading. We engage this decoding task by developing a novel method for predicting the attention of pathologists as they read whole-slide Images (WSIs) of prostate and make cancer grade classifications. Our ground truth measure of a pathologists' attention is the x, y and z (magnification) movement of their viewport as they navigated through WSIs during readings, and to date we have the attention behavior of 43 pathologists reading 123 WSIs. These data revealed that specialists have higher agreement in both their attention and cancer grades compared to general pathologists and residents, suggesting that sufficient information may exist in their attention behavior to classify their expertise level. To attempt this, we trained a transformer-based model to predict the visual attention heatmaps of resident, general, and specialist (GU) pathologists during Gleason grading. Based solely on a pathologist's attention during a reading, our model was able to predict their level of expertise with 75.3%, 56.1%, and 77.2% accuracy, respectively, better than chance and baseline models. Our model therefore enables a pathologist's expertise level to be easily and objectively evaluated, important for pathology training and competency assessment. Tools developed from our model could also be used to help pathology trainees learn how to read WSIs like an expert.
Code (0)
등록된 구현이 없습니다.
Tasks
whole slide imagesSimilar Papers 제목 키워드 기반
Visual attention analysis of pathologists examining whole slide images of Prostate cancer
We study the attention of pathologists as they examine whole-slide images (WSIs) of prostate cancer tissue using a digital microscope. To the best of our knowledge, our study is the first to report in detail how patholog…
Navigatewhole slide imagesMeasuring and Predicting Where and When Pathologists Focus their Visual Attention while Grading Whole Slide Images of Cancer
The ability to predict the attention of expert pathologists could lead to decision support systems for better pathology training. We developed methods to predict the spatio-temporal (where and when) movements of patholog…
Scanpath predictionWhen Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs
Multimodal large language models (MLLMs) have become a key interface for visual reasoning and grounded question answering, yet they remain vulnerable to visual hallucinations, where generated responses contradict image c…
Question AnsweringVisual ReasoningPathologist Attention-Aligned Report Generation for Prostate Histopathology
The allocation of visual attention by pathologists during cancer diagnosis is a highly selective process that critically shapes the information extracted from whole-slide images (WSIs). Human attention helps medical imag…
Visual Question AnsweringWSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answering
Whole slide imaging is routinely adopted for carcinoma diagnosis and prognosis. Abundant experience is required for pathologists to achieve accurate and reliable diagnostic results of whole slide images (WSI). The huge s…
DiagnosticGenerative Visual Question AnsweringPrognosisQuestion Answering+5