paper-with-me

홈 › Papers

Decoding the visual attention of pathologists to reveal their level of expertise

2024-03-25 · Souradeep Chakraborty, Dana Perez, Paul Friedman, Natallia Sheuka, Constantin Friedman, Oksana Yaskiv, Rajarsi Gupta, Gregory J. Zelinsky, Joel H. Saltz, Dimitris Samaras

We present a method for classifying the expertise of a pathologist based on how they allocated their attention during a cancer reading. We engage this decoding task by developing a novel method for predicting the attention of pathologists as they read whole-slide Images (WSIs) of prostate and make cancer grade classifications. Our ground truth measure of a pathologists' attention is the x, y and z (magnification) movement of their viewport as they navigated through WSIs during readings, and to date we have the attention behavior of 43 pathologists reading 123 WSIs. These data revealed that specialists have higher agreement in both their attention and cancer grades compared to general pathologists and residents, suggesting that sufficient information may exist in their attention behavior to classify their expertise level. To attempt this, we trained a transformer-based model to predict the visual attention heatmaps of resident, general, and specialist (GU) pathologists during Gleason grading. Based solely on a pathologist's attention during a reading, our model was able to predict their level of expertise with 75.3%, 56.1%, and 77.2% accuracy, respectively, better than chance and baseline models. Our model therefore enables a pathologist's expertise level to be easily and objectively evaluated, important for pathology training and competency assessment. Tools developed from our model could also be used to help pathology trainees learn how to read WSIs like an expert.

📄 PDF Abstract BibTeX arXiv:2403.17255

Code (0)

등록된 구현이 없습니다.

Tasks

whole slide images

Similar Papers 제목 키워드 기반

Visual attention analysis of pathologists examining whole slide images of Prostate cancer

2022-02-17 · Souradeep Chakraborty, Ke Ma, Rajarsi Gupta, Beatrice Knudsen 외

We study the attention of pathologists as they examine whole-slide images (WSIs) of prostate cancer tissue using a digital microscope. To the best of our knowledge, our study is the first to report in detail how patholog…

Navigatewhole slide images

Measuring and Predicting Where and When Pathologists Focus their Visual Attention while Grading Whole Slide Images of Cancer

2025-08-03 · Souradeep Chakraborty, Ruoyu Xue, Rajarsi Gupta, Oksana Yaskiv 외 arxiv

The ability to predict the attention of expert pathologists could lead to decision support systems for better pathology training. We developed methods to predict the spatio-temporal (where and when) movements of patholog…

Scanpath prediction

When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs

2026-05-12 · Fanpu Cao, Xin Zou, Xuming Hu, Hui Xiong arxiv

Multimodal large language models (MLLMs) have become a key interface for visual reasoning and grounded question answering, yet they remain vulnerable to visual hallucinations, where generated responses contradict image c…

Question AnsweringVisual Reasoning

Pathologist Attention-Aligned Report Generation for Prostate Histopathology

2026-07-21 · Ruoyu Xue, Suryakant Singh, Souradeep Chakraborty, Pierre Marza 외 arxiv

The allocation of visual attention by pathologists during cancer diagnosis is a highly selective process that critically shapes the information extracted from whole-slide images (WSIs). Human attention helps medical imag…

Visual Question Answering

WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answering

2024-07-08 · Pingyi Chen, Chenglu Zhu, Sunyi Zheng, Honglin Li 외

Whole slide imaging is routinely adopted for carcinoma diagnosis and prognosis. Abundant experience is required for pathologists to achieve accurate and reliable diagnostic results of whole slide images (WSI). The huge s…

DiagnosticGenerative Visual Question AnsweringPrognosisQuestion Answering+5