Investigating the Robustness of Vision Transformers against Label Noise in Medical Image Classification
Label noise in medical image classification datasets significantly hampers the training of supervised deep learning methods, undermining their generalizability. The test performance of a model tends to decrease as the label noise rate increases. Over recent years, several methods have been proposed to mitigate the impact of label noise in medical image classification and enhance the robustness of the model. Predominantly, these works have employed CNN-based architectures as the backbone of their classifiers for feature extraction. However, in recent years, Vision Transformer (ViT)-based backbones have replaced CNNs, demonstrating improved performance and a greater ability to learn more generalizable features, especially when the dataset is large. Nevertheless, no prior work has rigorously investigated how transformer-based backbones handle the impact of label noise in medical image classification. In this paper, we investigate the architectural robustness of ViT against label noise and compare it to that of CNNs. We use two medical image classification datasets -- COVID-DU-Ex, and NCT-CRC-HE-100K -- both corrupted by injecting label noise at various rates. Additionally, we show that pretraining is crucial for ensuring ViT's improved robustness against label noise in supervised training.
Code (0)
등록된 구현이 없습니다.
Tasks
Classificationimage-classificationImage ClassificationMedical Image ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exploring the Robustness of In-Context Learning with Noisy Labels
Recently, the mysterious In-Context Learning (ICL) ability exhibited by Transformer architectures, especially in large language models (LLMs), has sparked significant research interest. However, the resilience of Transfo…
Data AugmentationIn-Context LearningLearning with noisy labelsMechanistic Understandings of Representation Vulnerabilities and Engineering Robust Vision Transformers
While transformer-based models dominate NLP and vision applications, their underlying mechanisms to map the input space to the label space semantically are not well understood. In this paper, we study the sources of know…
Zero-shot Generalization[Re] Improving Interpretation Faithfulness for Vision Transformers
This work aims to reproduce the results of Faithful Vision Transformers (FViTs) proposed by arXiv:2311.17983 alongside interpretability methods for Vision Transformers from arXiv:2012.09838 and Xu (2022) et al. We invest…
Curved Representation Space of Vision Transformers
Neural networks with self-attention (a.k.a. Transformers) like ViT and Swin have emerged as a better alternative to traditional convolutional neural networks (CNNs). However, our understanding of how the new architecture…
Exploring Adversarial Robustness of Vision Transformers in the Spectral Perspective
The Vision Transformer has emerged as a powerful tool for image classification tasks, surpassing the performance of convolutional neural networks (CNNs). Recently, many researchers have attempted to understand the robust…
Adversarial Robustnessimage-classificationImage Classification