GradMask: Reduce Overfitting by Regularizing Saliency
With too few samples or too many model parameters, overfitting can inhibit the ability to generalise predictions to new data. Within medical imaging, this can occur when features are incorrectly assigned importance such as distinct hospital specific artifacts, leading to poor performance on a new dataset from a different institution without those features, which is undesirable. Most regularization methods do not explicitly penalize the incorrect association of these features to the target class and hence fail to address this issue. We propose a regularization method, GradMask, which penalizes saliency maps inferred from the classifier gradients when they are not consistent with the lesion segmentation. This prevents non-tumor related features to contribute to the classification of unhealthy samples. We demonstrate that this method can improve test accuracy between 1-3% compared to the baseline without GradMask, showing that it has an impact on reducing overfitting.
Code (0)
등록된 구현이 없습니다.
Tasks
Lesion SegmentationSimilar Papers 제목 키워드 기반
GradMask: Effective Fine-tuning on Large-scale Pretrained Language Models via Gradient Masking
Pretrained language models have dominated a variety of NLP tasks. However, fine-tuning large pretrained models on downstream tasks tend to achieve degenerated and unstable results, especially when there are only a limite…
GradMask: Gradient-Guided Token Masking for Textual Adversarial Example Detection
We present a simple model-agnostic textual adversarial example detection scheme called GradMask. It uses gradient signals to detect adversarially perturbed tokens in an input sequence and occludes such tokens by a maskin…
Regularizing CNNs with Locally Constrained Decorrelations
Regularization is key for deep learning since it allows training more complex models while keeping lower levels of overfitting. However, the most prevalent regularizations do not leverage all the capacity of the models s…
DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks
Variants dropout methods have been designed for the fully-connected layer, convolutional layer and recurrent layer in neural networks, and shown to be effective to avoid overfitting. As an appealing alternative to recurr…
Regularizing Trajectories to Mitigate Catastrophic Forgetting
Regularization-based continual learning approaches generally prevent catastrophic forgetting by augmenting the training loss with an auxiliary objective. However in most practical optimization scenarios with noisy data a…
Continual Learning