How Does a Neural Network's Architecture Impact Its Robustness to Noisy Labels?
Noisy labels are inevitable in large real-world datasets. In this work, we explore an area understudied by previous works -- how the network's architecture impacts its robustness to noisy labels. We provide a formal framework connecting the robustness of a network to the alignments between its architecture and target/noise functions. Our framework measures a network's robustness via the predictive power in its representations -- the test performance of a linear model trained on the learned representations using a small set of clean labels. We hypothesize that a network is more robust to noisy labels if its architecture is more aligned with the target function than the noise. To support our hypothesis, we provide both theoretical and empirical evidence across various neural network architectures and different domains. We also find that when the network is well-aligned with the target function, its predictive power in representations could improve upon state-of-the-art (SOTA) noisy-label-training methods in terms of test accuracy and even outperform sophisticated methods that use clean labels.
Code (0)
등록된 구현이 없습니다.
Tasks
Image ClassificationLearning with noisy labelsSimilar Papers 제목 키워드 기반
How does self-supervised pretraining improve robustness against noisy labels across various medical image classification datasets?
Noisy labels can significantly impact medical image classification, particularly in deep learning, by corrupting learned features. Self-supervised pretraining, which doesn't rely on labeled data, can enhance robustness a…
Contrastive Learningimage-classificationImage ClassificationMedical Image Classification+1Robust Feature Learning Against Noisy Labels
Supervised learning of deep neural networks heavily relies on large-scale datasets annotated by high-quality labels. In contrast, mislabeled samples can significantly degrade the generalization of models and result in me…
Local K-Similarity Constraint for Federated Learning with Label Noise
Federated learning on clients with noisy labels is a challenging problem, as such clients can infiltrate the global model, impacting the overall generalizability of the system. Existing methods proposed to handle noisy c…
Medical Image ClassificationFederated LearningOpen-set Label Noise Can Improve Robustness Against Inherent Label Noise
Learning with noisy labels is a practically challenging problem in weakly supervised learning. In the existing literature, open-set noises are always considered to be poisonous for generalization, similar to closed-set n…
Learning with noisy labelsOut-of-Distribution DetectionWeakly-supervised LearningLeveraging Unlabeled Data to Track Memorization
Deep neural networks may easily memorize noisy labels present in real-world data, which degrades their ability to generalize. It is therefore important to track and evaluate the robustness of models against noisy label m…
Memorization