paper-with-me

홈 › Papers

Learning from Noisy Labels with Distillation

2017-03-07 · ICCV 2017 10 · Yuncheng Li, Jianchao Yang, Yale Song, Liangliang Cao, Jiebo Luo, Li-Jia Li

The ability of learning from noisy labels is very useful in many visual recognition tasks, as a vast amount of data with noisy labels are relatively easy to obtain. Traditionally, the label noises have been treated as statistical outliers, and approaches such as importance re-weighting and bootstrap have been proposed to alleviate the problem. According to our observation, the real-world noisy labels exhibit multi-mode characteristics as the true labels, rather than behaving like independent random outliers. In this work, we propose a unified distillation framework to use side information, including a small clean dataset and label relations in knowledge graph, to "hedge the risk" of learning from noisy labels. Furthermore, unlike the traditional approaches evaluated based on simulated label noises, we propose a suite of new benchmark datasets, in Sports, Species and Artifacts domains, to evaluate the task of learning from noisy labels in the practical setting. The empirical study demonstrates the effectiveness of our proposed method in all the domains.

📄 PDF Abstract BibTeX arXiv:1703.02391

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Federated Learning with Extremely Noisy Clients via Negative Distillation

2023-12-20 · Yang Lu, Lin Chen, Yonggang Zhang, Yiliang Zhang 외

Federated learning (FL) has shown remarkable success in cooperatively training deep models, while typically struggling with noisy labels. Advanced works propose to tackle label noise by a re-weighting strategy with a str…

Federated LearningKnowledge Distillation

FlyKD: Graph Knowledge Distillation on the Fly with Curriculum Learning

2024-03-16 · Eugene Ku

Knowledge Distillation (KD) aims to transfer a more capable teacher model's knowledge to a lighter student model in order to improve the efficiency of the model, making it faster and more deployable. However, the student…

Knowledge Distillation

Blind Knowledge Distillation for Robust Image Classification

2022-11-21 · Timo Kaiser, Lukas Ehmann, Christoph Reinders, Bodo Rosenhahn

Optimizing neural networks with noisy labels is a challenging task, especially if the label set contains real-world noise. Networks tend to generalize to reasonable patterns in the early training stages and overfit to sp…

Classificationimage-classificationImage ClassificationKnowledge Distillation+1

Knowledge Distillation with Noisy Labels for Natural Language Understanding

2021-09-21 · WNUT (ACL) 2021 11 · Shivendra Bhardwaj, Abbas Ghaddar, Ahmad Rashid, Khalil Bibi 외

Knowledge Distillation (KD) is extensively used to compress and deploy large pre-trained language models on edge devices for real-world applications. However, one neglected area of research is the impact of noisy (corrup…

Knowledge DistillationNatural Language Understanding

Relation Modeling and Distillation for Learning with Noisy Labels

2024-05-30 · Xiaming Che, Junlin Zhang, Zhuang Qi, Xin Qi

Learning with noisy labels has become an effective strategy for enhancing the robustness of models, which enables models to better tolerate inaccurate data. Existing methods either focus on optimizing the loss function t…

Contrastive LearningKnowledge DistillationLearning with noisy labelsRelation+2