Loss factorization, weakly supervised learning and label noise robustness

2016-02-08 · Giorgio Patrini, Frank Nielsen, Richard Nock, Marcello Carioni

We prove that the empirical risk of most well-known loss functions factors into a linear term aggregating all labels with a term that is label free, and can further be expressed by sums of the loss. This holds true even for non-smooth, non-convex losses and in any RKHS. The first term is a (kernel) mean operator --the focal quantity of this work-- which we characterize as the sufficient statistic for the labels. The result tightens known generalization bounds and sheds new light on their interpretation. Factorization has a direct application on weakly supervised learning. In particular, we demonstrate that algorithms like SGD and proximal methods can be adapted with minimal effort to handle weak supervision, once the mean operator has been estimated. We apply this idea to learning with asymmetric noisy labels, connecting and extending prior work. Furthermore, we show that most losses enjoy a data-dependent (by the mean operator) form of noise robustness, in contrast with known negative results.

📄 PDF Abstract BibTeX arXiv:1602.02450

Code (0)

등록된 구현이 없습니다.

Tasks

Generalization BoundsWeakly-supervised Learning

Methods 이 논문이 사용한 방법론

SGD Stochastic Gradient Descent is an iterative optimization technique that uses minibatches of data to form an expectation of the gradient, rather than the full gradient using…

Theoretical Guarantees of Deep Embedding Losses Under Label Noise

2018-12-06 · Nam Le, Jean-Marc Odobez

Collecting labeled data to train deep neural networks is costly and even impractical for many tasks. Thus, research effort has been focused in automatically curated datasets or unsupervised and weakly supervised learning…

Triplet Weakly-supervised Learning

TrustNet: Learning from Trusted Data Against (A)symmetric Label Noise

2020-07-13 · Amirmasoud Ghiassi, Taraneh Younesian, Robert Birke, Lydia Y. Chen

Robustness to label noise is a critical property for weakly-supervised classifiers trained on massive datasets. Robustness to label noise is a critical property for weakly-supervised classifiers trained on massive datase…

Self-training with noisy student model and semi-supervised loss function for dcase 2021 challenge task 4

2021-07-06 · Nam Kyun Kim, Hong Kook Kim

This report proposes a polyphonic sound event detection (SED) method for the DCASE 2021 Challenge Task 4. The proposed SED model consists of two stages: a mean-teacher model for providing target labels regarding weakly l…

Event Detection Sound Event Detection

Background-Aware Pooling and Noise-Aware Loss for Weakly-Supervised Semantic Segmentation

2021-04-02 · CVPR 2021 1 · Youngmin Oh, Beomjun Kim, Bumsub Ham

We address the problem of weakly-supervised semantic segmentation (WSSS) using bounding box annotations. Although object bounding boxes are good indicators to segment corresponding objects, they do not specify object bou…

Object Segmentation Semantic Segmentation Weakly supervised Semantic Segmentation+1

BoxTeacher: Exploring High-Quality Pseudo Labels for Weakly Supervised Instance Segmentation

2022-10-11 · CVPR 2023 1 · Tianheng Cheng, Xinggang Wang, Shaoyu Chen, Qian Zhang 외

Labeling objects with pixel-wise segmentation requires a huge amount of human labor compared to bounding boxes. Most existing methods for weakly supervised instance segmentation focus on designing heuristic losses with p…

Box-supervised Instance Segmentation Instance Segmentation Segmentation Semantic Segmentation+2