Weakly Supervised Label Smoothing

2020-12-15 · Gustavo Penha, Claudia Hauff

We study Label Smoothing (LS), a widely used regularization technique, in the context of neural learning to rank (L2R) models. LS combines the ground-truth labels with a uniform distribution, encouraging the model to be less confident in its predictions. We analyze the relationship between the non-relevant documents-specifically how they are sampled-and the effectiveness of LS, discussing how LS can be capturing "hidden similarity knowledge" between the relevantand non-relevant document classes. We further analyze LS by testing if a curriculum-learning approach, i.e., starting with LS and after anumber of iterations using only ground-truth labels, is beneficial. Inspired by our investigation of LS in the context of neural L2R models, we propose a novel technique called Weakly Supervised Label Smoothing (WSLS) that takes advantage of the retrieval scores of the negative sampled documents as a weak supervision signal in the process of modifying the ground-truth labels. WSLS is simple to implement, requiring no modification to the neural ranker architecture. Our experiments across three retrieval tasks-passage retrieval, similar question retrieval and conversation response ranking-show that WSLS for pointwise BERT-based rankers leads to consistent effectiveness gains. The source code is available at https://anonymous.4open.science/r/dac85d48-6f71-4261-a7d8-040da6021c52/.

📄 PDF Abstract BibTeX arXiv:2012.08575

Code (1)

Guzpenha/transformer_rankers pytorch

Tasks

Learning-To-RankPassage Retrieval Retrieval

Methods 이 논문이 사용한 방법론

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Pseudo-labelling meets Label Smoothing for Noisy Partial Label Learning

2024-02-07 · Darshana Saravanan, Naresh Manwani, Vineet Gandhi

Partial label learning (PLL) is a weakly-supervised learning paradigm where each training instance is paired with a set of candidate labels (partial label), one of which is the true label. Noisy PLL (NPLL) relaxes this c…

Partial Label Learning Pseudo Label Weakly-supervised Learning

Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing

2020-07-21 · ECCV 2020 8 · Yapeng Tian, DIngzeyu Li, Chenliang Xu

In this paper, we introduce a new problem, named audio-visual video parsing, which aims to parse a video into temporal event segments and label them as either audible, visible, or both. Such a problem is essential for a …

Multiple Instance Learning

From Image-level to Pixel-level Labeling with Convolutional Networks

2014-11-23 · CVPR 2015 6 · Pedro O. Pinheiro, Ronan Collobert

We are interested in inferring object segmentation by leveraging only object class information, and by considering only minimal priors on the object segmentation task. This problem could be viewed as a kind of weakly sup…

Multiple Instance Learning Object Segmentation Semantic Segmentation+3

Few-shot brain segmentation from weakly labeled data with deep heteroscedastic multi-task networks

2019-04-04 · Richard McKinley, Michael Rebsamen, Raphael Meier, Mauricio Reyes 외

In applications of supervised learning applied to medical image segmentation, the need for large amounts of labeled data typically goes unquestioned. In particular, in the case of brain anatomy segmentation, hundreds or …

Anatomy Brain Segmentation Data Augmentation Image Segmentation+2

Audio Event and Scene Recognition: A Unified Approach using Strongly and Weakly Labeled Data

2016-11-12 · Anurag Kumar, Bhiksha Raj

In this paper we propose a novel learning framework called Supervised and Weakly Supervised Learning where the goal is to learn simultaneously from weakly and strongly labeled data. Strongly labeled data can be simply un…

Scene Recognition Weakly-supervised Learning