WHICH SAMPLES SHOULD BE LEARNED FIRST：EASY OR HARD？
An effective weighting scheme for training samples is essential for learning tasks. Numerous weighting schemes have been proposed. Some schemes take the easy-first mode on samples, whereas some others take the hard-first mode. Naturally, an interesting yet realistic question is raised. Which samples should be learned first given a new learning task, easy or hard? To answer this question, three aspects of research are carried out. First, a high-level unified weighted loss is proposed, providing a more comprehensive view for existing schemes. Theoretical analysis is subsequently conducted and preliminary conclusions are obtained. Second, a flexible weighting scheme is proposed to overcome the defects of existing schemes. The three modes, namely, easy/medium/hard-first, can be flexibly switched in the proposed scheme. Third, a wide range of experiments are conducted to further compare the weighting schemes in different modes. On the basis of these works, reasonable answers are obtained. Factors including prior knowledge and data characteristics determine which samples should be learned first in a learning task.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Which Samples Should be Learned First: Easy or Hard?
An effective weighting scheme for training samples is essential for learning tasks. Numerous weighting schemes have been proposed. Some schemes take the easy-first mode, whereas some others take the hard-first one. Natur…
Curriculum Learning for Graph Neural Networks: Which Edges Should We Learn First
Graph Neural Networks (GNNs) have achieved great success in representing data with dependencies by recursively propagating and aggregating messages along the edges. However, edges in real-world graphs often have varying …
Robust Positive-Unlabeled Learning via Noise Negative Sample Self-correction
Learning from positive and unlabeled data is known as positive-unlabeled (PU) learning in literature and has attracted much attention in recent years. One common approach in PU learning is to sample a set of pseudo-negat…
Emphasis on Easy Samples for Distantly Supervised Relation Extraction
There are many wrongly-labeled samples and low-quality samples in automatically generated Distantly Supervised Relation Extraction datasets. Overfitting these samples leads to decline of generalization. To address this i…
RelationRelation ExtractionSentencek* Distribution: Evaluating the Latent Space of Deep Neural Networks using Local Neighborhood Analysis
Most examinations of neural networks' learned latent spaces typically employ dimensionality reduction techniques such as t-SNE or UMAP. These methods distort the local neighborhood in the visualization, making it hard to…
Dimensionality Reduction