Cold-start Active Learning through Self-supervised Language Modeling
Active learning strives to reduce annotation costs by choosing the most critical examples to label. Typically, the active learning strategy is contingent on the classification model. For instance, uncertainty sampling depends on poorly calibrated model confidence scores. In the cold-start setting, active learning is impractical because of model instability and data scarcity. Fortunately, modern NLP provides an additional source of information: pre-trained language models. The pre-training loss can find examples that surprise the model and should be labeled for efficient fine-tuning. Therefore, we treat the language modeling loss as a proxy for classification uncertainty. With BERT, we develop a simple strategy based on the masked language modeling loss that minimizes labeling costs for text classification. Compared to other baselines, our approach reaches higher accuracy within less sampling iterations and computation time.
Code (1)
Tasks
Active LearningClassificationGeneral ClassificationLanguage ModelingLanguage ModellingMasked Language Modelingtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Cold-Start Active Preference Learning in Socio-Economic Domains
Active preference learning offers an efficient approach to modeling preferences, but it is hindered by the cold-start problem, which leads to a marked decline in performance when no initial labeled data are available. Wh…
Active LearningFoundation Model Makes Clustering A Better Initialization For Cold-Start Active Learning
Active learning selects the most informative samples from the unlabelled dataset to annotate in the context of a limited annotation budget. While numerous methods have been proposed for subsequent sample selection based …
Active LearningClusteringimage-classificationImage ClassificationDataset-Aware Cold-Start Active Learning for Annotation-Efficient 3D Medical Image Segmentation
Deep learning for 3D medical image segmentation requires extensive manual annotations, a major bottleneck in volumetric medical imaging. Active learning aims to reduce this burden by selecting informative samples for ann…
Medical Image SegmentationActive LearningSelf-supervised Assisted Active Learning for Skin Lesion Segmentation
Label scarcity has been a long-standing issue for biomedical image segmentation, due to high annotation costs and professional requirements. Recently, active learning (AL) strategies strive to reduce annotation costs by …
Active LearningDiversityImage SegmentationLesion Segmentation+4Cold-Start Active Correlation Clustering
We study active correlation clustering where pairwise similarities are not provided upfront and must be queried in a cost-efficient manner through active learning. Specifically, we focus on the cold-start scenario, where…
Active Learning