ALLSH: Active Learning Guided by Local Sensitivity and Hardness
Active learning, which effectively collects informative unlabeled data for annotation, reduces the demand for labeled data. In this work, we propose to retrieve unlabeled samples with a local sensitivity and hardness-aware acquisition function. The proposed method generates data copies through local perturbations and selects data points whose predictive likelihoods diverge the most from their copies. We further empower our acquisition function by injecting the select-worst case perturbation. Our method achieves consistent gains over the commonly used active learning strategies in various classification tasks. Furthermore, we observe consistent improvements over the baselines on the study of prompt selection in prompt-based few-shot learning. These experiments demonstrate that our acquisition guided by local sensitivity and hardness can be effective and beneficial for many NLP tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningFew-Shot LearningSensitivitySimilar Papers 제목 키워드 기반
AllShowers: One model for all calorimeter showers
Accurate and efficient detector simulation is essential for modern collider experiments. To reduce the high computational cost, various fast machine learning surrogate models have been proposed. Traditional surrogate mod…
Sheaf-Laplacian Obstruction and Projection Hardness for Cross-Modal Compatibility on a Modality-Independent Site
Cross-modal representations vary in how easily they can be aligned, and compatibility is generally non-transitive: two modalities may align through an intermediate modality at lower complexity than through a direct map. …
Statistical guided-waves-based SHM via stochastic non-parametric time series models
Damage detection in active-sensing, guided-waves-based Structural Health Monitoring (SHM) has evolved through multiple eras of development during the past decades. Nevertheless, there still exists a number of challenges …
Decision MakingSensitivityStructural Health MonitoringTime Series+1Investigating Active Sampling for Hardness Classification with Vision-Based Tactile Sensors
One of the most important object properties that humans and robots perceive through touch is hardness. This paper investigates information-theoretic active sampling strategies for sample-efficient hardness classification…
Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios
Domain adaptation is an effective solution to data scarcity in low-resource scenarios. However, when applied to token-level tasks such as bioNER, domain adaptation methods often suffer from the challenging linguistic cha…
Domain Adaptation