Crowdsourced PAC Learning under Classification Noise
In this paper, we analyze PAC learnability from labels produced by crowdsourcing. In our setting, unlabeled examples are drawn from a distribution and labels are crowdsourced from workers who operate under classification noise, each with their own noise parameter. We develop an end-to-end crowdsourced PAC learning algorithm that takes unlabeled data points as input and outputs a trained classifier. Our three-step algorithm incorporates majority voting, pure-exploration bandits, and noisy-PAC learning. We prove several guarantees on the number of tasks labeled by workers for PAC learning in this setting and show that our algorithm improves upon the baseline by reducing the total number of tasks given to workers. We demonstrate the robustness of our algorithm by exploring its application to additional realistic crowdsourcing settings.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral ClassificationPAC learningSimilar Papers 제목 키워드 기반
Bayesian Detector Combination for Object Detection with Crowdsourced Annotations
Acquiring fine-grained object detection annotations in unconstrained images is time-consuming, expensive, and prone to noise, especially in crowdsourcing scenarios. Most prior object detection methods assume accurate ann…
Objectobject-detectionObject DetectionDetecting Label Errors by using Pre-Trained Language Models
We show that large pre-trained language models are inherently highly capable of identifying label errors in natural language datasets: simply examining out-of-sample data points in descending order of fine-tuned task los…
Learning Effective Embeddings From Crowdsourced Labels: An Educational Case Study
Learning representation has been proven to be helpful in numerous machine learning tasks. The success of the majority of existing representation learning approaches often requires a large amount of consistent and noise-f…
Representation LearningPractitioner-Centric Approach for Early Incident Detection Using Crowdsourced Data for Emergency Services
Emergency response is highly dependent on the time of incident reporting. Unfortunately, the traditional approach to receiving incident reports (e.g., calling 911 in the USA) has time delays. Crowdsourcing platforms such…
Event DetectionManagementTemporal LocalizationA Deep Learning Spatiotemporal Prediction Framework for Mobile Crowdsourced Services
This papers presents a deep learning-based framework to predict crowdsourced service availability spatially and temporally. A novel two-stage prediction model is introduced based on historical spatio-temporal traces of m…
Deep LearningGeneral ClassificationPredictionTime Series+1