paper-with-me

홈 › Papers

Crowdsourcing via Annotator Co-occurrence Imputation and Provable Symmetric Nonnegative Matrix Factorization

2021-06-14 · Shahana Ibrahim, Xiao Fu

Unsupervised learning of the Dawid-Skene (D&S) model from noisy, incomplete and crowdsourced annotations has been a long-standing challenge, and is a critical step towards reliably labeling massive data. A recent work takes a coupled nonnegative matrix factorization (CNMF) perspective, and shows appealing features: It ensures the identifiability of the D\&S model and enjoys low sample complexity, as only the estimates of the co-occurrences of annotator labels are involved. However, the identifiability holds only when certain somewhat restrictive conditions are met in the context of crowdsourcing. Optimizing the CNMF criterion is also costly -- and convergence assurances are elusive. This work recasts the pairwise co-occurrence based D&S model learning problem as a symmetric NMF (SymNMF) problem -- which offers enhanced identifiability relative to CNMF. In practice, the SymNMF model is often (largely) incomplete, due to the lack of co-labeled items by some annotators. Two lightweight algorithms are proposed for co-occurrence imputation. Then, a low-complexity shifted rectified linear unit (ReLU)-empowered SymNMF algorithm is proposed to identify the D&S model. Various performance characterizations (e.g., missing co-occurrence recoverability, stability, and convergence) and evaluations are also presented.

📄 PDF Abstract BibTeX arXiv:2106.07193

Code (0)

등록된 구현이 없습니다.

Tasks

Imputation

Similar Papers 제목 키워드 기반

Annotation Imputation to Individualize Predictions: Initial Studies on Distribution Dynamics and Model Predictions

2023-05-24 · London Lowmanstone, Ruyuan Wan, Risako Owan, Jaehyung Kim 외

Annotating data via crowdsourcing is time-consuming and expensive. Due to these costs, dataset creators often have each annotator label only a small subset of the data. This leads to sparse datasets with examples that ar…

Imputationvalid

Crowdsourcing via Pairwise Co-occurrences: Identifiability and Algorithms

2019-09-26 · NeurIPS 2019 12 · Shahana Ibrahim, Xiao Fu, Nikos Kargas, Kejun Huang

The data deluge comes with high demands for data labeling. Crowdsourcing (or, more generally, ensemble learning) techniques aim to produce accurate labels via integrating noisy, non-expert labeling from annotators. The c…

Ensemble Learning

The MASC Word Sense Corpus

2012-05-01 · LREC 2012 5 · Rebecca J. Passonneau, Collin F. Baker, Christiane Fellbaum, Nancy Ide

The MASC project has produced a multi-genre corpus with multiple layers of linguistic annotation, together with a sentence corpus containing WordNet 3.1 sense tags for 1000 occurrences of each of 100 words produced by mu…

Sentence

A Robust UCB Scheme for Active Learning in Regression from Strategic Crowds

2016-01-25 · Divya Padmanabhan, Satyanath Bhat, Dinesh Garg, Shirish Shevade 외

We study the problem of training an accurate linear regression model by procuring labels from multiple noisy crowd annotators, under a budget constraint. We propose a Bayesian model for linear regression in crowdsourcing…

Active Learningparameter estimationregressionVariational Inference

Identifying Chinese Opinion Expressions with Extremely-Noisy Crowdsourcing Annotations

2022-04-22 · ACL 2022 5 · Xin Zhang, Guangwei Xu, Yueheng Sun, Meishan Zhang 외

Recent works of opinion expression identification (OEI) rely heavily on the quality and scale of the manually-constructed training corpus, which could be extremely difficult to satisfy. Crowdsourcing is one practical sol…