AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging
We introduce an open-source web-based data annotation framework (AlpacaTag) for sequence tagging tasks such as named-entity recognition (NER). The distinctive advantages of AlpacaTag are three-fold. 1) Active intelligent recommendation: dynamically suggesting annotations and sampling the most informative unlabeled instances with a back-end active learned model; 2) Automatic crowd consolidation: enhancing real-time inter-annotator agreement by merging inconsistent labels from multiple annotators; 3) Real-time model deployment: users can deploy their models in downstream systems while new annotations are being made. AlpacaTag is a comprehensive solution for sequence labeling tasks, ranging from rapid tagging with recommendations powered by active learning and auto-consolidation of crowd annotations to real-time model deployment.
Code (0)
등록된 구현이 없습니다.
Tasks
Active Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERSimilar Papers 제목 키워드 기반
A Bayesian Approach for Sequence Tagging with Crowds
Current methods for sequence tagging, a core task in NLP, are data hungry, which motivates the use of crowdsourcing as a cheap way to obtain labelled data. However, annotators are often unreliable and current aggregation…
Active LearningArgument Miningnamed-entity-recognitionNamed Entity Recognition+1Modeling sequential annotations for sequence labeling with crowds
Crowd sequential annotations can be an efficient and cost-effective way to build large datasets for sequence labeling. Different from tagging independent instances, for crowd sequential annotations the quality of label s…
validLearning from Measurements in Crowdsourcing Models: Inferring Ground Truth from Diverse Annotation Types
Annotated corpora enable supervised machine learning and data analysis. To reduce the cost of manual annotation, tasks are often assigned to internet workers whose judgments are reconciled by crowdsourcing models. We app…
Leveraging Crowdsourcing Data For Deep Active Learning - An Application: Learning Intents in Alexa
This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesian deep learning, and extends existing wo…
Active LearningDeep Learningintent-classificationIntent ClassificationAggregating and Predicting Sequence Labels from Crowd Annotations
Despite sequences being core to NLP, scant work has considered how to handle noisy sequence labels from multiple annotators for the same text. Given such annotations, we consider two complementary tasks: (1) aggregating …
Articlesnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+1